Hey Folks, Chris Ramsdale, Lead Product Manager for App Engine here. This is definitely some great feedback. I’ve taken a stab at responding to some of the key points below [in bold]. Don’t hesitate to follow-up on this thread or reach out to me privately. Thanks…
=============================================================== Default way to handle 3rd party dependencies in an app - there are options and they work ok, but this is so common it would be nice to have an officially supported method. Especially since the common ones break on managed VMs because you don’t support appengine_config.py there. Going forward, for App Engine Managed VMs (soon to be v2), we’ll support native ways of specifying third party dependencies, e.g requirements.txt for Python developers, pom.xml for Java developers, some love for Node.js developers, etc. The overarching vision is to align App Engine with development practices that are common within the industry. Webapp2 - it’s OK, but definitely not great. From the outside there doesn’t seem to be a strong reason to be using something that has not gained broader traction when supporting something like Flask shouldn’t be hard. That would gain a much broader ecosystem rather than the very narrow, App Engine focused ecosystem around webapp2. I completely agree here, but do want to point out that App Engine supports Flask for Python developers. More info can be found here: https://github.com/GoogleCloudPlatform/appengine-python-flask-skeleton User service - when evaluating options things like the User services popped out to me. I thought, great, I’ll use a PaaS and things like user auth will be sanely handled. Except that it is so limited as to not be viable at all for a public product. The Users API is primarily focused on integration with Google Accounts. Its main use case is internal IT apps that are building on top of, or extending Google Apps. We are actively discussing whether to update it to work with OAuth2 or simply help users understand how to more easily add OAuth2 support to their apps. The biggest beginner problem I see right now is the architectural options around App Engine, Managed VMs, and GCE. On paper you guys have a nice blend of offerings that can allow developers to choose the right amount of control that they want / need. The reality, though, is that there are some hard choices to be made because of inconsistencies around what services are available on the different options and how to effectively wire together the components. A concrete example for me: I need to use numpy and some other non-pure python code in some background processing, so I needed to move off of App Engine for that work. I wasted quite a bit of time figuring out how / whether I could use the Datastore (via ndb) and task queues on the different options (including auth, network architecture, and scaling). None of it is rocket science, but I felt like I was inventing things when I knew that others had already come up with effective strategies. For me - the ideal situation would be if you just supported the same services through the same APIs everywhere, so I hope that is where you were headed. For starters, and off-the-cuff, I would recommend using App Engine Managed VMs for this use case. It allows you to use the Python bits that you want and preserves the network path back to the other services you mentioned (Datastore and Task Queues). Looking ahead though, the core services that are bundled into the App Engine SDK today (Datastore, Memcache, Task Queues, and Logging) will all be exposed as standalone services that one can call from any compute offering. Cloud Datastore is the initial step in this direction, and the team is actively working on resolving a few latency issues and adding NDB and Objectify support. GCP Roadmap - it would be nice to have some nice, clear roadmap on where things are headed. Right now, I restrict myself to only what is available and supported right now because I have no idea where you guys are headed. I’ve seen this tried several times before and ultimately it too becomes out of date. That’s not to say that we shouldn’t expose such information, I’m simply commenting on my hesitation to do so. We’ll take this feedback and have a follow-up discussion (or set of discussions) internally. Community - there is not a supportive / coherent community around GCP. Coming from a largely open source background this gives me a lot of culture shock. It would be great if there was at least a set of your engineers out engaging effectively in the various forums. Not just support - talking about best practices, explaining how things work, etc. It’s worth noting that our engineering team has been actively engaged with Beta programs that are run within App Engine. That said, I hear you on the broader point and will take this back to the Developer Relations within Google (in fact, Brad Abrams already has). Stack Overflow - the whole notion of pushing all of the questions from this mailing list to stack overflow is really off putting to me. I understand what you are trying to do but a) stack overflow seems to be where GCP questions go to be completely ignored and b) the way it’s done is pretty heavy handed. Why not at least post a link to the answer back to the list? I’ve always found mailing lists as an effective way to passively be aware of common questions and gain knowledge. Stack overflow is not effective in that role for me. I’ve heard other feedback on this point -- that the mailing list was too dense with conversation and hard to use as an effective tool in which to resolve technical questions. To which users found stack overflow to be more useful. This mailing list - honestly, I keep thinking that I should unsubscribe from this list because so many of the questions are very basic and they are generally just ignored. It’s kind of painful to watch - especially given that one volunteer is handling so much of this single handedly. I think it gives a terrible impression of GCP and makes me feel like very few experienced developers are using GCP. I feel compelled to point out that how this discussion going is a good example of some of the things that I - and it seems others - are frustrated about. You’ve asked for and received concrete feedback. Yet we’ve received no answers or discussion back from Google engineers. Based on these two points, and the one that precedes, it sounds like the requirement is: please provide a forum that is free, focused on technical questions, and has Google engineers actively engaged answering user’s questions. Does that sound correct? I just want to make sure that I’m interpreting the feedback correctly. I would like to have a Customer Support for all GCP products. I’m not sure that I follow here -- Google Cloud Support <https://cloud.google.com/support/> covers all Google Cloud Platform services. Does this actually dovetail with the request above, to provide free support across all GCP services? A lot of Karl's random thoughts and complaints ring true for me Just to clarify, Karl, your feedback is not random at all. It was well thought out, to the point, and clear. In fact, all the feedback is and we appreciate it. CI/CD set up, with dev/staging environments and one button deployments to production. It was a pretty long process of trial and error to achieve this. I would like to hear more about how you are achieving this and what your pain points are. Mind following-up off thread ([email protected]). A single page app with separate the web front end and the backend (endpoints) modules. This was also tricky since endpoints live behind /_ah which can't be routed away from the default application. I think separating these out and their build processes is healthy separation of concerns. Dan C (replied earlier) is most likely the Google contact that you want to reach out to. Integrating gulp build processes into GAE dev servers/build processes. In my case I'm using gradle app engine plugin. I’m not completely up-to-speed on gulp and would love to hear more. "ismorphic" javascript app, with server side rendering via something like react (I assume this would be a managed VM running node js) that speaks to endpoints, from both client side and the node js layer. That sounds correct and we’ll be enhancing support for Node.js over the coming months. For example the lack of java 8 on GAE is a concern of mine - https://code.google.com/p/googleappengine/issues/detail?id=9537 . There isn't any communication as to what the status of this is (note: AWS beanstalk supports java 8). I love the minimal configuration/maintenance of the GAE sandbox, but I need to know if a shift to managed VMs is the longer term direction for java support. It is not clear when starting a new java project if I should bet on GAE java sandbox being supported in the long term or just go with java 8 on a managed VM. For Java 8 support, you should be looking into Managed VMs, this is the hosting environment where we’ll rollout new runtimes. And...we’re actively porting over many of the features that give users the minimal configuration/maintenance. The existing App Engine sandbox isn’t going away. -- Chris Product Manager, Google Cloud Platform On Tue, Apr 21, 2015 at 2:33 PM, Karl MacMillan <[email protected]> wrote: > > On Apr 21, 2015, at 12:17 PM, Dan Ciruli <[email protected]> wrote: > > Hi, Karl - > > I'm Dan Ciruli, and I recently took over as Product Manager on Endpoints. > I really appreciate your feedback. My team is currently looking at > improvements that we’d like to make in the next version of Endpoints and > your comments jibe with what I’ve been hearing from a lot of our users. We > are working on both the developer experience as well as providing some > features that help you with managing your API (controlling access, etc). > > > You know - one thing that I would suggest is separating the serialization > features from the overall endpoints functionality. My hand-rolled solution > has to dig around in the guts of the ndb.Models (using > ndb.Model._properties) and I’m afraid you guys will break that since it’s > not a publicly documented API. > > What I would find useful is an API that gives you the ability to: > > 1. Introspect ndb.Models (and the equivalent for other runtimes) to find > just the datastore properties. > 2. Convert the property datatypes into JSON / Protobufs / whatever > 3. Helpers to selectively serialize entities and entity graphs > 4. Callbacks / transforms during serialization to handle API and data > model changes, friendly name changes, and things like automatically > fetching keys (which I do). > > With that I could easily turn an entity or entity graph into a serialized > format that I care about. I could then more easily build out my REST APIs > using whatever I wanted. You could, of course, build the rest of Cloud > Enpoints on that (and probably already have all of this functionality in > the internals), but for those of us that don’t find Endpoints to be a great > fit this functionality would really make life easier. > > Some other notes about Endpoints: > > 1. Discovery is just not that interesting to me for APIs that are > completely internal (i.e., only accessed by code that I control) and it > comes with additional complication and a pretty significant startup delay > for the default clients (from what I remember). Yeah, yeah, I know all of > the REST purist arguments about discovery, but it just isn’t worth it a lot > of times for me and I’m guessing others as well. Think about being able to > just decorate a single URL handler so that it handles serialization / > de-serialization, auth, etc. without having to muck around with discovery > and the client libraries (I would in a lot of cases just use a standard > HTTP API to call the APIs). > > 2. It would have helped immensely to document how the URLs would be > mapped. I had some trouble getting things running initially (can’t quite > remember what the problem was) and I was trying to just use curl or > something to figure out where the problem was. But I had to figure out what > all of the actual URLs were. I remember having trouble even figuring out > what the discovery url was, but that seems to be a little better documented > now ( > https://cloud.google.com/appengine/docs/python/endpoints/access_from_python > ). > > 3. Authorization - it would be nice if this wasn’t endpoints specific but > fit in with the rest of GAE (like the minimal authorizations that are > available now in app.yaml). And having a fully functional users API would > make it much better by having a standardized identity (presumably with > roles) that you could use in authorization statements. > > 4. Like I mentioned before, having separate types for requests and > responses is a basic requirement for me. > > I would be interested in a follow-up conversation with you -- send me an > email (my last name @google.com) and I’d like to set something up. > > > Great - I’ll drop you an email. > > Karl > > Thanks - > > Dan > On Monday, April 20, 2015 at 7:38:45 PM UTC-7, Alistair Burrowes wrote: >> >> Hi, >> >> I would echo a lot of what Karl said. >> >> I would like to see more examples of complex usage of GAE and or managed >> VMs. These are the kind of usages that more advanced developers might want. >> Here are a few examples of things that I have figured out or want: >> >> - CI/CD set up, with dev/staging environments and one button deployments >> to production. It was a pretty long process of trial and error to achieve >> this. >> >> - a single page app with separate the web front end and the backend >> (endpoints) modules. This was also tricky since endpoints live behind /_ah >> which can't be routed away from the default application. I think separating >> these out and their build processes is healthy separation of concerns. >> >> - integrating gulp build processes into GAE dev servers/build processes. >> In my case I'm using gradle app engine plugin. >> >> - "ismorphic" javascript app, with server side rendering via something >> like react (I assume this would be a managed VM running node js) that >> speaks to endpoints, from both client side and the node js layer. >> >> Also I agree that more transparency on the roadmap/discussions on >> direction would be really useful. >> >> For example the lack of java 8 on GAE is a concern of mine - >> https://code.google.com/p/googleappengine/issues/detail?id=9537. There >> isn't any communication as to what the status of this is (note: AWS >> beanstalk supports java 8). >> >> I love the minimal configuration/maintenance of the GAE sandbox, but I >> need to know if a shift to managed VMs is the longer term direction for >> java support. It is not clear when starting a new java project if I should >> bet on GAE java sandbox being supported in the long term or just go with >> java 8 on a managed VM. >> >> Other than this, I have found GAE/GCP to be fantastic and I am really >> happy with the different tools and quality of the libraries provided. >> >> On Thursday, April 16, 2015 at 3:37:31 AM UTC+10, Katie Ball (Google >> Cloud Support) wrote: >>> >>> Hi, >>> >>> My name is Katie, and I am on the Google Cloud Platform technical >>> support <http://cloud.google.com/support> team. >>> >>> This message is to Google Cloud Platform community members, especially >>> if you are newer to GCP. I would like to know what our team can do to help >>> you have a better and more enjoyable experience during the first days on >>> GCP. >>> >>> Did you need technical support? If so, I’d like to hear all about it. >>> >>> I’d also like to know: >>> >>> - >>> >>> What did you find most difficult about the first-time user >>> experience? >>> - >>> >>> Where did you get stuck? >>> >>> >>> Please reply to the group with your answers or any ideas you have on how >>> the technical support team can help new customers get familiar with GCP. >>> >>> And as a thank you for the great ideas, we will be giving away support >>> coupons worth $450 (equivalent to 3 months of silver support) to 5 lucky >>> community members who post a response. Please make sure to reply before >>> April 22nd. >>> >>> Thanks for your insights, and cloud on! >>> >>> Katie >>> >> -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/google-appengine. > To view this discussion on the web visit > https://groups.google.com/d/msgid/google-appengine/5610bc4d-852b-4476-a10b-51574cb2e925%40googlegroups.com > <https://groups.google.com/d/msgid/google-appengine/5610bc4d-852b-4476-a10b-51574cb2e925%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/google-appengine. > To view this discussion on the web visit > https://groups.google.com/d/msgid/google-appengine/1AA8B6DC-16ED-4737-A559-A626B18E3E48%40rakkoon.com > <https://groups.google.com/d/msgid/google-appengine/1AA8B6DC-16ED-4737-A559-A626B18E3E48%40rakkoon.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAD1DWLiE7WEeL2Kpd%2BhCBTA1b2cLpy1CokixYeN1XJ76OV6Zug%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
