Hey John, The thread linked by Kyle does indeed have some good ideas for background context logging. I believe the explanation for the HTTP load balancer not working with websockets is that it's only meant to transfer HTTP traffic, and won't respond properly to socket connection attempts. If your sockets library is capable of falling back to HTTP long polling (such as socketio tends to do), that might work through the HTTP load balancer. A common pattern to side-step this difficulty is to have an endpoint on each instance which allows the client to request its IP, so that the socket connection can be made to the machine itself, rather than through the appspot front-end or an HTTP load balancer. I think that's your best bet for a scalable solution to websocket connections against a pool of instances, keeping the HTTP traffic for the load balancer.
On Wednesday, December 23, 2015 at 6:02:03 AM UTC-5, John Smith wrote: > > Hey Nick & Kyle, > > @Nick > Yes I do need to call App Engine services in response to WebSocket > requests. > There are multiple issues here and I'm trying to work around them > > to sum things up: > > - WebSocket connections are not supported in the Classic App Engine OR > the HTTP load balancers > - The App Engine log API does not export API for writing specific log > entries and the BackgroundContext logs get cluttered > > > @Kyle > You probably could not connect to your MVM instance because the default > firewall rule forbids access to port 80, try :8080 instead > After browsing the thread you linked to I have found this > <https://godoc.org/google.golang.org/cloud/logging#Client.Log> API, which > I will look into and might help me mitigate App Engine's lack of finer > resolution log infrastructure. > > Thanks again, > John > > > On Wednesday, December 23, 2015 at 7:03:47 AM UTC+2, Kyle Finley wrote: >> >> John, >> >> I just tested accessing my VM by IP. Unfortunately the request timed out: >> >> curl: (7) Failed to connect to 23.XX.XX.XX port 80: Operation timed out >> >> >> Incase you haven't seen it yet. There's a thread in the App Engine Go >> group where the issue with background context logging is discussed. There's >> a few alternatives suggested. >> >> >> https://groups.google.com/forum/#!searchin/google-appengine-go/background/google-appengine-go/V90qiZKJ5Tk/CWiXsxkNDgAJ >> >> I hope you find a solution I'll be looking into websockets in the near >> future. >> >> Kyle >> >> On Tuesday, December 22, 2015 at 9:15:28 AM UTC-6, John Smith wrote: >>> >>> Hey Kyle, >>> >>> yes I am using appengine.Main() to start serving requests. >>> did you try to access your VMs directly? (by IP, or by >>> network-load-balancer) >>> >>> (note: when using google's default my-game.appspot.com >>> appengine.NewContext works but WebSockets dont) >>> >>> Thanks >>> >>> >>> On Tuesday, December 22, 2015 at 4:16:56 PM UTC+2, Kyle Finley wrote: >>>> >>>> John, >>>> >>>> Are you calling appengine.Main() in your main() function? >>>> >>>> I'm not sure if it works with the network-load-balancer, but when I >>>> switched from classic to MVM I was receiving the same error. Adding the >>>> appengine.Main() call fixed the unknown http.Request error. >>>> >>>> Kyle >>>> >>>> e.g. >>>> >>>> package main >>>> >>>> import ( >>>> "google.golang.org/appengine" >>>> ) >>>> >>>> func main() { >>>> appengine.Main() >>>> } >>>> >>>> On Tuesday, December 22, 2015 at 7:42:16 AM UTC-6, John Smith wrote: >>>>> >>>>> Hey Nick, >>>>> >>>>> Thank you for your response, I would like to further clarify the flow >>>>> of things: >>>>> >>>>> - I have to use WebSockets >>>>> - WebSockets are not supported on the Classic App Engine (when I >>>>> attempt to open a websocket connection to my-game.appspot.com it >>>>> is being filtered out and not even making it to my app code) >>>>> - WebSockets seem to work on MVM via a network-load-balancer >>>>> <https://cloud.google.com/compute/docs/load-balancing/network/> (and >>>>> not via the default my-game.appspot.com domain) >>>>> - using MVM and a network-load-balancer I cannot create a request >>>>> context [to create a new context I use the supplied >>>>> appengine.NewContext(r) >>>>> function and get the mentioned error] >>>>> - I need a context to use the datastore.GetMulti() or log.Debugf() >>>>> APIs >>>>> - I am able to create a background context (supported by MVM) and >>>>> use these APIs but when I use this context (with the >>>>> appengine/log.Debugf() >>>>> API) the logs get cluttered (aggregated under /_ah/background) >>>>> >>>>> >>>>> 1. are the above points clarify how my app receives requests? >>>>> 2. no, when a request is dispatched via network-load-balancer (and >>>>> not my-game.appspot.com) my application gets no X-Appengine-* >>>>> headers at all >>>>> 3. yes all of my logs are piling up under one /_ah/background >>>>> entry. as mentioned above I am using the golang standard appengine/log >>>>> library as I did with my previous Classic App engine. >>>>> >>>>> >>>>> ideally the appengine.NewContext(r) function would work and logging >>>>> via the appengine/log.Debugf() would split the lines per request. >>>>> do you find this explanation sufficient? >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> >>>>> On Tuesday, December 22, 2015 at 12:21:02 AM UTC+2, Nick (Cloud >>>>> Platform Support) wrote: >>>>>> >>>>>> As someone somewhat familiar with the Go runtime but not with your >>>>>> exact use case, I'd like to help, but I'm wondering if you could provide >>>>>> some more concrete specification of the following from your post, in >>>>>> terms >>>>>> of code or logs: >>>>>> >>>>>> While porting my app from the Classic App Engine I was facing an >>>>>>> issue with the request context (which now "skips" the default HTTP load >>>>>>> balancer and goes straight to my MVM(gce) node). >>>>>>> >>>>>> >>>>>> >>>>>>> I was getting an error ("appengine: NewContext passed an unknown >>>>>>> http.Request"). >>>>>>> >>>>>> >>>>>> >>>>>>> To overcome my need for datastore/logging/other App Engine >>>>>>> facilities I used context.BackgroundContext() everywhere and it seemed >>>>>>> to >>>>>>> work well for the datastore/urlfetch/etc.. >>>>>> >>>>>> >>>>>> This might help me to reproduce your situation and attempt different >>>>>> solutions. >>>>>> >>>>>> As to your numbered questions, here are my first responses: >>>>>> >>>>>> 1. Are you talking about "Network Loadbalancing >>>>>> <https://cloud.google.com/compute/docs/load-balancing/network/>" or >>>>>> the GCE "Http Load balancer >>>>>> <https://cloud.google.com/container-engine/docs/tutorials/http-balancer>", >>>>>> >>>>>> and could you clarify again what kind of errors you observed when >>>>>> attempting this? What is meant by context "skipping"? >>>>>> >>>>>> 2. Do you notice that you're receiving X-Appengine-* headers? If so, >>>>>> you can certainly use them. If you attempt to add them to requests, do >>>>>> you >>>>>> notice if they're stripped? If they are, it won't work, if they aren't, >>>>>> it >>>>>> will. Although this is a quite empirical way to proceed, and if >>>>>> something >>>>>> isn't explicitly documented, you should file a Public Issue Tracker >>>>>> feature request >>>>>> <http://code.google.com/p/google-appengine/issues/list> to have the >>>>>> behaviour defined. >>>>>> >>>>>> 3. Your log lines are all piling up under one /_ah/background entry, >>>>>> as far as I can tell. You could use the Logging API >>>>>> <https://cloud.google.com/logging/docs/api/tasks/creating-logs> to >>>>>> write logs, and see if that shows up properly. Another option is logging >>>>>> to >>>>>> a BigQuery or SQL table. The final, most trivial answer, is to figure >>>>>> out >>>>>> how to get request logging to work without this BackgroundContext which >>>>>> seems to flatten all its logs into one expando in the Developers Console >>>>>> UI >>>>>> (if I'm understanding you correctly, again, it seems like this is what >>>>>> you're reporting). >>>>>> >>>>>> 4. A better approach could only be found through further >>>>>> investigation, if it exists. I'm optimistic that a better approach is >>>>>> out >>>>>> there, it would just take more thought. >>>>>> >>>>>> On Sunday, December 20, 2015 at 11:17:12 AM UTC-5, John Smith wrote: >>>>>>> >>>>>>> *TL;DR: *Need to use WebSockets on AE. Only works on MVM. >>>>>>> NewContext fails, BackgroundContext works. Logs are polluted & >>>>>>> disorganized >>>>>>> (/_ah/background) >>>>>>> >>>>>>> Hey, >>>>>>> >>>>>>> I would like to communicate with my App Engine app over WebSockets. >>>>>>> I have found that the Java library has an official (github) example >>>>>>> of using WebSockets under the Managed VMs framework, >>>>>>> after browsing the implementation I noticed that it queries the >>>>>>> internally exposed metadata service to get the external IP of the >>>>>>> machine >>>>>>> and redirects the client to it. >>>>>>> >>>>>>> However I expect my application to receive more traffic than a >>>>>>> single server would handle and so I need a load-balanced solution >>>>>>> (cannot >>>>>>> relay on a single static-IP) >>>>>>> so I've looked into App Engine's "Network Load Balancer" and after >>>>>>> an hour (and a long deploy time) I was up and running with now my >>>>>>> domain set to the load balancer's static IP. >>>>>>> >>>>>>> Now is a good time to note that I use the Go standard runtime. >>>>>>> While porting my app from the Classic App Engine I was facing an >>>>>>> issue with the request context (which now "skips" the default HTTP load >>>>>>> balancer and goes straight to my MVM(gce) node). >>>>>>> I was getting an error ("appengine: NewContext passed an unknown >>>>>>> http.Request"). >>>>>>> To overcome my need for datastore/logging/other App Engine >>>>>>> facilities I used context.BackgroundContext() everywhere and it seemed >>>>>>> to >>>>>>> work well for the datastore/urlfetch/etc.. >>>>>>> >>>>>>> Everything worked but the logging facility which was polluting my >>>>>>> logs with empty "/_ah/background" lines and everything else was >>>>>>> multiplexed >>>>>>> into a single huge "/_ah/background" entry. >>>>>>> This makes it very difficult to debug or monitor my application. >>>>>>> >>>>>>> I have browsed the google.golang.org/appengine package and tried to >>>>>>> split the log lines to no avail. >>>>>>> >>>>>>> >>>>>>> My questions are: >>>>>>> 1) Can I use the classic App Engine/default http load >>>>>>> balancer/custom http load balancer to service my WebSocket requests? >>>>>>> 2) Can I, with my current solution, get the X-AppEngine-* headers? >>>>>>> 3) How could I, with my current solution, split the log lines to per >>>>>>> request entries (very like the classic app engine logs) >>>>>>> 4) Is this the right approach? >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/25f494a1-bc28-4329-bf84-1c8f9f6acad3%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
