Sorry for the late reply, yeah I think I'm probably asking too much to App Engine :) So I'll try to use Compute Engine (or related) directly.
Thanks for the reply, was really helpful On Tue, Apr 12, 2016 at 5:11 PM 'Nick (Cloud Platform Support)' via Google App Engine <[email protected]> wrote: > Hey Patrick, > > It seems likely that attempting to fetch the full contents of the file > into memory (via the URL Fetch response body) is causing the instance to > run out of memory. You could try to break the download / upload of the file > into chunks if the remote server can handle chunked downloads, or you could > increase the App Engine instance class to be one with more memory > <https://cloud.google.com/appengine/docs/go/modules/#Go_Instance_scaling_and_class>. > Finally, you could have a Compute Engine > <https://cloud.google.com/compute/> machine do the work of downloading to > disk and uploading to the Drive API (using service account > <https://developers.google.com/identity/protocols/OAuth2ServiceAccount#authorizingrequests> > credentials), communicating with your App Engine app via requests or some > other means to trigger this work. > > I hope this has helped show some of the alternatives and the nature of the > problem. > > Regards, > > Nick > Cloud Platform Community Support > > > On Tuesday, April 12, 2016 at 7:21:45 AM UTC-4, Patrick Arminio wrote: >> >> Hi everyone, >> >> I'm trying to download a file from an URL and upload it to Google drive >> using GO and Appengine >> Here is the code I'm using >> https://gist.github.com/patrick91/49bd59cc69b43c06e556ad472242e63d >> >> Unfortunately I'm getting an OutOfMemory error on appengine, is there >> anything I can do? >> >> I know that you can't store a file on Appengine, so I was thinking if is >> there >> a way to do it in memory or shall I use Google Cloud Storage? >> >> Patrick >> > -- > You received this message because you are subscribed to a topic in the > Google Groups "Google App Engine" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/google-appengine/wdDV8wfz2k4/unsubscribe > . > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/google-appengine. > To view this discussion on the web visit > https://groups.google.com/d/msgid/google-appengine/f1b3cb2e-1c5f-40e8-a2ff-df004240d8f9%40googlegroups.com > <https://groups.google.com/d/msgid/google-appengine/f1b3cb2e-1c5f-40e8-a2ff-df004240d8f9%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- Patrick -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAOUxZcu8hy-4MFi3KQD5uXjKFyc-6-gG%3DT-syFxf2zC2QxMchA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
