Well, the idea is that the blob server would run as a "Lambda" function and would store the blobs in S3 (and possibly use DynamoDB as a k/v store).
Go is not officially supported on AWS Lambda but somebody just built this: https://github.com/eawsy/aws-lambda-go-net The handler is created in the init functions (and I am still trying to figure out if there is a way to intercept a "shutdown" phase, if it's needed). The "server" runs for a while, at least if there are requests coming and it's stopped / shutdown when not needed. And yes, I know Camlistore already run on the Google infrastructure (and I will try that too) but as I said I already use AWS for other purposes and I thought this could be an interesting exercise. Thanks! -- Raffaele On Wed, Nov 23, 2016 at 2:55 PM, Mathieu Lonjaret < [email protected]> wrote: > Hi. > > I don't know more about Amazon Lambda other than what I've just read by > skimming through their docs, but my first impression is that it might be an > interesting exercise but it would be quite a lot of work too? > I mean, assuming that Go is supported as you say (I haven't seen it > mentioned in the docs), how do you envision even the most basic pieces to > work? How/where would the blobserver run? Where would it store? on S3? Then > you need at least an http handler on top of that for the blobserver to be > of any use? How/where does that handler run? How is it created/started? > > I don't want to dive too much into the lambda docs right now, but I can > try and help with the Camlistore parts and tell you what pieces you need if > you have specific questions though. > > Btw, since you mention appengine, do you know what we also run on Google > Compute Engine/Cloud Storage? And that we have a launcher that does most of > the work for you? https://camlistore.org/launch/ > > Cheers, > Mathieu > > On 23 November 2016 at 20:08, Raffaele Sena <[email protected]> wrote: > >> I would like to use Camlistore for my pictures, that I rarely browse >> until I need them (I am currently hosting most of them on a VPS and I don't >> remember when was last time I looked at them) and I thought I'll store them >> in S3 since I am already using AWS for other purposes. >> >> I know I could run a local server configured to use S3 but I do sometime >> access the pictures from random browsers (and maybe for that use case I >> could just run camlistored on a micro instance and that would be good >> enough). >> >> But I have been toying with the idea of running the blob server out of >> AWS Lambda and I am trying to figure out if this makes any sense or not (if >> it does maybe the next step would be running other parts of camlistore on >> Lambda). >> >> Basically I was going to start from the "appengine" code ( >> https://github.com/camlistore/camlistore/tree/master/server/appengine) >> and tweak it to run on AWS. >> >> Somebody recently created a package to run Go application "natively" on >> AWS lambda (no JS or python proxy) so there would be no overhead there. >> And Lambda services once started do run for a while (not sure of how long >> they run, and if there is a way to get a signal when they are terminated, >> but I am doing some experiments to figure that out). >> >> I am still debating if there would be more overhead running a local >> server that push data directly to S3 vs. pushing blob to the hosted blob >> server but again one of the purpose of this exercise would be to have the >> full environment running in AWS "on demand". >> >> So, is this a crazy idea that I should abandon right now or does it have >> some value ? Is there anything major that I am missing (I didn't look at >> the code that much but I am assuming the blob server should keep a "global >> state" that requires for the server to be running all the time, and the >> overhead of starting the process is relatively small. >> >> Thanks! >> >> -- Raffaele >> >> >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Camlistore" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "Camlistore" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Camlistore" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
