[go-nuts] Re: [ANN] A Go based map reduce system (second attempt)

2016-09-13 Thread Daniel Whitenack
Chris, This is great to see! Nice work so far. I enjoyed experimenting with glow and am excited to follow this project. If you are able you might provide some updates in #data-science on gophers slack. I know people there would love to hear about gleam. Best, Daniel On Monday, September 1

Re: [go-nuts] Re: [ANN] A Go based map reduce system (second attempt)

2016-09-13 Thread Daniel Whitenack
Henrik and David, Yes, this is precisely what pachyderm is trying to accomplish (along with a built in data versioning system). It's pretty wonderful, and I recommend running through their quick start guide: http://pachyderm.readthedocs.io/en/latest/ Daniel On Monday, September 12, 2016 at 1

Re: [go-nuts] Re: [ANN] A Go based map reduce system (second attempt)

2016-09-12 Thread Henrik Johansson
Perhaps something like this? https://github.com/pachyderm/chess mån 12 sep. 2016 kl 18:32 skrev David Crawshaw : > The map-reduce systems I've worked with in the past bundle up the map > and reduce functions in a binary, distribute it, and then communicate > with it by an IPC or RPC system. Th

Re: [go-nuts] Re: [ANN] A Go based map reduce system (second attempt)

2016-09-12 Thread David Crawshaw
The map-reduce systems I've worked with in the past bundle up the map and reduce functions in a binary, distribute it, and then communicate with it by an IPC or RPC system. This lets them take advantage of the operating system to control resource allocation. (Given how popular containers are becomi

[go-nuts] Re: [ANN] A Go based map reduce system (second attempt)

2016-09-11 Thread Raj
>> However, it is still limited by the fact that Go code can not be sent and executed remotely. I am not sure if this makes the situation better for this particular use case, but I see that David Crawshaw is working on plugin mode for Go. It looks like it will be part of Go1.8. -- You receive