Hi, On Wed, Feb 4, 2009 at 8:00 PM, Luke <[email protected]> wrote: > > On Feb 4, 10:48 am, "S." <[email protected]> wrote: >> "The mapreduce (MR) is currently supported by the Hadoop DFS only. "
This in fact is an editing error. I don't know why I wrote it the way it. I have fixed the mistake. It should have said "(...) supported by the Hadoop only." > Don't be afraid to ask any questions here. We really appreciate them > as we want to understand what our (new) users are thinking. > >> Is there any plan to implement a native Mapreduce support? > > For post 1.0 releases, we definitely thought about implementing some > form of map-reduce that makes sense for our system. My current integration efforts after doing 2 throw-away prototypes are here: http://github.com/mateuszb/hypertable/tree/3206b864687cc565ffb9fe10e82f0b37bce7979b/src/cc/MapReduce It's not much as things are moving slowly after work hours but still forward. In case anyone is _really_ interested in lending a helping hand please contact me. >> Or are >> Mapreduce algorithms so dependant of the filesystem that there is no >> way to design a general support environment? The file system is used mostly as a mean of distributing applications in the cluster, i.e. a scheduler copies an application to the DFS, and the location is then broadcast to the worker servers which then copy the application from the DFS and run it. Other than that there's also a concept of providing output from the mapreduce tasks. The original implementation does output data into the DFS files. But, thanks to Hypertable's DfsBroker the concept of a DFS is abstracted away, so you can use both Hadoop DFS and KFS. Mateusz --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en -~----------~----~----~----~------~----~------~--~---
