Hi Stefan, I'm continuing to plug away at code myself. My goal is to have a fully-working (though feature-poor) system before committing any code. I don't want to clutter the Hadoop workspace with experimental stuff. (I feel like I did that a little too much in the early days of DFS, and I want to avoid that experience ;)
A wiki for current docs, status, etc, is a good idea. I'll look to add that to the Hadoop pages. (Also, let's talk offline about how we can best work together on this. Once the first commit is finished, then I think it should enter the general bug-patch-commit development mode.) Storing namenode and MapReduce status in a BigTable-like structure is intriguing, but seems dangerous. (All the database stuff is built on top of DFS and MapReduce. What happens when the database needs a file in DFS, which needs a record from the database? Oh lord...) --Mike On 5/30/06, Stefan Groschupf <[EMAIL PROTECTED]> wrote:
Hi Michael, what you think are the next steps? I really would love to help in the implementation, ideally we define interfaces or create lists of functionality and may be people can provide implementations for the different components. Or is your idea to provide a first implementation and continue on a patch file kind of development? What you think to create a wiki page were we at least list all components within a short description and list of functionalities. We can start with your emails and discuss each component in detail here in the list. Makes that sense? BTW, Yoram and me discussed that a kind of big table could be a interesting storage for data that are hold in memory in jobtracker and namenode and cause some scalability problems. What you think? Greetings, Stefan
