Thanks for the warm welcome guys :). I agree that smaller patches will be easier to parse and review. Right now, I have Debian and RPM packaging complete. I have created a JIRA ticket (https://issues.apache.org/jira/browse/BIGTOP-2283) which I will use to track the integration.
I’ll start sending in pull requests as I complete each of the tasks necessary to get this integration done. I’d love all of your guys’ guidance along the way so that the integration can be as correct to bigtop’s standards as possible. Also, regarding the single point of failure sidenote discussed in another thread, Quantcast is actively working on updating qfs to support a distributed metaserver so that we can avoid the single point of failure. It’s not ready for release yet but we will announce it when it is :). Faraaz On 1/26/16, 11:33 AM, "jay vyas" <[email protected]> wrote: >Please add Your name to the MAINTAINERS.txt file in the PR for "QFS" ! > >On Tue, Jan 26, 2016 at 2:24 PM, Konstantin Boudnik <[email protected]> wrote: > >> On Wed, Jan 27, 2016 at 01:05AM, Evans Ye wrote: >> > Oh, if I made you uncomfortable, you know I didn't mean that :) >> >> Oh believe me - it takes a lot more than that to make me uncomfortable ;) >> Besides, you gave me an opportunity to speak up about the SPOFs in the >> Hadoop >> ecosystem. Which I can do for hours! >> >> > I just want to know more about the design because I'm always interested >> in >> > the architecture things, especially when I see something different. >> > Even Spark just added HA in their master in recent release last year. So >> > it's totally fine. >> > >> > And the following is just for the discussion: >> > >> >> Just a side node: as you know literally _all_ components in Hadoop-based >> >> stack have SPOF. >> > >> > I think it's more related to the maturity of a component, but somehow it >> > will get there to eliminate the SPOF just like what HDFS, YARN, Oozie, >> Spark >> > did. For operations, two SPOFs is good enough for them to have a good >> sleep >> > at night ;) >> >> It might be ok for Oozie, but not for a file system. Because once your >> master >> has failed over to the standby you don't know how long that one has. It >> might >> crash in 5 months or in 2 minutes. That's why if you're an admin for a >> production critical cluster you'll get paged immediately. Perhaps 3 >> o'clock in >> the morning. >> >> I spent last three years working on the active-active software for mission >> critical systems, so I would love to share some architectural thoughts with >> you, if you're interested. >> >> Cos >> >> > 2016-01-26 12:08 GMT+08:00 Jay Vyas <[email protected]>: >> > >> > > Awesome! It's great to hear about another HDFS alternative :) >> > > >> > > , We tried pretty hard to make bigtop more "HCFS" compliant, by adding >> the >> > > init hcfs tooling and so on, but never completed making the puppet >> recipes >> > > add support for other file systems. >> > > >> > > >> > > Would be very interesting to see if the quant cast team can take over >> on >> > > the HCFS integration so that bigtop is a home for more Than just one >> under >> > > filesystem . >> > > >> > > Let us know what your planning on working on, looking forward to it...! >> > > >> > > >> > > >> > > >> > > > On Jan 25, 2016, at 9:06 PM, Faraaz Sareshwala < >> > > [email protected]> wrote: >> > > > >> > > > Hello everyone in the Bigtop community! >> > > > >> > > > My name is Faraaz Sareshwala and I am an engineer working at >> Quantcast >> > > to integrate QFS (https://quantcast.github.io/qfs) into the Bigtop >> > > project. I just wanted to send a small note out to introduce myself to >> the >> > > community and announce that we are working on an integration. Feel >> free to >> > > reach out to me if you have any questions about QFS, our integration, >> or >> > > Quantcast itself. >> > > > >> > > > At this point, I am nearly complete integrating QFS. I plan to >> submit a >> > > patch through github and JIRA later in the week or early next week >> once I >> > > get a chance to test everything is working correctly. I’ve read the >> wiki on >> > > contribution to the project, but if you guys have any other tips or >> > > guidance for me, I’d really appreciate that. >> > > > >> > > > Patch coming soon! >> > > > >> > > > Take care, >> > > > Faraaz >> > > > >> > > >> > > > >-- >jay vyas
