> Hence, I am making Hbase a required component for trunk for a full end to end > deployment.
Does that mean it's not required for a minimal end to end deployment? :) I think it's fine to have HICC require HBase, and I hope to use it at some point. But I think users should still be able to use the agent/collector/demux part of Chukwa without requiring HBase if they don't need that functionality. On Mon, Oct 11, 2010 at 4:49 PM, Ariel Rabkin <[email protected]> wrote: > I think I'm with Bill on this. > > There are useful Chukwa deployments without HBase, and I don't want to > scare anybody away by saying "a full deployment has HBase." > > I would say "you can use Chukwa on either HDFS or HBase. HBase has > lower latency, and requires Hadoop 0.20. Using HDFS+Mapred works with > 0.18" > > --Ari > > On Mon, Oct 11, 2010 at 4:31 PM, Eric Yang <[email protected]> wrote: >> For streaming data to Hbase 0.20.6, Hadoop 0.20 is required. If you are >> using the sequence file and mapreduce, then Hadoop 0.18+ should work fine. >> I would like to move forward with map reduce on table instead of map reduce >> on file to reduce data processing latency. Hence, I am making Hbase a >> required component for trunk for a full end to end deployment. The setup >> may sound difficult to do, but it is trivial to setup. In the long run, >> demux does not need to be a map reduce job, and demux will only get >> optimized for incremental updates, when developers start to adopt the table >> approach. Considering this migration as planning for a better future. :) >> For now, the existing Chukwa 0.4 approach is still works in trunk but I am >> not enhancing the batch model. >> >> Regards, >> Eric >> >> On 10/11/10 1:11 PM, "Bill Graham" <[email protected]> wrote: >> >>> Hi Eric, >>> >>> A read this over and have a few comments: >>> >>> - Prerequisites say that Hadoop 0.20+ is required, which I think isn't >>> entirely true. Agents, collectors and the data processor processes all >>> can run with 0.18.3. I though HICC was the only thing that required >>> 0.20, no? Either way, we should clarify which components require what >>> version. >>> >>> - You talk about a minimal Chukwa install including HBase, which also >>> seems misleading. Will HBase be required to use Chukwa going forward? >>> If not, no need to make the barrier for entry sound higher than it >>> needs to be IMO. >>> >>> thanks, >>> Bill >>> >>> On Sat, Oct 9, 2010 at 6:47 PM, Eric Yang <[email protected]> wrote: >>>> Hi, >>>> >>>> I have written some instructions on how to deploy Chukwa 0.5 >>>> pre-release system with HBase. Instructions are available here: >>>> >>>> http://wiki.apache.org/hadoop/Chukwa_Quick_Start >>>> >>>> Suggestions and feedback are welcome. >>>> >>>> regards, >>>> Eric >>>> >>> >> >> > > > > -- > Ari Rabkin [email protected] > UC Berkeley Computer Science Department >
