Nice! Thanks for sharing St.Ack. JM
2014-05-12 13:32 GMT-04:00 Stack <[email protected]>: > Below are some rough notes taken during first 20 minutes of our > hackathon/dev meetup last Tuesday (Your secretary had to abandon > note-taking for white-boarding duty and failed to pass the baton). > > There were about ~50 folks in attendance with representatives > from a variety of organizations. > > We started by promoting the items listed on the hackathon > meetup page to the white board: > > http://files.meetup.com/1350316/IMG_20140506_190458.jpg > > The list of topics up for discussions were: > > + Agreeing on types among hbase-related projects (phoenix, > kite, etc.). > + Discussion around colocation of hbase master and meta > and the master also doing regionserver services and whether > we should retain the current topology (no regions on the > master or backup master) for 1.0. > + The ongoing 'consensus' work (putting zk usage behind > a pluggable interface). > + Transactions over HBase (Continuuity work, Xiaomi > percolator, and VCNC's work). > + Netty the server > + HTrace > + Multitenancy > > We started talking types. It took a while to get going . Below are the raw > notes: > > Static, dynamic or what? > > Lars Hofhansl: Is hbase a byte store? Should types be in there? If hbase > knew > about the type then could we exploit it internally. > > Should HBase know about types? Thought is that HBase could do some > perf stuff if it was cognizant of types. Maybe later do this or > start out w/ a few common types first. > > At least one encoding strategy, work on this first. > > Make it so we don't design ourselves into a corner. > > Talking to Enis, Hive & Pig... what do these projects want? Plug in codec? > > Kite project, tries to do a layer above byte store rather than in the fs. > > Encodings are not the same across storage engines for Kite. > > Does Parquet need to do float or integer? But that is where it gets its > perf advantage. > > Parquet does not necessarily work inside hbase. > > Kite as a lib? That phoenix could use? > > Could add a phoenix encoding to Kite? Yes. But lets agree and then retire > the avro encoding. > > Encoding is separate from schema. > > Ryan Blue: Don't want to say phoenix is on kite. Just want to focus on > encoding. > > James Taylor: Phoenix should be under kite, not on top of kite. > > Ryan: move to common encoding, and a lib to serialize.... > > Jon Hsieh: where this could be used... phoenix encoding for kite... then > all > could use it writing out. > > Lars: What you doing at HP for type encoding. > > HP: Our code is in C. All byte arrays. We have a way of doing floats, > etc. > > Ryan: But you want a serializing too? > > Lars: How we start in on this? > > Ryan: I sent out a doc. Ryan then presented this doc: > > > https://docs.google.com/a/cloudera.com/document/d/15INOaxyifycpFxvB6xNdoj96JbOpslEywjkaoG8MURE/edit#heading=h.o1cgqtsqgqyg > > Pick just a few simple types and get agreement on these first? > > HP: We have seen in the past that this works for a while but then folks > figure > out that their complex types are slower than they should be and they start > coming up w/ their own encodings as workarounds; now you are back to > square one.... you need to version it all. > > Was proposed that we try and unify around a typing strategy that would work > as > the ONE way to do it in HBase with Phoenix going to try and come up on it > first. > To be continued up on the HBase dev list. > > Another proposal that had alot of nods in favor was that we not require > 1.0 and 2.0 to be compatible. > > Notes ran out at this stage. > > St.Ack >
