Stack, thanks for sharing.
I can't access the google doc mentioned after "...Ryan then presented this doc..." due to permission. Is there a way to get it? thanks Demai On Mon, May 12, 2014 at 12:12 PM, Jean-Marc Spaggiari < [email protected]> wrote: > Nice! Thanks for sharing St.Ack. > > JM > > > 2014-05-12 13:32 GMT-04:00 Stack <[email protected]>: > > > Below are some rough notes taken during first 20 minutes of our > > hackathon/dev meetup last Tuesday (Your secretary had to abandon > > note-taking for white-boarding duty and failed to pass the baton). > > > > There were about ~50 folks in attendance with representatives > > from a variety of organizations. > > > > We started by promoting the items listed on the hackathon > > meetup page to the white board: > > > > http://files.meetup.com/1350316/IMG_20140506_190458.jpg > > > > The list of topics up for discussions were: > > > > + Agreeing on types among hbase-related projects (phoenix, > > kite, etc.). > > + Discussion around colocation of hbase master and meta > > and the master also doing regionserver services and whether > > we should retain the current topology (no regions on the > > master or backup master) for 1.0. > > + The ongoing 'consensus' work (putting zk usage behind > > a pluggable interface). > > + Transactions over HBase (Continuuity work, Xiaomi > > percolator, and VCNC's work). > > + Netty the server > > + HTrace > > + Multitenancy > > > > We started talking types. It took a while to get going . Below are the > raw > > notes: > > > > Static, dynamic or what? > > > > Lars Hofhansl: Is hbase a byte store? Should types be in there? If hbase > > knew > > about the type then could we exploit it internally. > > > > Should HBase know about types? Thought is that HBase could do some > > perf stuff if it was cognizant of types. Maybe later do this or > > start out w/ a few common types first. > > > > At least one encoding strategy, work on this first. > > > > Make it so we don't design ourselves into a corner. > > > > Talking to Enis, Hive & Pig... what do these projects want? Plug in > codec? > > > > Kite project, tries to do a layer above byte store rather than in the fs. > > > > Encodings are not the same across storage engines for Kite. > > > > Does Parquet need to do float or integer? But that is where it gets its > > perf advantage. > > > > Parquet does not necessarily work inside hbase. > > > > Kite as a lib? That phoenix could use? > > > > Could add a phoenix encoding to Kite? Yes. But lets agree and then > retire > > the avro encoding. > > > > Encoding is separate from schema. > > > > Ryan Blue: Don't want to say phoenix is on kite. Just want to focus on > > encoding. > > > > James Taylor: Phoenix should be under kite, not on top of kite. > > > > Ryan: move to common encoding, and a lib to serialize.... > > > > Jon Hsieh: where this could be used... phoenix encoding for kite... then > > all > > could use it writing out. > > > > Lars: What you doing at HP for type encoding. > > > > HP: Our code is in C. All byte arrays. We have a way of doing floats, > > etc. > > > > Ryan: But you want a serializing too? > > > > Lars: How we start in on this? > > > > Ryan: I sent out a doc. Ryan then presented this doc: > > > > > > > https://docs.google.com/a/cloudera.com/document/d/15INOaxyifycpFxvB6xNdoj96JbOpslEywjkaoG8MURE/edit#heading=h.o1cgqtsqgqyg > > > > Pick just a few simple types and get agreement on these first? > > > > HP: We have seen in the past that this works for a while but then folks > > figure > > out that their complex types are slower than they should be and they > start > > coming up w/ their own encodings as workarounds; now you are back to > > square one.... you need to version it all. > > > > Was proposed that we try and unify around a typing strategy that would > work > > as > > the ONE way to do it in HBase with Phoenix going to try and come up on it > > first. > > To be continued up on the HBase dev list. > > > > Another proposal that had alot of nods in favor was that we not require > > 1.0 and 2.0 to be compatible. > > > > Notes ran out at this stage. > > > > St.Ack > > >
