Also. I was going to start separate threads but probably there is not that much
to discuss now or off-JIRA:
- I spoke with Jia Liu about HBASE-4120. After 0.92 is out I intend to champion
/ mentor / co-develop 4120 and the follow on table allocation work and target
0.94 for it. I think the RPC QoS aspect is not too controversial. The
allocation/reservation aspects I'd like to aim for a coprocessor or at least
master plugin based integration so they won't impact stability for users who
don't enable it. Unlike RPC QoS I suspect the changes needed to core can be
minimized to coprocessor framework additions. Follow up in new JIRAs soon.
- I also met with the author of "CCIndex". He is still interested in pursuing
an upstreaming of this but may not have a lot of free time. The current
implementation powers the data cube product at Taobao with half a million users
IIRC, but the implementation is an invasive hack of the 0.20 core code. I think
the vast majority of it is an application layer concern that can be moved to a
coprocessor implementation. The coprocessor framework will need some
modification as well, specifically hooks for handling IOEs taken on HFile
reads. Anyway I'm going to play around with this and follow up with a JIRA.
Best regards,
- Andy
On Dec 7, 2011, at 4:00 AM, Andrew Purtell <[email protected]> wrote:
> I also attended day 1 of Hadoop in China and would add these notes:
>
> - Taobao has their own HA NameNode and is interested in contributing their
> changes upstream. I spoke with a platform dev lead. I told him do so now
> while HA is under active development rather than wait and miss an opportunity
> to participate/contribute. I need to follow up with him with a list of active
> HA jiras.
>
> - Mingjie Lai presented on HBase and ZooKeeper security. I'll ask him to
> upload his slides somewhere.
>
>
> Best regards,
>
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein (via
> Tom White)
>
>
> ----- Original Message -----
>> From: Todd Lipcon <[email protected]>
>> To: [email protected]
>> Cc:
>> Sent: Wednesday, December 7, 2011 2:59 AM
>> Subject: Re: trip report for Hadoop In China
>>
>> Ah, I clicked on "hadooper.cn" directly instead of the earlier link to
>> the agenda. Oops :)
>>
>> Maybe you can put them in a public S3 bucket?
>>
>> -Todd
>>
>> On Tue, Dec 6, 2011 at 10:56 AM, Ted Yu <[email protected]> wrote:
>>> From http://hic2011.hadooper.cn/dct/page/1 (the first two lines):
>>> Slides would be posted after getting permission from presenters
>>>
>>> I can share my slides. My attempt at uploading ppt to slideshare failed and
>>> I don't have PRO account.
>>>
>>> I think apache.org would filter attachment.
>>>
>>> Suggestion on where I can put my slides would be appreciated.
>>>
>>> On Tue, Dec 6, 2011 at 10:39 AM, Todd Lipcon <[email protected]>
>> wrote:
>>>
>>>> Hey Ted,
>>>>
>>>> Thanks for the trip report. Interesting stuff.
>>>>
>>>> For those of us who can't read mandarin, can you link directly to
>> any
>>>> english slide decks? The hadooper.cn link seems to go to an empty wiki
>>>> template page or something, if I understand Google Translate correctly
>>>> :)
>>>>
>>>> -Todd
>>>>
>>>> On Tue, Dec 6, 2011 at 10:36 AM, Ted Yu <[email protected]>
>> wrote:
>>>>> Hi,
>>>>> There were around 1000 attendees for the first day of HiC.
>>>>> The enthusiasm for cloud computing, hadoop in particular, was
>>>> overwhelming.
>>>>>
>>>>> Doug Cutting's keynote was well received by attendees.
>>>>> **Grzegorz's presentation about latest development @ Google,
>> Pregel in
>>>>> particular, was impressive.
>>>>> Xiaodong Zhang's keynote about DOT, an analytical model, was
>> interesting.
>>>>>
>>>>> I listened to several presentations. Here're a few:
>>>>>
>>>>> Zheng Shao talked about Puma3 architecture which his group
>> developed at
>>>>> Facebook
>>>>> Jerry and Liyin talked about how Facebook messaging uses Hbase.
>> Jerry
>>>>> covered some best practices in using Hbase. Liyin talked about
>> some
>>>>> features developed by their team, including Hfile V2, new bloom
>> filters,
>>>> etc
>>>>> Jinglong Liu from Baidu talked about how his team implemented
>> compression
>>>>> in hdfs (this is not RAID).
>>>>>
>>>>> I met with developers / managers from Taobao, Huawei, Renren
>> (Chinese
>>>>> version of Facebook), etc and shared experience using HBase.
>>>>>
>>>>> You can find the complete listing of talks here:
>>>>> http://hic2011.hadooper.cn/dct/page/65580
>>>>>
>>>>> On my way back, I met Doug Cutting at the airport.
>>>>> He endorsed the idea of promoting committers from China so that
>> hadoop
>>>>> community would have organic growth.
>>>>>
>>>>> I didn't take detailed notes. So please find the slides from
>> hadooper.cn
>>>>>
>>>>> Thanks
>>>>
>>>>
>>>>
>>>> --
>>>> Todd Lipcon
>>>> Software Engineer, Cloudera
>>>>
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
