[20191210] Meeting Notes

2019-12-10 Thread vbal...@apache.org
Hello all, Please find the meeting minutes for today's weekly sync meeting in the link below https://cwiki.apache.org/confluence/display/HUDI/20191210+Weekly+Sync+Minutes Thanks, Balaji.V

Re: [DISCUSS] Scaling community support

2019-12-10 Thread Y Ethan Guo
Here are my two cents in addition to the great suggestions in the thread: I agree with @Sivabalan that folks in Hudi community have different levels of expertise and amount of effort to put in the community. So in general, it may be good to have PoCs or contributors for each area in Hudi, e.g., i

Re: [DISCUSS] Scaling community support

2019-12-10 Thread leesf
Agree with @Sivabalan that any person who is interested in PRs would help to review, I think assign PR to someone just indicates that he/she is responsible for the progress as the lead person. Anyone is highly appreciated and encouraged to review PRs if they are interested. Best, Leesf Sivabalan

Re: [DISCUSS] Scaling community support

2019-12-10 Thread Sivabalan
IMO, open source community has folks with different needs and interests. For eg: “pro active” folks like you and Balaji who is very active in all aspects of the project. 2nd category is “selectively active” people, who will have good knowledge in few code blocks like Sudha in case of presto and so

Re:Re: [DISCUSS] Refactor of the configuration framework of hudi project

2019-12-10 Thread lamberken
Hi, @Balaji @Vinoth I'm sorry, some places are not very clear, 1, We can see that HoodieMetricsConfig, HoodieStorageConfig, etc.. already defined in project. But we get property value by methods which defined in HoodieWriteConfig, like HoodieWriteConfig#getParquetMaxFileSize, HoodieWr

Re: [DISCUSS] Refactor of the configuration framework of hudi project

2019-12-10 Thread Balaji Varadarajan
Hi Lamber-Ken,  Thanks for the time writing the proposal and thinking about improving Hudi usability. My preference would be to keep the Builder pattern when configuring. It is something I find it natural when configuring. It is not clear to me whether there is any external facing changes which

Re: [DISCUSS] Scaling community support

2019-12-10 Thread vbal...@apache.org
Regarding (1), I support the "on-call" model for answering to dev@ emails, triaging GH and Jira. This would help reduce context-switch for the contributor community as a whole. Also, Trying to answer questions about Hudi is a good way to ramp up on the internals of it.  A 2 day schedule would

Re: Re: [DISCUSS] Refactor of the configuration framework of hudi project

2019-12-10 Thread Vinoth Chandar
Hi , Thanks for the proposal. Some parts I agree, some parts I don't and some parts are unclear Agree : - On introducing a class that binds key, default value, provided value, and also may be a doc along with it (?). - Designing the framework to have fallback keys is good IMO. It helps us do thin

Re:Re: [DISCUSS] Refactor of the configuration framework of hudi project

2019-12-10 Thread lamberken
Hi, vino Reasonable, we can refactor this step by step. The first step now is to introduce the config framework. When our community is familiar with the config framework mechanism, it's easy to integrate FallbackKey in the future. Best, lamber-ken At 2019-12-10 11:51:22, "vino yang" w