As Hudi matures as a project, we need to get our devX and test infra rock solid. Availability of test utils and base classes for ease of writing more tests, stable integration tests, ease of debuggability, micro benchmarks, performance test infra, automating checkstyle formatting, nightly snapshot builds and so on.
We have identified and categorized these into different areas as below. - Test fixes and some clean up. // There are a lot of jira tickets lying around in this section. - Test refactoring. // For ease of development, and reduce clutter, we need to work on refactoring test infra like having more test utils, base classes etc. - More tests to improve coverage in some areas. - CI stability and ease of debugging integration tests. - Checkstyle, sl4j, warnings, spotless, etc. - Micro benchmarks. // add benchmarking framework to hudi. and then identify regressions on any key paths. - Long running test suite - Config clean ups in hudi client - Perf test environment - Nightly builds As we plan out work in each of these sections, we are looking for help from the community in getting these done. Plan is to put together a few umbrella tickets for each of these areas and will have a coordinator. Coordinator will be one who has expertise in the area of interest. Coordinator will plan out the work in their resp area and will help drive the initiative with help from the community depending on who volunteers to help out. I understand the list is huge. Some work areas will be well defined and should be able to get it done if we allocate enough time and resources. But some are exploratory in nature and need some initial push to get the ball rolling. Very likely some of the work items in these would be well defined and should be easy for new folks to contribute. We are not really having any target timeframe in mind(as we had 1 month for bug bash), but would like to get concrete work items done in decent time and have others ready by the next major release(for eg, perf test env) depending on resources. Let us know if you would be interested to help our community in this regard. -- Regards, -Sivabalan
