[
https://issues.apache.org/jira/browse/HUDI-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17410946#comment-17410946
]
ASF GitHub Bot commented on HUDI-2395:
--------------------------------------
xushiyan commented on pull request #3595:
URL: https://github.com/apache/hudi/pull/3595#issuecomment-914038407
> In general, I want to make sure the new methods added to HoodieTestTable
are in line with its design. @xushiyan any comments on that?
@vinothchandar @nsivabalan At the beginning we were thinking make
`HoodieTestTable` provide concise APIs doing basic table operations with empty
files and `HoodieWriteableTestTable` writing actual data. `FileCreateUtils` is
used by `HoodieTestTable` at lower level interfacing with the actual files.
When more and more APIs are introduced, we have to re-write many logics and
re-implement action-to-file-change translations. This resulted in the testutils
having cumbersome code and learning hurdle, also error-prone. I felt some
re-design is needed.
Some thoughts on the re-design: developers are familiar with HoodieXXXClient
so we would need `HoodieTestTable` make use of some client with similar public
APIs, say `HoodieTestTableClient`.
- HoodieTestTable owns a HoodieTestTableClient and exposes the client's APIs
for dev to prep their own data
- HoodieTestTableClient should manipulate timeline and metadata as normal
- For UTs don't need real data, the client writes out empty log and base
files.
- For FTs, the test table and client should be aware of the EngineContext
set by `XXXFunctionalTestHarness` and act like the actual clients
- Eventually dev only learn APIs provided by `HoodieTestTableClient` to prep
their tests. It can include high-level APIs like
`.write100RecordsIn3Partitions()` as such.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> Make metadata tests lean and consistent
> ---------------------------------------
>
> Key: HUDI-2395
> URL: https://issues.apache.org/jira/browse/HUDI-2395
> Project: Apache Hudi
> Issue Type: Sub-task
> Components: Testing
> Reporter: sivabalan narayanan
> Priority: Major
> Labels: pull-request-available
>
> Make metadata tests lean and consistent using HoodieTestTable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
