[
https://issues.apache.org/jira/browse/HUDI-8416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sagar Sumit updated HUDI-8416:
------------------------------
Description:
The tests in TestHoodieMultipleBaseFileFormat do not generate base files of
different file formats even if "hoodie.table.multiple.base.file.formats.enable"
is set to true in Spark datasource write.
The test above does the following:
1. Insert some records in one partition with parquet format.
2. Insert some records in another partition with orc format.
3. Snapshot load of the table.
4. One round of update to generate new file slice.
5. Snapshot read.
We should look into why base file in other format is not being generated.
was:The tests in TestHoodieMultipleBaseFileFormat do not generate base files
of different file formats even if
"hoodie.table.multiple.base.file.formats.enable" is set to true in Spark
datasource write.
> Multiple base file formats cannot be turned on by the config
> ------------------------------------------------------------
>
> Key: HUDI-8416
> URL: https://issues.apache.org/jira/browse/HUDI-8416
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Y Ethan Guo
> Assignee: Lin Liu
> Priority: Blocker
> Fix For: 1.0.0
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> The tests in TestHoodieMultipleBaseFileFormat do not generate base files of
> different file formats even if
> "hoodie.table.multiple.base.file.formats.enable" is set to true in Spark
> datasource write.
> The test above does the following:
> 1. Insert some records in one partition with parquet format.
> 2. Insert some records in another partition with orc format.
> 3. Snapshot load of the table.
> 4. One round of update to generate new file slice.
> 5. Snapshot read.
> We should look into why base file in other format is not being generated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
