[
https://issues.apache.org/jira/browse/HUDI-4967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-4967:
----------------------------
Description:
Related fix: HUDI-4966
We need to add docs on how to properly set the meta sync configuration,
especially the hoodie.datasource.hive_sync.partition_value_extractor, in
[https://hudi.apache.org/docs/key_generation] (for different Hudi versions, the
config can be different). Check the ticket above and PR description of
[https://github.com/apache/hudi/pull/6851] for more details.
We should also add the migration setup on the key generation page as well:
[https://hudi.apache.org/releases/release-0.12.0/#configuration-updates]
* {{{}hoodie.datasource.hive_sync.partition_value_extractor{}}}: This config
is used to extract and transform partition value during Hive sync. Its default
value has been changed from {{SlashEncodedDayPartitionValueExtractor}} to
{{{}MultiPartKeysValueExtractor{}}}. If you relied on the previous default
value (i.e., have not set it explicitly), you are required to set the config to
{{{}org.apache.hudi.hive.SlashEncodedDayPartitionValueExtractor{}}}. From this
release, if this config is not set and Hive sync is enabled, then partition
value extractor class will be *automatically inferred* on the basis of number
of partition fields and whether or not hive style partitioning is enabled.
was:
Related fix: HUDI-4966
We need to add docs on how to properly set the meta sync configuration,
especially the hoodie.datasource.hive_sync.partition_value_extractor, in
[https://hudi.apache.org/docs/key_generation] (for different Hudi versions, the
config can be different).
We should also add the migration setup on the key generation page as well:
[https://hudi.apache.org/releases/release-0.12.0/#configuration-updates]
* {{{}hoodie.datasource.hive_sync.partition_value_extractor{}}}: This config
is used to extract and transform partition value during Hive sync. Its default
value has been changed from {{SlashEncodedDayPartitionValueExtractor}} to
{{{}MultiPartKeysValueExtractor{}}}. If you relied on the previous default
value (i.e., have not set it explicitly), you are required to set the config to
{{{}org.apache.hudi.hive.SlashEncodedDayPartitionValueExtractor{}}}. From this
release, if this config is not set and Hive sync is enabled, then partition
value extractor class will be *automatically inferred* on the basis of number
of partition fields and whether or not hive style partitioning is enabled.
> Improve docs for meta sync with TimestampBasedKeyGenerator
> ----------------------------------------------------------
>
> Key: HUDI-4967
> URL: https://issues.apache.org/jira/browse/HUDI-4967
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Ethan Guo
> Assignee: Jonathan Vexler
> Priority: Major
> Fix For: 0.12.2
>
>
> Related fix: HUDI-4966
> We need to add docs on how to properly set the meta sync configuration,
> especially the hoodie.datasource.hive_sync.partition_value_extractor, in
> [https://hudi.apache.org/docs/key_generation] (for different Hudi versions,
> the config can be different). Check the ticket above and PR description of
> [https://github.com/apache/hudi/pull/6851] for more details.
> We should also add the migration setup on the key generation page as well:
> [https://hudi.apache.org/releases/release-0.12.0/#configuration-updates]
> * {{{}hoodie.datasource.hive_sync.partition_value_extractor{}}}: This config
> is used to extract and transform partition value during Hive sync. Its
> default value has been changed from
> {{SlashEncodedDayPartitionValueExtractor}} to
> {{{}MultiPartKeysValueExtractor{}}}. If you relied on the previous default
> value (i.e., have not set it explicitly), you are required to set the config
> to {{{}org.apache.hudi.hive.SlashEncodedDayPartitionValueExtractor{}}}. From
> this release, if this config is not set and Hive sync is enabled, then
> partition value extractor class will be *automatically inferred* on the basis
> of number of partition fields and whether or not hive style partitioning is
> enabled.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
