[
https://issues.apache.org/jira/browse/HUDI-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan resolved HUDI-2494.
---------------------------------------
Resolution: Fixed
> Fix usage of different key generators with metadata enabled
> -----------------------------------------------------------
>
> Key: HUDI-2494
> URL: https://issues.apache.org/jira/browse/HUDI-2494
> Project: Apache Hudi
> Issue Type: Sub-task
> Components: Spark Integration
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Major
> Labels: oct18_2021, pull-request-available, sev:critical
> Fix For: 0.10.0
>
>
> With [sync metadata patch|https://github.com/apache/hudi/pull/3590/], when
> metadata is enabled by default, some spark datasource tests failed which were
> using timestamp based key gen and custom key gen. Metadata table's records
> are getting picked up when we do
>
> {code:java}
> spark.read.format(hudi).load(basePath + "/*/*")
> {code}
>
> For now, I have disabled metadata for these tests.
> testSparkPartitonByWithTimestampBasedKeyGenerator
> testSparkPartitonByWithCustomKeyGenerator
>
> I was looking at
> [options|https://spark.apache.org/docs/latest/sql-data-sources-generic-options.html]
> to ignore certain path, but looks like there is none.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
