[
https://issues.apache.org/jira/browse/HUDI-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Raymond Xu updated HUDI-2818:
-----------------------------
Description:
{noformat}
21/11/22 05:26:10 INFO UpgradeDowngrade: Attempting to move table from version
ONE to THREE
21/11/22 05:26:10 ERROR HoodieDeltaStreamer: Shutting down delta-sync due to
exception
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:460)
at java.util.Properties.setProperty(Properties.java:166)
at org.apache.hudi.common.config.HoodieConfig.setValue(HoodieConfig.java:58)
at
org.apache.hudi.table.upgrade.UpgradeDowngrade.run(UpgradeDowngrade.java:132){noformat}
2to3 looks for config
{noformat}
hoodie.table.keygenerator.class{noformat}
and will result in putting null value to hashtable if user only sets
{noformat}
hoodie.datasource.write.keygenerator.class{noformat}
solution:
2to3 should check both props to set. And use {{hashtable}} in upgrade/downgrade
handlers instead of hashmap. in case of getting null value, hashtable will
error out earlier.
was:
21/11/22 05:26:10 INFO UpgradeDowngrade: Attempting to move table from version
ONE to THREE
21/11/22 05:26:10 ERROR HoodieDeltaStreamer: Shutting down delta-sync due to
exception
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:460)
at java.util.Properties.setProperty(Properties.java:166)
at
org.apache.hudi.common.config.HoodieConfig.setValue(HoodieConfig.java:58)
at
org.apache.hudi.table.upgrade.UpgradeDowngrade.run(UpgradeDowngrade.java:132)
2to3 looks for config
hoodie.table.keygenerator.class
and will result in putting null value to hashtable if user only sets
hoodie.datasource.write.keygenerator.class
solution:
2to3 should check both props to set. And use {{hashtable}} in upgrade/downgrade
handlers instead of hashmap. in case of getting null value, hashtable will
error out earlier.
> 2to3 upgrade only checks hoodie.table.keygenerator.class
> --------------------------------------------------------
>
> Key: HUDI-2818
> URL: https://issues.apache.org/jira/browse/HUDI-2818
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Raymond Xu
> Assignee: Raymond Xu
> Priority: Blocker
> Fix For: 0.10.0
>
>
> {noformat}
> 21/11/22 05:26:10 INFO UpgradeDowngrade: Attempting to move table from
> version ONE to THREE
> 21/11/22 05:26:10 ERROR HoodieDeltaStreamer: Shutting down delta-sync due to
> exception
> java.lang.NullPointerException
> at java.util.Hashtable.put(Hashtable.java:460)
> at java.util.Properties.setProperty(Properties.java:166)
> at org.apache.hudi.common.config.HoodieConfig.setValue(HoodieConfig.java:58)
> at
> org.apache.hudi.table.upgrade.UpgradeDowngrade.run(UpgradeDowngrade.java:132){noformat}
>
> 2to3 looks for config
> {noformat}
> hoodie.table.keygenerator.class{noformat}
> and will result in putting null value to hashtable if user only sets
> {noformat}
> hoodie.datasource.write.keygenerator.class{noformat}
>
> solution:
> 2to3 should check both props to set. And use {{hashtable}} in
> upgrade/downgrade handlers instead of hashmap. in case of getting null value,
> hashtable will error out earlier.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
