Rahil Chertara created HUDI-5714:
------------------------------------
Summary: Hudi CLI Bundle Spark 2 Issues
Key: HUDI-5714
URL: https://issues.apache.org/jira/browse/HUDI-5714
Project: Apache Hudi
Issue Type: Task
Reporter: Rahil Chertara
Currently it seems that when trying the cli bundle with spark2, the cli faces
the following issue.
```
is java.lang.NoSuchMethodError:
javax.validation.BootstrapConfiguration.getClockProviderClassName()Ljava/lang/String;
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1804)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
at
org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at
org.springframework.boot.SpringApplication.refresh(SpringApplication.java:734)
at
org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:308)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1295)
at org.apache.hudi.cli.Main.main(Main.java:35)
```
This issue is not present on spark3, since the cli bundle contains jakarta
deps, and spark 3 jars contains the corresponding jakarta validation jar. The
problem is spark2 does not provide the jakarta and instead provides the javax
version of this jar which is incorrect. From what ive seen it seems both types
of these jars should not be present on class path.
[https://stackoverflow.com/questions/58878713/spring-boot-application-fails-to-start-with-javax-validation-bootstrapconfigurat]
The workaround is moving the javax validation api jar under spark2 jars to a
temp folder. After this a gson exception occurs with spark2, the only
unofficial "workaround" ive found is moving the spark2 gson also to a temp
folder (which is not ideal) to get the hudi cli bundle to start with spark2
Well need to revisit this at later point For now spark 3.3. and spark 3.2
should work
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
