rahil-c commented on code in PR #5786:
URL: https://github.com/apache/hudi/pull/5786#discussion_r917450559
##########
pom.xml:
##########
@@ -1669,15 +1762,17 @@
<fasterxml.jackson.dataformat.yaml.version>${fasterxml.spark3.version}
</fasterxml.jackson.dataformat.yaml.version>
<skip.hudi-spark2.unit.tests>true</skip.hudi-spark2.unit.tests>
- <skipITs>true</skipITs>
+ <skipITs>false</skipITs>
</properties>
<modules>
<module>hudi-spark-datasource/hudi-spark3</module>
<module>hudi-spark-datasource/hudi-spark3-common</module>
</modules>
<activation>
+ <activeByDefault>true</activeByDefault>
<property>
<name>spark3</name>
+ <value>!disabled</value>
Review Comment:
This is still an ongoing issue and possibly might be a bug with `mvn`. For
example in current hudi master branch we have this value tag in spark 2 profile
https://github.com/apache/hudi/blob/master/pom.xml#L1645. If you see the
comment it will add the modules for `spark2` to all profiles(even if you
specified spark3). This value tag is what allows these modules to be included
from my testing.
```
<module>hudi-spark-datasource/hudi-spark2</module>
<module>hudi-spark-datasource/hudi-spark2-common</module>
</modules>
```
but the issue is without this value tag above it will not include these two
modules.
For our pr we follow a similar approach but this causes an issue for spark2
if you dont also explicitly pass the scala version as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
