vinishjail97 commented on issue #941:
URL: 
https://github.com/apache/incubator-xtable/issues/941#issuecomment-6534513354

   Thanks again for the detailed report, @salimTutuncu. Here is where this 
stands for 0.5.0.
   
   `xtable-spark-runtime` is now on `main` (#964) and is built against Hudi 
1.2.1. On `main` the Hudi sync works with Hudi 1.2.x on the classpath, and it 
reads tables written by Hudi 0.15.0 and 1.0.2 without upgrading them.
   
   On Glue 5.1, which includes Hudi 1.0.2, the sync job can use Hudi 1.2.1 as 
AWS documents under [Using a different Hudi 
version](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-hudi.html):
   
   - pass `xtable-spark-runtime` and `hudi-spark3.5-bundle_2.12-1.2.1.jar` with 
`--extra-jars`
   - set `--user-jars-first true`
   - leave `hudi` out of `--datalake-formats`
   
   The jobs that write the table can keep using Glue's built-in Hudi. We have 
validated these Hudi versions on Spark, but not on Glue itself, so feedback 
from a Glue run would be very welcome.
   
   #995 adds this setup to the Spark runtime page. As you suggested earlier, it 
would be good to close this issue against 0.5.0 once that docs PR lands.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to