xiarixiaoyao edited a comment on pull request #3330:
URL: https://github.com/apache/hudi/pull/3330#issuecomment-954487215
@vinothchandar thanks for your suggestion.
_the question:Side note: At-least from IDE builds, I get the following error
when running tests. did you fae this? have you tried running this on a cluster
using a spark bundle?_
ans: i use java 8 to run those code , of course i test those code in
cluster. but i find if we uses java 11 to run those code in IDE, IDE will
throw "package sun.misc. does not exist", since Packages com.sun.* and sun.*
hold internal stuff, and should not be used by thirdparty applications (like
yours probably) in general case. now i removed all unsafe function, it should
be ok for java 11.
update the code.
**1. rewrite most of scala functions(ZOptimze.scala) by java in
ZCurveOptimizeHelper.java and remove ZOptimze.scala
2. remove unsafe lexicographical cmp and use java lexicographical cmp. now
no unsafe class exisit.
3. dataskipping should work with sql and datasource, not just datasource.
4. Introduce a new conf ENABLE_DATA_SKIPPING in DataSourceReadOptions to
protect the query side changes.**
i think we may no need to move data skipping config to
DataSourceReadOptions. we can introduce a config in DataSourceReadOptions.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
