[
https://issues.apache.org/jira/browse/HUDI-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17396542#comment-17396542
]
ASF GitHub Bot commented on HUDI-2101:
--------------------------------------
xiarixiaoyao commented on pull request #3330:
URL: https://github.com/apache/hudi/pull/3330#issuecomment-895843948
@satishkotha @leesf
update this pr
1. add javadoc for UnsafeAccess.java
2. remove SparkOptimizeDataLayoutStrategy, and introduce a new sort for
z-order/hilbert which can be used by SparkSortAndSizeExecutionStrategy
3. Abstract updateStatistics method for HoodieTable to make it easily extend
to future usecases. now this method only support update z-order statistics
info.
4. we support update index table which generate by z-order. since the index
table is small, we update the index table by using full out join method to
overwrite index table, which has a better reading and writing efficiency than
use hoodie table
from my view, i think we can keep the OPTIMIZE operation Type in hudi. since
use api is more flexible than cluster operation,
we can optimize table by zorder/hilbert by dataFrame api directly and
specify some conditions,
dataFrame.optimize(Seq("begin_lat", "begin_lon"), options, "p > 10",
outputFileNum = 4): optimize all the partitions which partition value > 10
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> support z-order for hudi
> ------------------------
>
> Key: HUDI-2101
> URL: https://issues.apache.org/jira/browse/HUDI-2101
> Project: Apache Hudi
> Issue Type: Sub-task
> Components: Spark Integration
> Reporter: tao meng
> Assignee: tao meng
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.10.0
>
>
> support z-order for hudi to optimze the query
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
