[
https://issues.apache.org/jira/browse/HIVE-28010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17809768#comment-17809768
]
yongzhi.shao edited comment on HIVE-28010 at 1/23/24 7:18 AM:
--------------------------------------------------------------
[~zhangbutao] :
# For serializing/deserializing Hive records, Hive has its own custom serde.
mabye we can using fury.
# Tez shuffle using protobuf, we can using fury too.
# Kryo is used to serializing/deserializing Hive operators. This can be
replaced by Fury
Since fury's performance is currently much higher than protobuf/kyro, we can
fully expect a performance boost from fury.
For Example:
[[Flink] Optimize CDC sink serde with Fury by xuchen-plus · Pull Request #307 ·
lakesoul-io/LakeSoul|https://github.com/lakesoul-io/LakeSoul/pull/307]
was (Author: lisoda):
[~zhangbutao] :
# For serializing/deserializing Hive records, Hive has its own custom serde.
mabye we can using fury.
# Tez shuffle using protobuf, we can using fury too.
# Kryo is used to serializing/deserializing Hive operators. This can be
replaced by Fury
Since fury's performance is currently much higher than protobuf/kyro, we can
fully expect a performance boost from fury.
> Using apache fury instead of kyro/protubuf
> ------------------------------------------
>
> Key: HIVE-28010
> URL: https://issues.apache.org/jira/browse/HIVE-28010
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Affects Versions: 4.0.0
> Reporter: yongzhi.shao
> Priority: Minor
>
> APACHE FURY is a new serialisation framework that can significantly improve
> serialisation/deserialisation performance compared to Kyro and Protobuf. Do
> we need Fury in HIVE?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
