LiangliangSui commented on code in PR #1663:
URL: https://github.com/apache/incubator-fury/pull/1663#discussion_r1622450167
##########
java/fury-core/src/main/java/org/apache/fury/config/Config.java:
##########
@@ -192,6 +195,10 @@ public boolean isScopedMetaShareEnabled() {
return scopedMetaShareEnabled;
}
+ public MetaCompressor getMetaCompressor() {
Review Comment:
Could we add javadoc comments here?
##########
java/fury-core/src/main/java/org/apache/fury/meta/ClassDefEncoder.java:
##########
@@ -146,36 +135,63 @@ static MemoryBuffer encodeClassDef(
int currentClassHeader = (fields.size() << 1);
if (classResolver.isRegistered(type)) {
currentClassHeader |= 1;
- buffer.writeVarUint32Small7(currentClassHeader);
- buffer.writeVarUint32Small7(classResolver.getRegisteredClassId(type));
+ classDefBuf.writeVarUint32Small7(currentClassHeader);
+
classDefBuf.writeVarUint32Small7(classResolver.getRegisteredClassId(type));
} else {
- buffer.writeVarUint32Small7(currentClassHeader);
+ classDefBuf.writeVarUint32Small7(currentClassHeader);
Class<?> currentType = getType(type, className);
Tuple2<String, String> encoded =
Encoders.encodePkgAndClass(currentType);
- writePkgName(buffer, encoded.f0);
- writeTypeName(buffer, encoded.f1);
+ writePkgName(classDefBuf, encoded.f0);
+ writeTypeName(classDefBuf, encoded.f1);
}
- writeFieldsInfo(buffer, fields);
+ writeFieldsInfo(classDefBuf, fields);
+ }
+ byte[] compressed =
+ classResolver
+ .getFury()
Review Comment:
I had an idea when I was looking at the code recently.
Could `org.apache.fury.config.Config` be set to
`org.apache.fury.resolver.SerializationContext`?
I think it is more reasonable to get `Config` from `SerializationContext`
during runtime. It feels a bit weird to get it from `Fury` object.
This will reduce the burden on the `Fury` class.
##########
java/fury-core/src/main/java/org/apache/fury/config/FuryBuilder.java:
##########
@@ -251,6 +254,37 @@ public FuryBuilder withScopedMetaShare(boolean scoped) {
return this;
}
+ /**
+ * Set a compressor for meta compression. Note that the passed {@link
MetaCompressor} should be
+ * thread-safe. By default, a `Deflater` based compressor {@link
DeflaterMetaCompressor} will be
+ * used. Users can pass other compressor such as `zstd` for better
compression rate.
+ */
+ public FuryBuilder withMetaCompressor(MetaCompressor metaCompressor) {
+ Class<?> clz = metaCompressor.getClass();
+ if (clz != DeflaterMetaCompressor.class) {
+ while (clz != null) {
+ try {
+ clz.getDeclaredMethod("hashCode");
+ if (clz == Object.class) {
+ LOG.warn(
+ "{} should implement equals/hashCode method, "
+ + "otherwise compile cache may won't work. "
+ + "Use type to check MetaCompressor identity instead, but
this"
+ + "may be incorrect if different compressor instance of
same type "
+ + "indicates different compressor.",
+ metaCompressor);
+ metaCompressor =
MetaCompressor.typeEqualMetaCompressor(metaCompressor);
+ }
+ break;
+ } catch (NoSuchMethodException e) {
+ clz = clz.getSuperclass();
+ }
+ }
Review Comment:
Could this part of the logic be moved to `MetaCompressor`? This is strongly
related to `MetaCompressor` rather than `FuryBuilder`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
