xiangfu0 commented on code in PR #13905:
URL: https://github.com/apache/pinot/pull/13905#discussion_r1737782739
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java:
##########
@@ -633,6 +647,55 @@ public static void
addColumnMetadataInfo(PropertiesConfiguration properties, Str
}
}
+ /**
+ * In order to persist complex field metadata, we need to recursively add
child field specs
+ * So, each complex field spec will have a property for its child field
names and each child field will have its
+ * own properties of the detailed field spec.
+ * E.g. a COMPLEX type `intMap` of Map<String, Integer> has 2 child fields:
+ * - key in STRING type and value in INT type.
+ * Then we will have the following properties to define a COMPLEX field:
+ * column.intMap.childFieldNames = [key, value]
+ * column.intMap$$key.columnType = DIMENSION
+ * column.intMap$$key.dataType = STRING
+ * column.intMap$$key.isSingleValued = true
+ * column.intMap$$value.columnType = DIMENSION
+ * column.intMap$$value.dataType = INT
+ * column.intMap$$value.isSingleValued = true
+ */
+ public static void addFieldSpec(PropertiesConfiguration properties, String
column, FieldSpec fieldSpec) {
+ properties.setProperty(getKeyFor(column, COLUMN_TYPE),
String.valueOf(fieldSpec.getFieldType()));
+ if (!column.equals(fieldSpec.getName())) {
+ properties.setProperty(getKeyFor(column, COLUMN_NAME),
String.valueOf(fieldSpec.getName()));
Review Comment:
Yes for the childFieldSpec we could configure a String DimensionSpec called
`key`, then during the persistent, the column name is `stringMap$$key`.
Btw, this is only used for ColumnMetadataImpl persist serde.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
