walterddr commented on a change in pull request #7839:
URL: https://github.com/apache/pinot/pull/7839#discussion_r766919103
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/common/datatable/DataTableImplV2.java
##########
@@ -188,10 +193,13 @@ public void addException(ProcessingException
processingException) {
}
// Write metadata.
- dataOutputStream.writeInt(dataOffset);
- byte[] metadataBytes = serializeMetadata();
- dataOutputStream.writeInt(metadataBytes.length);
- dataOffset += metadataBytes.length;
+ byte[] metadataBytes = null;
Review comment:
good catch! made it write 0 length metadataBytes
##########
File path:
pinot-common/src/main/java/org/apache/pinot/common/utils/DataTable.java
##########
@@ -35,9 +35,16 @@
void addException(ProcessingException processingException);
+ void addException(int exceptionCode, String exceptionMsg);
+
Map<Integer, String> getExceptions();
- byte[] toBytes()
+ default byte[] toBytes()
+ throws IOException {
+ return toBytes(false);
+ }
+
+ byte[] toBytes(boolean isStrippingMetadata)
Review comment:
added the stripMetadata flag to streaming response utils
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/common/datatable/DataTableUtils.java
##########
@@ -107,6 +108,34 @@ public static DataTable buildEmptyDataTable(QueryContext
queryContext)
}
}
+ public static DataTable buildMetadataOnlyDataTable(DataTable origDataTable) {
Review comment:
i can move this into DataTable as `dataTable.toMetadataOnlyDataTable()`
but I will still replicate and produce a copy. is this what you had in mind?
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/common/datatable/DataTableUtils.java
##########
@@ -107,6 +108,34 @@ public static DataTable buildEmptyDataTable(QueryContext
queryContext)
}
}
+ public static DataTable buildMetadataOnlyDataTable(DataTable origDataTable) {
Review comment:
oh! sounds good. if we guarantee that the data inside `DataTable` is
immutable. I can definitely go with this path!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
