[
https://issues.apache.org/jira/browse/HUDI-2177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17389046#comment-17389046
]
ASF GitHub Bot commented on HUDI-2177:
--------------------------------------
nsivabalan commented on a change in pull request #3315:
URL: https://github.com/apache/hudi/pull/3315#discussion_r678628690
##########
File path:
hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/RealtimeSplit.java
##########
@@ -34,49 +36,73 @@
*/
public interface RealtimeSplit extends InputSplitWithLocationInfo {
+
/**
* Return Log File Paths.
+ *
* @return
*/
List<String> getDeltaLogPaths();
/**
* Return Max Instant Time.
+ *
* @return
*/
String getMaxCommitTime();
/**
* Return Base Path of the dataset.
+ *
* @return
*/
String getBasePath();
+ /**
+ * Returns Virtual key info if meta fields are disabled.
+ * @return
+ */
+ Option<HoodieVirtualKeyInfo> getHoodieVirtualKeyInfoOpt();
+
/**
* Update Log File Paths.
+ *
* @param deltaLogPaths
*/
void setDeltaLogPaths(List<String> deltaLogPaths);
/**
* Update Maximum valid instant time.
+ *
* @param maxCommitTime
*/
void setMaxCommitTime(String maxCommitTime);
/**
* Set Base Path.
+ *
* @param basePath
*/
void setBasePath(String basePath);
+ void setHoodieVirtualKeyInfoOpt(Option<HoodieVirtualKeyInfo>
hoodieVirtualKeyInfoOpt);
+
default void writeToOutput(DataOutput out) throws IOException {
InputSplitUtils.writeString(getBasePath(), out);
InputSplitUtils.writeString(getMaxCommitTime(), out);
out.writeInt(getDeltaLogPaths().size());
for (String logFilePath : getDeltaLogPaths()) {
InputSplitUtils.writeString(logFilePath, out);
}
+ if (!getHoodieVirtualKeyInfoOpt().isPresent()) {
+ InputSplitUtils.writeString(HoodieRealtimeInputFormatUtils.NULL_STR,
out);
Review comment:
@vinothchandar : also, is there some other elegant way to do this. I can
do a boolean. but not sure if that has any major benefit over this. In
InputSpltiUtils, we have apis for only string. And so chose this way. But open
to setting 1 byte and decoding it. my understanding is that, these are not
stored in disk as such for us to worry about storage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Virtual keys support for Compaction
> -----------------------------------
>
> Key: HUDI-2177
> URL: https://issues.apache.org/jira/browse/HUDI-2177
> Project: Apache Hudi
> Issue Type: Sub-task
> Components: Writer Core
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Virtual keys support for Compaction
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
