[
https://issues.apache.org/jira/browse/GOBBLIN-699?focusedWorklogId=213391&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-213391
]
ASF GitHub Bot logged work on GOBBLIN-699:
------------------------------------------
Author: ASF GitHub Bot
Created on: 14/Mar/19 20:17
Start Date: 14/Mar/19 20:17
Worklog Time Spent: 10m
Work Description: htran1 commented on pull request #2570: [GOBBLIN-699]
Orc compaction impl.
URL: https://github.com/apache/incubator-gobblin/pull/2570#discussion_r265748952
##########
File path:
gobblin-compaction/src/main/java/org/apache/gobblin/compaction/mapreduce/orc/OrcKeyCompactorOutputFormat.java
##########
@@ -39,4 +49,22 @@ public synchronized OutputCommitter
getOutputCommitter(TaskAttemptContext contex
}
return this.committer;
}
+
+ /**
+ * Required for extension since super method hard-coded file extension as
".orc". To keep flexibility
+ * of extension name, we made it configuration driven.
+ * @param taskAttemptContext The source of configuration that determines the
file extension
+ * @return The {@link RecordWriter} that write out Orc object.
+ * @throws IOException
+ */
+ @Override
+ public RecordWriter getRecordWriter(TaskAttemptContext taskAttemptContext)
throws IOException {
+ Configuration conf = taskAttemptContext.getConfiguration();
+ String extension = "." + conf.get(COMPACTION_OUTPUT_EXTENSION, "orc" );
Review comment:
Do you think empty extension should be supported?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 213391)
Time Spent: 1h 10m (was: 1h)
> ORC Compaction Implementation
> -----------------------------
>
> Key: GOBBLIN-699
> URL: https://issues.apache.org/jira/browse/GOBBLIN-699
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Lei Sun
> Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
