[
https://issues.apache.org/jira/browse/GOBBLIN-1212?focusedWorklogId=459570&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-459570
]
ASF GitHub Bot logged work on GOBBLIN-1212:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 15/Jul/20 23:25
Start Date: 15/Jul/20 23:25
Worklog Time Spent: 10m
Work Description: sv2000 commented on a change in pull request #3060:
URL: https://github.com/apache/incubator-gobblin/pull/3060#discussion_r455421107
##########
File path:
gobblin-compaction/src/main/java/org/apache/gobblin/compaction/mapreduce/orc/OrcUtils.java
##########
@@ -491,18 +490,38 @@ public static WritableComparable
createValueRecursively(TypeDescription schema)
*/
public static boolean eligibleForUpConvertHelper(TypeDescription
originalSchema, TypeDescription targetSchema) {
if (!targetSchema.getCategory().isPrimitive()) {
- if
(!originalSchema.getFieldNames().containsAll(targetSchema.getFieldNames())) {
- return false;
- }
- boolean result = true;
+ if (targetSchema.getCategory().equals(TypeDescription.Category.LIST)) {
+ Preconditions
+ .checkArgument(originalSchema.getChildren() != null, "Illegal
format of ORC schema as:" + targetSchema);
+ return eligibleForUpConvertHelper(originalSchema.getChildren().get(0),
targetSchema.getChildren().get(0));
+ } else if
(targetSchema.getCategory().equals(TypeDescription.Category.MAP)) {
+ Preconditions
Review comment:
Should the error message be: "Illegal format of ORC schema as:" +
originalSchema?
##########
File path:
gobblin-compaction/src/test/java/org/apache/gobblin/compaction/mapreduce/orc/OrcUtilsTest.java
##########
@@ -294,6 +294,21 @@ public void testOrcStructProjection() throws Exception {
Assert.assertEquals(projectColumnStruct, projectedStructExpectedValue);
}
+ @Test
+ public void complextTypeEligibilityCheck() throws Exception {
Review comment:
minor typo: complextType -> complexType.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 459570)
Time Spent: 40m (was: 0.5h)
> Handle shuffle-key eligibility check properly
> ----------------------------------------------
>
> Key: GOBBLIN-1212
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1212
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Lei Sun
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
