[
https://issues.apache.org/jira/browse/HIVE-26939?focusedWorklogId=839083&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839083
]
ASF GitHub Bot logged work on HIVE-26939:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Jan/23 13:37
Start Date: 13/Jan/23 13:37
Worklog Time Spent: 10m
Work Description: amanraj2520 commented on PR #3941:
URL: https://github.com/apache/hive/pull/3941#issuecomment-1381860074
Hi @abstractdog
https://github.com/apache/hadoop/blob/a3b9c37a397ad4188041dd80621bdeefc46885f2/hadoop-project/pom.xml#L73
If you see Hadoop 3.3.1 uses 2.10.x databind, in which I think this issue
will also come while bringing up LLAP AM. Since I think jackson has modified
this case-sensitive comparison from 2.9. Anyhow it is better to change it to
node since Hadoop expects it as [node,rack]
Issue Time Tracking
-------------------
Worklog Id: (was: 839083)
Time Spent: 1h 10m (was: 1h)
> Hive LLAP Application Master fails to come up with Hadoop 3.3.4
> ---------------------------------------------------------------
>
> Key: HIVE-26939
> URL: https://issues.apache.org/jira/browse/HIVE-26939
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Reporter: Aman Raj
> Assignee: Aman Raj
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When current oss master hive tries to bring up the LLAP Application Master,
> it fails with this issue :
> {code:java}
> Executing the launch command\nINFO client.ServiceClient: Loading service
> definition from local FS:
> /var/lib/ambari-agent/tmp/llap-yarn-service_2023-01-10_07-56-46/Yarnfile\nERROR
> utils.JsonSerDeser: Exception while parsing json input
> stream\ncom.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot
> deserialize value of type
> `org.apache.hadoop.yarn.service.api.records.PlacementScope` from String
> \"NODE\": not one of the values accepted for Enum class: [node, rack]\n at
> [Source: (org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream);
> line: 31, column: 22] (through reference chain:
> org.apache.hadoop.yarn.service.api.records.Service[\"components\"]->java.util.ArrayList[0]->org.apache.hadoop.yarn.service.api.records.Component[\"placement_policy\"]->org.apache.hadoop.yarn.service.api.records.PlacementPolicy[\"constraints\"]->java.util.ArrayList[0]->org.apache.hadoop.yarn.service.api.records.PlacementConstraint[\"scope\"])\n\tat
>
> com.fasterxml.jackson.databind.exc.InvalidFormatException.from(InvalidFormatException.java:67)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.DeserializationContext.weirdStringException(DeserializationContext.java:1851)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.DeserializationContext.handleWeirdStringValue(DeserializationContext.java:1079)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.std.EnumDeserializer._deserializeAltString(EnumDeserializer.java:339)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.std.EnumDeserializer._fromString(EnumDeserializer.java:214)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.std.EnumDeserializer.deserialize(EnumDeserializer.java:188)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:324)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:355)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:324)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:324)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:355)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:324)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3585)
> ~[jackson-databind-2.12.7.jar:2.12.7]\n\tat
> org.apache.hadoop.yarn.service.utils.JsonSerDeser.fromStream(JsonSerDeser.java:137)
> [hadoop-yarn-services-core-3.3.4.5.1-SNAPSHOT.jar:?]\n\tat
> org.apache.hadoop.yarn.service.utils.JsonSerDeser.load(JsonSerDeser.java:179)
> [hadoop-yarn-services-core-3.3.4.5.1-SNAPSHOT.jar:?]\n\tat
> org.apache.hadoop.yarn.service.client.ServiceClient.loadAppJsonFromLocalFS(ServiceClient.java:208)
> [hadoop-yarn-services-core-3.3.4.5.1-SNAPSHOT.jar:?]\n\tat
> org.apache.hadoop.yarn.service.client.ServiceClient.actionLaunch(ServiceClient.java:546)
> [hadoop-yarn-services-core-3.3.4.5.1-SNAPSHOT.jar:?]\n\tat
> org.apache.hadoop.hive.llap.cli.LlapSliderUtils.startCluster(LlapSliderUtils.java:102)
> [hive-llap-server-3.1.2.5.1-SNAPSHOT.jar:3.1.2.5.1-SNAPSHOT]\n\tat
> org.apache.hadoop.hive.llap.cli.LlapServiceDriver.run(LlapServiceDriver.java:601)
> [hive-llap-server-3.1.2.5.1-SNAPSHOT.jar:3.1.2.5.1-SNAPSHOT]\n\tat
> org.apache.hadoop.hive.llap.cli.LlapServiceDriver.main(LlapServiceDriver.java:119)
> [hive-llap-server-3.1.2.5.1-SNAPSHOT.jar:3.1.2.5.1-SNAPSHOT]\n {code}
> This is because llap_server uses templates.py yarnfile to bringup the LLAP
> Application Master which has the placement_policy as "NODE". Hadoop has
> started using 2.12.7 jackson version which has issues in parsing NODE to
> either [node, rack]. Therefore it is not able to cast the placement policy
> correctly.
>
> Resolution :
> We will have to send placement_policy as "node" from the templates.py of Hive
> Llap Server module. Will raise a PR for this issue :
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
