[jira] [Commented] (TWILL-262) YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is removed from DFSUtils from hadoop-2.8
[
https://issues.apache.org/jira/browse/TWILL-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719621#comment-16719621
]
Hongyuan Li commented on TWILL-262:
---
Thanks for your review, Yim
> YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is
> removed from DFSUtils from hadoop-2.8
>
>
> Key: TWILL-262
> URL: https://issues.apache.org/jira/browse/TWILL-262
> Project: Apache Twill
> Issue Type: Bug
> Components: yarn
>Affects Versions: 0.8.0, 0.9.0, 0.11.0, 0.12.0, 0.13.0
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Major
> Fix For: 0.14.0
>
> Attachments: errors.txt
>
>
> YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is
> removed from DFSUtils from hadoop-2.8
> current code
> {code}
> public static void cloneHaNnCredentials(Configuration config) throws
> IOException {
> ……
> // Loop through all name services. Each name service could have multiple
> name node associated with it.
> for (Map.Entry> entry :
> DFSUtil.getHaNnRpcAddresses(config).entrySet()) {
> String nsId = entry.getKey();
> Map addressesInNN = entry.getValue();
> if (!HAUtil.isHAEnabled(config, nsId) || addressesInNN == null ||
> addressesInNN.isEmpty()) {
> continue;
> }
> ……
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TWILL-262) YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is removed from DFSUtils from hadoop-2.8
[
https://issues.apache.org/jira/browse/TWILL-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633305#comment-16633305
]
Hongyuan Li commented on TWILL-262:
---
seems other errorss follows.
2018-09-30T10:35:37,439Z ERROR o.a.t.i.y.AbstractYarnTwillService [node3]
[message-callback]
AbstractYarnTwillService:handleSecureStoreUpdate(AbstractYarnTwillService.java:110)
- Failed to update secure store.
java.lang.NoSuchMethodError:
org.apache.hadoop.hdfs.HAUtil.cloneDelegationTokenForLogicalUri(Lorg/apache/hadoop/security/UserGroupInformation;Ljava/net/URI;Ljava/util/Collection;)V
at
org.apache.twill.internal.yarn.YarnUtils.cloneHaNnCredentials(YarnUtils.java:222)
at
org.apache.twill.internal.yarn.AbstractYarnTwillService.handleSecureStoreUpdate(AbstractYarnTwillService.java:104)
at
org.apache.twill.internal.container.TwillContainerService.onReceived(TwillContainerService.java:107)
at
org.apache.twill.internal.AbstractTwillService.handleMessage(AbstractTwillService.java:333)
at
org.apache.twill.internal.AbstractTwillService.access$900(AbstractTwillService.java:85)
at
org.apache.twill.internal.AbstractTwillService$4.onSuccess(AbstractTwillService.java:284)
at
org.apache.twill.internal.AbstractTwillService$4.onSuccess(AbstractTwillService.java:264)
at com.google.common.util.concurrent.Futures$6.run(Futures.java:799)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
> YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is
> removed from DFSUtils from hadoop-2.8
>
>
> Key: TWILL-262
> URL: https://issues.apache.org/jira/browse/TWILL-262
> Project: Apache Twill
> Issue Type: Bug
> Components: yarn
>Affects Versions: 0.8.0, 0.9.0, 0.11.0, 0.12.0, 0.13.0
>Reporter: Hongyuan Li
>Priority: Major
> Attachments: errors.txt
>
>
> YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is
> removed from DFSUtils from hadoop-2.8
> current code
> {code}
> public static void cloneHaNnCredentials(Configuration config) throws
> IOException {
> ……
> // Loop through all name services. Each name service could have multiple
> name node associated with it.
> for (Map.Entry> entry :
> DFSUtil.getHaNnRpcAddresses(config).entrySet()) {
> String nsId = entry.getKey();
> Map addressesInNN = entry.getValue();
> if (!HAUtil.isHAEnabled(config, nsId) || addressesInNN == null ||
> addressesInNN.isEmpty()) {
> continue;
> }
> ……
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TWILL-262) YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is removed from DFSUtils from hadoop-2.8
[
https://issues.apache.org/jira/browse/TWILL-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632853#comment-16632853
]
Hongyuan Li commented on TWILL-262:
---
add errors information as attachment
> YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is
> removed from DFSUtils from hadoop-2.8
>
>
> Key: TWILL-262
> URL: https://issues.apache.org/jira/browse/TWILL-262
> Project: Apache Twill
> Issue Type: Bug
> Components: yarn
>Affects Versions: 0.8.0, 0.9.0, 0.11.0, 0.12.0, 0.13.0
>Reporter: Hongyuan Li
>Priority: Major
> Attachments: errors.txt
>
>
> YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is
> removed from DFSUtils from hadoop-2.8
> current code
> {code}
> public static void cloneHaNnCredentials(Configuration config) throws
> IOException {
> ……
> // Loop through all name services. Each name service could have multiple
> name node associated with it.
> for (Map.Entry> entry :
> DFSUtil.getHaNnRpcAddresses(config).entrySet()) {
> String nsId = entry.getKey();
> Map addressesInNN = entry.getValue();
> if (!HAUtil.isHAEnabled(config, nsId) || addressesInNN == null ||
> addressesInNN.isEmpty()) {
> continue;
> }
> ……
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (TWILL-262) YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is removed from DFSUtils from hadoop-2.8
[
https://issues.apache.org/jira/browse/TWILL-262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated TWILL-262:
--
Attachment: errors.txt
> YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is
> removed from DFSUtils from hadoop-2.8
>
>
> Key: TWILL-262
> URL: https://issues.apache.org/jira/browse/TWILL-262
> Project: Apache Twill
> Issue Type: Bug
> Components: yarn
>Affects Versions: 0.8.0, 0.9.0, 0.11.0, 0.12.0, 0.13.0
>Reporter: Hongyuan Li
>Priority: Major
> Attachments: errors.txt
>
>
> YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is
> removed from DFSUtils from hadoop-2.8
> current code
> {code}
> public static void cloneHaNnCredentials(Configuration config) throws
> IOException {
> ……
> // Loop through all name services. Each name service could have multiple
> name node associated with it.
> for (Map.Entry> entry :
> DFSUtil.getHaNnRpcAddresses(config).entrySet()) {
> String nsId = entry.getKey();
> Map addressesInNN = entry.getValue();
> if (!HAUtil.isHAEnabled(config, nsId) || addressesInNN == null ||
> addressesInNN.isEmpty()) {
> continue;
> }
> ……
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TWILL-262) YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is removed from DFSUtils from hadoop-2.8
[
https://issues.apache.org/jira/browse/TWILL-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632850#comment-16632850
]
Hongyuan Li commented on TWILL-262:
---
[~chtyim], can you give me a code review ? seems build failure is not caused by
what i cmmited.
> YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is
> removed from DFSUtils from hadoop-2.8
>
>
> Key: TWILL-262
> URL: https://issues.apache.org/jira/browse/TWILL-262
> Project: Apache Twill
> Issue Type: Bug
> Components: yarn
>Affects Versions: 0.8.0, 0.9.0, 0.11.0, 0.12.0, 0.13.0
>Reporter: Hongyuan Li
>Priority: Major
>
> YarnUtils#cloneHaNnCredentials uses DFSUtil#getHaNnRpcAddresses, which is
> removed from DFSUtils from hadoop-2.8
> current code
> {code}
> public static void cloneHaNnCredentials(Configuration config) throws
> IOException {
> ……
> // Loop through all name services. Each name service could have multiple
> name node associated with it.
> for (Map.Entry> entry :
> DFSUtil.getHaNnRpcAddresses(config).entrySet()) {
> String nsId = entry.getKey();
> Map addressesInNN = entry.getValue();
> if (!HAUtil.isHAEnabled(config, nsId) || addressesInNN == null ||
> addressesInNN.isEmpty()) {
> continue;
> }
> ……
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TWILL-169) Add YARN node label support
[ https://issues.apache.org/jira/browse/TWILL-169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081499#comment-16081499 ] Hongyuan Li commented on TWILL-169: --- Hi [~chtyim], i set the queue of twill application, and find out that the node-label functions. so, i'm confused of what the jira will do if i am interested in it ? > Add YARN node label support > --- > > Key: TWILL-169 > URL: https://issues.apache.org/jira/browse/TWILL-169 > Project: Apache Twill > Issue Type: Improvement > Components: yarn >Reporter: Terence Yim > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TWILL-169) Add YARN node label support
[ https://issues.apache.org/jira/browse/TWILL-169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079060#comment-16079060 ] Hongyuan Li commented on TWILL-169: --- [~chtyim], this jira has not updated for a long time. Is it being resolved? > Add YARN node label support > --- > > Key: TWILL-169 > URL: https://issues.apache.org/jira/browse/TWILL-169 > Project: Apache Twill > Issue Type: Improvement > Components: yarn >Reporter: Terence Yim > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TWILL-236) PrinterHandler must use localTime for log print instead of using utc
[ https://issues.apache.org/jira/browse/TWILL-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045890#comment-16045890 ] Hongyuan Li commented on TWILL-236: --- yes, i'm interested in it, but i am not sure i have enough time. i can only do this in my free time. > PrinterHandler must use localTime for log print instead of using utc > - > > Key: TWILL-236 > URL: https://issues.apache.org/jira/browse/TWILL-236 > Project: Apache Twill > Issue Type: Improvement >Reporter: Hongyuan Li > > PrinterHandler must use localTime for log print instead of using utc. > Can this assigned to me? [~chtyim] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TWILL-236) PrinterHandler must use localTime for log print instead of using utc
[ https://issues.apache.org/jira/browse/TWILL-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045529#comment-16045529 ] Hongyuan Li commented on TWILL-236: --- in which version, does twill supply LogHanlder implementation that logs to slf4j logger, > PrinterHandler must use localTime for log print instead of using utc > - > > Key: TWILL-236 > URL: https://issues.apache.org/jira/browse/TWILL-236 > Project: Apache Twill > Issue Type: Improvement >Reporter: Hongyuan Li > > PrinterHandler must use localTime for log print instead of using utc. > Can this assigned to me? [~chtyim] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TWILL-235) PrinterLogHandler should support roll time strategy
[ https://issues.apache.org/jira/browse/TWILL-235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045528#comment-16045528 ] Hongyuan Li commented on TWILL-235: --- Ok, i see. > PrinterLogHandler should support roll time strategy > --- > > Key: TWILL-235 > URL: https://issues.apache.org/jira/browse/TWILL-235 > Project: Apache Twill > Issue Type: Improvement >Reporter: Hongyuan Li > > LogHandler suplied by twill should support roll time strategy -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (TWILL-235) PrinterLogHandler should support roll time strategy
[ https://issues.apache.org/jira/browse/TWILL-235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongyuan Li updated TWILL-235: -- Summary: PrinterLogHandler should support roll time strategy (was: PrinterLogHandler should support roll strategy) > PrinterLogHandler should support roll time strategy > --- > > Key: TWILL-235 > URL: https://issues.apache.org/jira/browse/TWILL-235 > Project: Apache Twill > Issue Type: Improvement >Reporter: Hongyuan Li > > LogHandler suplied by twill should support roll -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (TWILL-235) PrinterLogHandler should support roll time strategy
[ https://issues.apache.org/jira/browse/TWILL-235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongyuan Li updated TWILL-235: -- Description: LogHandler suplied by twill should support roll time strategy (was: LogHandler suplied by twill should support roll) > PrinterLogHandler should support roll time strategy > --- > > Key: TWILL-235 > URL: https://issues.apache.org/jira/browse/TWILL-235 > Project: Apache Twill > Issue Type: Improvement >Reporter: Hongyuan Li > > LogHandler suplied by twill should support roll time strategy -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (TWILL-236) PrinterHandler must use localTime for log print instead of using utc
[ https://issues.apache.org/jira/browse/TWILL-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongyuan Li updated TWILL-236: -- Description: PrinterHandler must use localTime for log print instead of using utc. Can this assigned to me? [~chtyim] was: PrinterHandler must use localTime for log print instead of using utc. Can this assigned to me? > PrinterHandler must use localTime for log print instead of using utc > - > > Key: TWILL-236 > URL: https://issues.apache.org/jira/browse/TWILL-236 > Project: Apache Twill > Issue Type: Improvement >Reporter: Hongyuan Li > > PrinterHandler must use localTime for log print instead of using utc. > Can this assigned to me? [~chtyim] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (TWILL-236) PrinterHandler must use localTime for log print instead of using utc
Hongyuan Li created TWILL-236: - Summary: PrinterHandler must use localTime for log print instead of using utc Key: TWILL-236 URL: https://issues.apache.org/jira/browse/TWILL-236 Project: Apache Twill Issue Type: Improvement Reporter: Hongyuan Li PrinterHandler must use localTime for log print instead of using utc. Can this assigned to me? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (TWILL-235) PrinterLogHandler should support roll
[ https://issues.apache.org/jira/browse/TWILL-235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongyuan Li updated TWILL-235: -- Summary: PrinterLogHandler should support roll (was: LogHandler suplied by twill should support roll) > PrinterLogHandler should support roll > - > > Key: TWILL-235 > URL: https://issues.apache.org/jira/browse/TWILL-235 > Project: Apache Twill > Issue Type: Improvement >Reporter: Hongyuan Li > > LogHandler suplied by twill should support roll -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (TWILL-235) LogHandler suplied by twill should support roll
Hongyuan Li created TWILL-235: - Summary: LogHandler suplied by twill should support roll Key: TWILL-235 URL: https://issues.apache.org/jira/browse/TWILL-235 Project: Apache Twill Issue Type: Improvement Reporter: Hongyuan Li LogHandler suplied by twill should support roll -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (TWILL-234) Twill submit yarn job does not support set User
[
https://issues.apache.org/jira/browse/TWILL-234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li closed TWILL-234.
-
Resolution: Fixed
Depreacted
> Twill submit yarn job does not support set User
>
>
> Key: TWILL-234
> URL: https://issues.apache.org/jira/browse/TWILL-234
> Project: Apache Twill
> Issue Type: Bug
>Reporter: Hongyuan Li
>
> Twill submit yarn job does not support set User , code in YarnTwillPreparer
> below:
> {code}
> @Override
> public TwillPreparer setUser(String user) {
> return this;
> }
> {code}.
> From the code, the user set is not suported.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (TWILL-234) Twill submit yarn job does not support set User
[
https://issues.apache.org/jira/browse/TWILL-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16027420#comment-16027420
]
Hongyuan Li commented on TWILL-234:
---
ok. Known that, you can close the jira . [~chtyim]
> Twill submit yarn job does not support set User
>
>
> Key: TWILL-234
> URL: https://issues.apache.org/jira/browse/TWILL-234
> Project: Apache Twill
> Issue Type: Bug
>Reporter: Hongyuan Li
>
> Twill submit yarn job does not support set User , code in YarnTwillPreparer
> below:
> {code}
> @Override
> public TwillPreparer setUser(String user) {
> return this;
> }
> {code}.
> From the code, the user set is not suported.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (TWILL-234) Twill submit yarn job does not support set User
Hongyuan Li created TWILL-234:
-
Summary: Twill submit yarn job does not support set User
Key: TWILL-234
URL: https://issues.apache.org/jira/browse/TWILL-234
Project: Apache Twill
Issue Type: Bug
Reporter: Hongyuan Li
Twill submit yarn job does not support set User , code in YarnTwillPreparer
below:
{code}
@Override
public TwillPreparer setUser(String user) {
return this;
}
{code}.
>From the code, the user set is not suported.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
