[jira] [Moved] (HADOOP-19167) Change of Codec configuration does not work
[
https://issues.apache.org/jira/browse/HADOOP-19167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin moved HDFS-17510 to HADOOP-19167:
---
Component/s: compress
(was: compress)
Key: HADOOP-19167 (was: HDFS-17510)
Project: Hadoop Common (was: Hadoop HDFS)
> Change of Codec configuration does not work
> ---
>
> Key: HADOOP-19167
> URL: https://issues.apache.org/jira/browse/HADOOP-19167
> Project: Hadoop Common
> Issue Type: Bug
> Components: compress
>Reporter: Zhikai Hu
>Priority: Minor
> Labels: pull-request-available
>
> In one of my projects, I need to dynamically adjust compression level for
> different files.
> However, I found that in most cases the new compression level does not take
> effect as expected, the old compression level continues to be used.
> Here is the relevant code snippet:
> ZStandardCodec zStandardCodec = new ZStandardCodec();
> zStandardCodec.setConf(conf);
> conf.set("io.compression.codec.zstd.level", "5"); // level may change
> dynamically
> conf.set("io.compression.codec.zstd", zStandardCodec.getClass().getName());
> writer = SequenceFile.createWriter(conf,
> SequenceFile.Writer.file(sequenceFilePath),
>
> SequenceFile.Writer.keyClass(LongWritable.class),
>
> SequenceFile.Writer.valueClass(BytesWritable.class),
>
> SequenceFile.Writer.compression(CompressionType.BLOCK));
> The reason is SequenceFile.Writer.init() method will call
> CodecPool.getCompressor(codec, null) to get a compressor.
> If the compressor is a reused instance, the conf is not applied because it is
> passed as null:
> public static Compressor getCompressor(CompressionCodec codec, Configuration
> conf) {
> Compressor compressor = borrow(compressorPool, codec.getCompressorType());
> if (compressor == null)
> { compressor = codec.createCompressor(); LOG.info("Got brand-new compressor
> ["+codec.getDefaultExtension()+"]"); }
> else {
> compressor.reinit(conf); //conf is null here
> ..
>
> Please also refer to my unit test to reproduce the bug.
> To address this bug, I modified the code to ensure that the configuration is
> read back from the codec when a compressor is reused.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16097) Provide proper documentation for FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-16097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-16097: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) Committed this to trunk. Thanks [~xkrogen] for completing the document of the Fair Call Queue. It will help users a lot, :). > Provide proper documentation for FairCallQueue > -- > > Key: HADOOP-16097 > URL: https://issues.apache.org/jira/browse/HADOOP-16097 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation, ipc >Reporter: Erik Krogen >Assignee: Erik Krogen >Priority: Major > Fix For: 3.3.0 > > Attachments: FairCallQueueGuide_Rendered.pdf, HADOOP-16097.000.patch, > HADOOP-16097.001.patch, HADOOP-16097.002.patch, faircallqueue-overview.png > > > FairCallQueue, added in HADOOP-10282, doesn't seem to be well-documented > anywhere. Let's add in a new documentation for it and related components. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16097) Provide proper documentation for FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-16097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766726#comment-16766726 ] Yiqun Lin commented on HADOOP-16097: LGTM, +1. Commit this shortly. > Provide proper documentation for FairCallQueue > -- > > Key: HADOOP-16097 > URL: https://issues.apache.org/jira/browse/HADOOP-16097 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation, ipc >Reporter: Erik Krogen >Assignee: Erik Krogen >Priority: Major > Attachments: FairCallQueueGuide_Rendered.pdf, HADOOP-16097.000.patch, > HADOOP-16097.001.patch, HADOOP-16097.002.patch, faircallqueue-overview.png > > > FairCallQueue, added in HADOOP-10282, doesn't seem to be well-documented > anywhere. Let's add in a new documentation for it and related components. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16097) Provide proper documentation for FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-16097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765744#comment-16765744
]
Yiqun Lin commented on HADOOP-16097:
Thanks for addressing the comments, [~xkrogen]! The patch looks great now.
{quote}The priority level computation for the users is started from low
priority levels since they will be most common.
{quote}
This sentence is from the comment in
{{DecayRpcScheduler#computePriorityLevel}}. I just want let users know the
computation for the priority level. But it's ok to mention this or not. Current
description is enough I think.
Rendered the doc in my local, the imgae cannot be rendered well,
Can we update {{(../resources/images/faircallqueue-overview.png)}} to
{{(./images/faircallqueue-overview.png)}}? For the hadoop site page, the latter
should be the right path, the former only makes sense for the github page.
BTW, [~xkrogen], can you attach a screen shot of the rendered page once this
addressed?
Follow steps:
* cd hadoop-common-project/hadoop-common
* run mvn site:site
* open the page
hadoop-common-project/hadoop-common/target/site/FairCallQueue.html
> Provide proper documentation for FairCallQueue
> --
>
> Key: HADOOP-16097
> URL: https://issues.apache.org/jira/browse/HADOOP-16097
> Project: Hadoop Common
> Issue Type: Improvement
> Components: documentation, ipc
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16097.000.patch, HADOOP-16097.001.patch,
> faircallqueue-overview.png
>
>
> FairCallQueue, added in HADOOP-10282, doesn't seem to be well-documented
> anywhere. Let's add in a new documentation for it and related components.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-12980) Document RPC scheduler/callqueue configuration keys
[ https://issues.apache.org/jira/browse/HADOOP-12980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin resolved HADOOP-12980. Resolution: Duplicate > Document RPC scheduler/callqueue configuration keys > --- > > Key: HADOOP-12980 > URL: https://issues.apache.org/jira/browse/HADOOP-12980 > Project: Hadoop Common > Issue Type: Sub-task > Components: documentation >Reporter: Xiaoyu Yao >Assignee: Ryan Wu >Priority: Major > > This ticket is opened to document RPC scheduler, callqueue and handler > related keys. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12980) Document RPC scheduler/callqueue configuration keys
[ https://issues.apache.org/jira/browse/HADOOP-12980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16764442#comment-16764442 ] Yiqun Lin commented on HADOOP-12980: As HADOOP-16097 is working-in-progress and being reviewed. This JIRA is duplicated. Closing this. > Document RPC scheduler/callqueue configuration keys > --- > > Key: HADOOP-12980 > URL: https://issues.apache.org/jira/browse/HADOOP-12980 > Project: Hadoop Common > Issue Type: Sub-task > Components: documentation >Reporter: Xiaoyu Yao >Assignee: Ryan Wu >Priority: Major > > This ticket is opened to document RPC scheduler, callqueue and handler > related keys. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16097) Provide proper documentation for FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-16097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16764431#comment-16764431
]
Yiqun Lin commented on HADOOP-16097:
Nice work, [~xkrogen]! The patch looks great, some minor comments:
*FairCallQueue*
Line 64: Seems not accurate, can we update from '{{users whose calls make up
more than 25% of'}} to '{{users whose calls make up more than 25% and less than
50% of'}}
Line 65: {{more than 12.5% are in}} --> {{more than 12.5% and less than 25%
are in}} Can we add one sentence behind this:
{noformat}
...are placed in the highest priority. The priority level computation for the
users is started from low priority levels since they will be most common.
{noformat}
Line 72: Typo 'an RpcMultiplexer' --> 'a RpcMultiplexer'
Line 85: Can we reword this sentence? Looks a little confused: a incoming
request would receive a backoff request?
{noformat}
an incoming request at priority levels 2 or lower would receive a backoff
request
{noformat}
Line 86: {{and 1 would proceed as normal}} --> {{and 1 would be proceed as
normal}}
Line 111: Can we use {{WeightedRoundRobinMultiplexer}} to replace
{{FairCallQueue}} used Component column?
Two other comments:
* I suppose we lack of the Description for the IdentityProvider controlled by
{{identity-provider.impl}}. Can we document a little of this?
* The image cannot be applied from the patch.
> Provide proper documentation for FairCallQueue
> --
>
> Key: HADOOP-16097
> URL: https://issues.apache.org/jira/browse/HADOOP-16097
> Project: Hadoop Common
> Issue Type: Improvement
> Components: documentation, ipc
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16097.000.patch, faircallqueue-overview.png
>
>
> FairCallQueue, added in HADOOP-10282, doesn't seem to be well-documented
> anywhere. Let's add in a new documentation for it and related components.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15817) Reuse Object Mapper in KMSJSONReader
[
https://issues.apache.org/jira/browse/HADOOP-15817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-15817:
---
Affects Version/s: 3.1.1
> Reuse Object Mapper in KMSJSONReader
>
>
> Key: HADOOP-15817
> URL: https://issues.apache.org/jira/browse/HADOOP-15817
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.1.1
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
>Priority: Major
> Labels: performance
> Fix For: 2.10.0, 2.9.2, 3.0.4, 3.3.0, 3.1.2, 2.8.6
>
> Attachments: HADOOP-15817.001.patch
>
>
> Paying an expensive cost to construct object mapper deserializer cache.
>
> {code:title=KMS Server Stack Trace}
> "qtp1926764753-117" #117 prio=5 os_prio=0 tid=0x0321c000 nid=0x1f0bd
> runnable [0x2b4caabf7000]
>java.lang.Thread.State: RUNNABLE
> at
> java.lang.reflect.Executable.sharedGetParameterAnnotations(Executable.java:553)
> at
> java.lang.reflect.Constructor.getParameterAnnotations(Constructor.java:523)
> at
> org.codehaus.jackson.map.introspect.AnnotatedClass._constructConstructor(AnnotatedClass.java:784)
> at
> org.codehaus.jackson.map.introspect.AnnotatedClass.resolveCreators(AnnotatedClass.java:327)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.classWithCreators(BasicClassIntrospector.java:187)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.collectProperties(BasicClassIntrospector.java:157)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.forCreation(BasicClassIntrospector.java:119)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.forCreation(BasicClassIntrospector.java:16)
> at
> org.codehaus.jackson.map.DeserializationConfig.introspectForCreation(DeserializationConfig.java:877)
> at
> org.codehaus.jackson.map.deser.BasicDeserializerFactory.createMapDeserializer(BasicDeserializerFactory.java:430)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createDeserializer(StdDeserializerProvider.java:380)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCache2(StdDeserializerProvider.java:310)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCacheValueDeserializer(StdDeserializerProvider.java:290)
> - locked <0x000752fbbc28> (a java.util.HashMap)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider.findValueDeserializer(StdDeserializerProvider.java:159)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider.findTypedValueDeserializer(StdDeserializerProvider.java:180)
> at
> org.codehaus.jackson.map.ObjectMapper._findRootDeserializer(ObjectMapper.java:2829)
> at
> org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2728)
> at
> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1909)
> at
> org.apache.hadoop.crypto.key.kms.server.KMSJSONReader.readFrom(KMSJSONReader.java:52)
> at
> org.apache.hadoop.crypto.key.kms.server.KMSJSONReader.readFrom(KMSJSONReader.java:35)
> at
> com.sun.jersey.spi.container.ContainerRequest.getEntity(ContainerRequest.java:474)
> at
> com.sun.jersey.server.impl.model.method.dispatch.EntityParamDispatchProvider$EntityInjectable.getValue(EntityParamDispatchProvider.java:123)
> at
> com.sun.jersey.server.impl.inject.InjectableValuesProvider.getInjectableValues(InjectableValuesProvider.java:46)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$EntityParamInInvoker.getParams(AbstractResourceMethodDispatchProvider.java:153)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:203)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> at
>
[jira] [Updated] (HADOOP-15817) Reuse Object Mapper in KMSJSONReader
[
https://issues.apache.org/jira/browse/HADOOP-15817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-15817:
---
Fix Version/s: 3.2.0
> Reuse Object Mapper in KMSJSONReader
>
>
> Key: HADOOP-15817
> URL: https://issues.apache.org/jira/browse/HADOOP-15817
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.1.1
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
>Priority: Major
> Labels: performance
> Fix For: 2.10.0, 3.2.0, 2.9.2, 3.0.4, 3.3.0, 3.1.2, 2.8.6
>
> Attachments: HADOOP-15817.001.patch
>
>
> Paying an expensive cost to construct object mapper deserializer cache.
>
> {code:title=KMS Server Stack Trace}
> "qtp1926764753-117" #117 prio=5 os_prio=0 tid=0x0321c000 nid=0x1f0bd
> runnable [0x2b4caabf7000]
>java.lang.Thread.State: RUNNABLE
> at
> java.lang.reflect.Executable.sharedGetParameterAnnotations(Executable.java:553)
> at
> java.lang.reflect.Constructor.getParameterAnnotations(Constructor.java:523)
> at
> org.codehaus.jackson.map.introspect.AnnotatedClass._constructConstructor(AnnotatedClass.java:784)
> at
> org.codehaus.jackson.map.introspect.AnnotatedClass.resolveCreators(AnnotatedClass.java:327)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.classWithCreators(BasicClassIntrospector.java:187)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.collectProperties(BasicClassIntrospector.java:157)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.forCreation(BasicClassIntrospector.java:119)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.forCreation(BasicClassIntrospector.java:16)
> at
> org.codehaus.jackson.map.DeserializationConfig.introspectForCreation(DeserializationConfig.java:877)
> at
> org.codehaus.jackson.map.deser.BasicDeserializerFactory.createMapDeserializer(BasicDeserializerFactory.java:430)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createDeserializer(StdDeserializerProvider.java:380)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCache2(StdDeserializerProvider.java:310)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCacheValueDeserializer(StdDeserializerProvider.java:290)
> - locked <0x000752fbbc28> (a java.util.HashMap)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider.findValueDeserializer(StdDeserializerProvider.java:159)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider.findTypedValueDeserializer(StdDeserializerProvider.java:180)
> at
> org.codehaus.jackson.map.ObjectMapper._findRootDeserializer(ObjectMapper.java:2829)
> at
> org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2728)
> at
> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1909)
> at
> org.apache.hadoop.crypto.key.kms.server.KMSJSONReader.readFrom(KMSJSONReader.java:52)
> at
> org.apache.hadoop.crypto.key.kms.server.KMSJSONReader.readFrom(KMSJSONReader.java:35)
> at
> com.sun.jersey.spi.container.ContainerRequest.getEntity(ContainerRequest.java:474)
> at
> com.sun.jersey.server.impl.model.method.dispatch.EntityParamDispatchProvider$EntityInjectable.getValue(EntityParamDispatchProvider.java:123)
> at
> com.sun.jersey.server.impl.inject.InjectableValuesProvider.getInjectableValues(InjectableValuesProvider.java:46)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$EntityParamInInvoker.getParams(AbstractResourceMethodDispatchProvider.java:153)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:203)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> at
>
[jira] [Comment Edited] (HADOOP-15817) Reuse Object Mapper in KMSJSONReader
[
https://issues.apache.org/jira/browse/HADOOP-15817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637743#comment-16637743
]
Yiqun Lin edited comment on HADOOP-15817 at 10/4/18 3:06 AM:
-
I have committed this to trunk, branch-3.2, branch-3.1, branch-3.0, branch-2,
branch-2.8 and branch-2.9.
Thanks [~jeagles] for the contribution and thanks [~ajisakaa] for the review.
was (Author: linyiqun):
I have committed this to trunk, branch-3.1, branch-3.0, branch-2, branch-2.8
and branch-2.9.
Thanks [~jeagles] for the contribution and thanks [~ajisakaa] for the review.
> Reuse Object Mapper in KMSJSONReader
>
>
> Key: HADOOP-15817
> URL: https://issues.apache.org/jira/browse/HADOOP-15817
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.1.1
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
>Priority: Major
> Labels: performance
> Fix For: 2.10.0, 3.2.0, 2.9.2, 3.0.4, 3.3.0, 3.1.2, 2.8.6
>
> Attachments: HADOOP-15817.001.patch
>
>
> Paying an expensive cost to construct object mapper deserializer cache.
>
> {code:title=KMS Server Stack Trace}
> "qtp1926764753-117" #117 prio=5 os_prio=0 tid=0x0321c000 nid=0x1f0bd
> runnable [0x2b4caabf7000]
>java.lang.Thread.State: RUNNABLE
> at
> java.lang.reflect.Executable.sharedGetParameterAnnotations(Executable.java:553)
> at
> java.lang.reflect.Constructor.getParameterAnnotations(Constructor.java:523)
> at
> org.codehaus.jackson.map.introspect.AnnotatedClass._constructConstructor(AnnotatedClass.java:784)
> at
> org.codehaus.jackson.map.introspect.AnnotatedClass.resolveCreators(AnnotatedClass.java:327)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.classWithCreators(BasicClassIntrospector.java:187)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.collectProperties(BasicClassIntrospector.java:157)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.forCreation(BasicClassIntrospector.java:119)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.forCreation(BasicClassIntrospector.java:16)
> at
> org.codehaus.jackson.map.DeserializationConfig.introspectForCreation(DeserializationConfig.java:877)
> at
> org.codehaus.jackson.map.deser.BasicDeserializerFactory.createMapDeserializer(BasicDeserializerFactory.java:430)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createDeserializer(StdDeserializerProvider.java:380)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCache2(StdDeserializerProvider.java:310)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCacheValueDeserializer(StdDeserializerProvider.java:290)
> - locked <0x000752fbbc28> (a java.util.HashMap)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider.findValueDeserializer(StdDeserializerProvider.java:159)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider.findTypedValueDeserializer(StdDeserializerProvider.java:180)
> at
> org.codehaus.jackson.map.ObjectMapper._findRootDeserializer(ObjectMapper.java:2829)
> at
> org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2728)
> at
> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1909)
> at
> org.apache.hadoop.crypto.key.kms.server.KMSJSONReader.readFrom(KMSJSONReader.java:52)
> at
> org.apache.hadoop.crypto.key.kms.server.KMSJSONReader.readFrom(KMSJSONReader.java:35)
> at
> com.sun.jersey.spi.container.ContainerRequest.getEntity(ContainerRequest.java:474)
> at
> com.sun.jersey.server.impl.model.method.dispatch.EntityParamDispatchProvider$EntityInjectable.getValue(EntityParamDispatchProvider.java:123)
> at
> com.sun.jersey.server.impl.inject.InjectableValuesProvider.getInjectableValues(InjectableValuesProvider.java:46)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$EntityParamInInvoker.getParams(AbstractResourceMethodDispatchProvider.java:153)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:203)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
>
[jira] [Updated] (HADOOP-15817) Reuse Object Mapper in KMSJSONReader
[
https://issues.apache.org/jira/browse/HADOOP-15817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-15817:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.8.6
3.1.2
3.3.0
3.0.4
2.9.2
2.10.0
Status: Resolved (was: Patch Available)
I have committed this to trunk, branch-3.1, branch-3.0, branch-2, branch-2.8
and branch-2.9.
Thanks [~jeagles] for the contribution and thanks [~ajisakaa] for the review.
> Reuse Object Mapper in KMSJSONReader
>
>
> Key: HADOOP-15817
> URL: https://issues.apache.org/jira/browse/HADOOP-15817
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.1.1
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
>Priority: Major
> Labels: performance
> Fix For: 2.10.0, 2.9.2, 3.0.4, 3.3.0, 3.1.2, 2.8.6
>
> Attachments: HADOOP-15817.001.patch
>
>
> Paying an expensive cost to construct object mapper deserializer cache.
>
> {code:title=KMS Server Stack Trace}
> "qtp1926764753-117" #117 prio=5 os_prio=0 tid=0x0321c000 nid=0x1f0bd
> runnable [0x2b4caabf7000]
>java.lang.Thread.State: RUNNABLE
> at
> java.lang.reflect.Executable.sharedGetParameterAnnotations(Executable.java:553)
> at
> java.lang.reflect.Constructor.getParameterAnnotations(Constructor.java:523)
> at
> org.codehaus.jackson.map.introspect.AnnotatedClass._constructConstructor(AnnotatedClass.java:784)
> at
> org.codehaus.jackson.map.introspect.AnnotatedClass.resolveCreators(AnnotatedClass.java:327)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.classWithCreators(BasicClassIntrospector.java:187)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.collectProperties(BasicClassIntrospector.java:157)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.forCreation(BasicClassIntrospector.java:119)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.forCreation(BasicClassIntrospector.java:16)
> at
> org.codehaus.jackson.map.DeserializationConfig.introspectForCreation(DeserializationConfig.java:877)
> at
> org.codehaus.jackson.map.deser.BasicDeserializerFactory.createMapDeserializer(BasicDeserializerFactory.java:430)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createDeserializer(StdDeserializerProvider.java:380)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCache2(StdDeserializerProvider.java:310)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCacheValueDeserializer(StdDeserializerProvider.java:290)
> - locked <0x000752fbbc28> (a java.util.HashMap)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider.findValueDeserializer(StdDeserializerProvider.java:159)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider.findTypedValueDeserializer(StdDeserializerProvider.java:180)
> at
> org.codehaus.jackson.map.ObjectMapper._findRootDeserializer(ObjectMapper.java:2829)
> at
> org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2728)
> at
> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1909)
> at
> org.apache.hadoop.crypto.key.kms.server.KMSJSONReader.readFrom(KMSJSONReader.java:52)
> at
> org.apache.hadoop.crypto.key.kms.server.KMSJSONReader.readFrom(KMSJSONReader.java:35)
> at
> com.sun.jersey.spi.container.ContainerRequest.getEntity(ContainerRequest.java:474)
> at
> com.sun.jersey.server.impl.model.method.dispatch.EntityParamDispatchProvider$EntityInjectable.getValue(EntityParamDispatchProvider.java:123)
> at
> com.sun.jersey.server.impl.inject.InjectableValuesProvider.getInjectableValues(InjectableValuesProvider.java:46)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$EntityParamInInvoker.getParams(AbstractResourceMethodDispatchProvider.java:153)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:203)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
>
[jira] [Commented] (HADOOP-15817) Reuse Object Mapper in KMSJSONReader
[
https://issues.apache.org/jira/browse/HADOOP-15817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637729#comment-16637729
]
Yiqun Lin commented on HADOOP-15817:
LGTM, +1. Committing this.
> Reuse Object Mapper in KMSJSONReader
>
>
> Key: HADOOP-15817
> URL: https://issues.apache.org/jira/browse/HADOOP-15817
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
>Priority: Major
> Labels: performance
> Attachments: HADOOP-15817.001.patch
>
>
> Paying an expensive cost to construct object mapper deserializer cache.
>
> {code:title=KMS Server Stack Trace}
> "qtp1926764753-117" #117 prio=5 os_prio=0 tid=0x0321c000 nid=0x1f0bd
> runnable [0x2b4caabf7000]
>java.lang.Thread.State: RUNNABLE
> at
> java.lang.reflect.Executable.sharedGetParameterAnnotations(Executable.java:553)
> at
> java.lang.reflect.Constructor.getParameterAnnotations(Constructor.java:523)
> at
> org.codehaus.jackson.map.introspect.AnnotatedClass._constructConstructor(AnnotatedClass.java:784)
> at
> org.codehaus.jackson.map.introspect.AnnotatedClass.resolveCreators(AnnotatedClass.java:327)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.classWithCreators(BasicClassIntrospector.java:187)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.collectProperties(BasicClassIntrospector.java:157)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.forCreation(BasicClassIntrospector.java:119)
> at
> org.codehaus.jackson.map.introspect.BasicClassIntrospector.forCreation(BasicClassIntrospector.java:16)
> at
> org.codehaus.jackson.map.DeserializationConfig.introspectForCreation(DeserializationConfig.java:877)
> at
> org.codehaus.jackson.map.deser.BasicDeserializerFactory.createMapDeserializer(BasicDeserializerFactory.java:430)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createDeserializer(StdDeserializerProvider.java:380)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCache2(StdDeserializerProvider.java:310)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCacheValueDeserializer(StdDeserializerProvider.java:290)
> - locked <0x000752fbbc28> (a java.util.HashMap)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider.findValueDeserializer(StdDeserializerProvider.java:159)
> at
> org.codehaus.jackson.map.deser.StdDeserializerProvider.findTypedValueDeserializer(StdDeserializerProvider.java:180)
> at
> org.codehaus.jackson.map.ObjectMapper._findRootDeserializer(ObjectMapper.java:2829)
> at
> org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2728)
> at
> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1909)
> at
> org.apache.hadoop.crypto.key.kms.server.KMSJSONReader.readFrom(KMSJSONReader.java:52)
> at
> org.apache.hadoop.crypto.key.kms.server.KMSJSONReader.readFrom(KMSJSONReader.java:35)
> at
> com.sun.jersey.spi.container.ContainerRequest.getEntity(ContainerRequest.java:474)
> at
> com.sun.jersey.server.impl.model.method.dispatch.EntityParamDispatchProvider$EntityInjectable.getValue(EntityParamDispatchProvider.java:123)
> at
> com.sun.jersey.server.impl.inject.InjectableValuesProvider.getInjectableValues(InjectableValuesProvider.java:46)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$EntityParamInInvoker.getParams(AbstractResourceMethodDispatchProvider.java:153)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:203)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> at
>
[jira] [Commented] (HADOOP-15805) Hadoop logo not showed correctly in old site
[
https://issues.apache.org/jira/browse/HADOOP-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16634827#comment-16634827
]
Yiqun Lin commented on HADOOP-15805:
Thanks [~Sandeep Nemuri] for fixing this. I agree to copy the picture rather
than update so many wrong urls. Let's wait [~elek] for the review since he will
be more familiar with this, :).
> Hadoop logo not showed correctly in old site
>
>
> Key: HADOOP-15805
> URL: https://issues.apache.org/jira/browse/HADOOP-15805
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Assignee: Sandeep Nemuri
>Priority: Major
> Attachments: Error-page.jpg, HADOOP-15805.001.patch
>
>
> Hadoop logo not showed correctly in old site. In old site pages, we use the
> address {{[http://hadoop.apache.org/images/hadoop-logo.jpg]}} to show the
> hadoop logo. Actually, this address is outdated. The right address now is
> [http://hadoop.apache.org/hadoop-logo.jpg].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15805) Hadoop logo not showed correctly in old site
[
https://issues.apache.org/jira/browse/HADOOP-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-15805:
---
Description: Hadoop logo not showed correctly in old site. In old site
pages, we use the address {{[http://hadoop.apache.org/images/hadoop-logo.jpg]}}
to show the hadoop logo. Actually, this address is outdated. The right address
now is [http://hadoop.apache.org/hadoop-logo.jpg]. (was: Hadoop logo not
showed correctly in old site. In old site pages, we use the address
{{[http://hadoop.apache.org/images/hadoop-logo.jpg]}} to show the hadoop logo.
Actually, this addressed is outdated. The right address now is
[http://hadoop.apache.org/hadoop-logo.jpg].)
> Hadoop logo not showed correctly in old site
>
>
> Key: HADOOP-15805
> URL: https://issues.apache.org/jira/browse/HADOOP-15805
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Priority: Major
> Attachments: Error-page.jpg
>
>
> Hadoop logo not showed correctly in old site. In old site pages, we use the
> address {{[http://hadoop.apache.org/images/hadoop-logo.jpg]}} to show the
> hadoop logo. Actually, this address is outdated. The right address now is
> [http://hadoop.apache.org/hadoop-logo.jpg].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-15805) Hadoop logo not showed correctly in old site
[
https://issues.apache.org/jira/browse/HADOOP-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1668#comment-1668
]
Yiqun Lin edited comment on HADOOP-15805 at 9/30/18 10:10 AM:
--
Attach the screen-shot in my local. In the top of left side, we can see hadoop
logo isn't showed correctly.
was (Author: linyiqun):
Attach the screen-shot in my local. In the top of left side, we can see hadoop
logo doesn't showed correctly.
> Hadoop logo not showed correctly in old site
>
>
> Key: HADOOP-15805
> URL: https://issues.apache.org/jira/browse/HADOOP-15805
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Priority: Major
> Attachments: Error-page.jpg
>
>
> Hadoop logo not showed correctly in old site. In old site pages, we use the
> address {{[http://hadoop.apache.org/images/hadoop-logo.jpg]}} to show the
> hadoop logo. Actually, this addressed is outdated. The right address now is
> [http://hadoop.apache.org/hadoop-logo.jpg].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15805) Hadoop logo not showed correctly in old site
[
https://issues.apache.org/jira/browse/HADOOP-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1668#comment-1668
]
Yiqun Lin commented on HADOOP-15805:
Attach the screen-shot in my local. In the top of left side, we can see hadoop
logo doesn't showed correctly.
> Hadoop logo not showed correctly in old site
>
>
> Key: HADOOP-15805
> URL: https://issues.apache.org/jira/browse/HADOOP-15805
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Priority: Major
> Attachments: Error-page.jpg
>
>
> Hadoop logo not showed correctly in old site. In old site pages, we use the
> address {{[http://hadoop.apache.org/images/hadoop-logo.jpg]}} to show the
> hadoop logo. Actually, this addressed is outdated. The right address now is
> [http://hadoop.apache.org/hadoop-logo.jpg].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15805) Hadoop logo not showed correctly in old site
[
https://issues.apache.org/jira/browse/HADOOP-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-15805:
---
Attachment: Error-page.jpg
> Hadoop logo not showed correctly in old site
>
>
> Key: HADOOP-15805
> URL: https://issues.apache.org/jira/browse/HADOOP-15805
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Priority: Major
> Attachments: Error-page.jpg
>
>
> Hadoop logo not showed correctly in old site. In old site pages, we use the
> address {{[http://hadoop.apache.org/images/hadoop-logo.jpg]}} to show the
> hadoop logo. Actually, this addressed is outdated. The right address now is
> [http://hadoop.apache.org/hadoop-logo.jpg].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15805) Hadoop logo not showed correctly in old site
Yiqun Lin created HADOOP-15805:
--
Summary: Hadoop logo not showed correctly in old site
Key: HADOOP-15805
URL: https://issues.apache.org/jira/browse/HADOOP-15805
Project: Hadoop Common
Issue Type: Bug
Components: documentation
Affects Versions: 3.1.1
Reporter: Yiqun Lin
Hadoop logo not showed correctly in old site. In old site pages, we use the
address {{[http://hadoop.apache.org/images/hadoop-logo.jpg]}} to show the
hadoop logo. Actually, this addressed is outdated. The right address now is
[http://hadoop.apache.org/hadoop-logo.jpg].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-15774) Discovery of HA servers
[ https://issues.apache.org/jira/browse/HADOOP-15774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621558#comment-16621558 ] Yiqun Lin edited comment on HADOOP-15774 at 9/20/18 6:13 AM: - This looks very interesting! I did a quick look for the design doc, some aspects I am concerning and maybe described more in the doc: * Once server addresses are updated, how sever manager plan to do and let workers known? * For the workers, do they need to get server address every time when they ask for a request for servers? Or they will just get the address once and cached it in itself? was (Author: linyiqun): This looks very interesting! I did a quick look for the design doc, some aspects I am concerning and maybe described more in the doc: * Once server addresses are updated, how sever manager plan to do and lets works known? * For the workers, do they need to get server address every time when they ask for a request for servers? Or they will just get the address once and cached the it in itself? > Discovery of HA servers > --- > > Key: HADOOP-15774 > URL: https://issues.apache.org/jira/browse/HADOOP-15774 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Íñigo Goiri >Priority: Major > Attachments: Discovery Service.pdf > > > Currently, Hadoop relies on configuration files to specify the servers. > This requires maintaining these configuration files and propagating the > changes. > Hadoop should have a framework to provide discovery. > For example, in HDFS, we could define the Namenodes in a shared location and > the DNs would use the framework to find the Namenodes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15774) Discovery of HA servers
[ https://issues.apache.org/jira/browse/HADOOP-15774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621558#comment-16621558 ] Yiqun Lin commented on HADOOP-15774: This looks very interesting! I did a quick look for the design doc, some aspects I am concerning and maybe described more in the doc: * Once server addresses are updated, how sever manager plan to do and lets works known? * For the workers, do they need to get server address every time when they ask for a request for servers? Or they will just get the address once and cached the it in itself? > Discovery of HA servers > --- > > Key: HADOOP-15774 > URL: https://issues.apache.org/jira/browse/HADOOP-15774 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Íñigo Goiri >Priority: Major > Attachments: Discovery Service.pdf > > > Currently, Hadoop relies on configuration files to specify the servers. > This requires maintaining these configuration files and propagating the > changes. > Hadoop should have a framework to provide discovery. > For example, in HDFS, we could define the Namenodes in a shared location and > the DNs would use the framework to find the Namenodes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
***UNCHECKED*** [jira] [Commented] (HADOOP-12980) Document RPC scheduler/callqueue configuration keys
[
https://issues.apache.org/jira/browse/HADOOP-12980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620196#comment-16620196
]
Yiqun Lin commented on HADOOP-12980:
Thanks [~xyao].
I did a quick look and plan to document missing settings related with
following:
{code:java}
/**
* CallQueue related settings. These are not used directly, but rather
* combined with a namespace and port. For instance:
* IPC_NAMESPACE + ".8020." + IPC_CALLQUEUE_IMPL_KEY
*/
public static final String IPC_NAMESPACE = "ipc";
public static final String IPC_CALLQUEUE_IMPL_KEY = "callqueue.impl";
public static final String IPC_SCHEDULER_IMPL_KEY = "scheduler.impl";
public static final String IPC_IDENTITY_PROVIDER_KEY =
"identity-provider.impl";
public static final String IPC_BACKOFF_ENABLE = "backoff.enable";
public static final boolean IPC_BACKOFF_ENABLE_DEFAULT = false;
{code}
Transformed to the real settings, will be total 4 settings:
* ipc..callqueue.impl
* ipc..scheduler.impl
* ipc..identity-provider.impl
* ipc..backoff.enable
For the more-detailed setting, e.g. decay-scheduler.decay-factor, we recommend
users to look into corresponding class and know how to use it. Here we just let
users know how to easily enable this and know the usage of the basic setting.
I have talked with [~jianliang.wu] offline, he is doing some work of this. I'd
like re-assign this for him. Let's looking forward to seeing his patch, :).
> Document RPC scheduler/callqueue configuration keys
> ---
>
> Key: HADOOP-12980
> URL: https://issues.apache.org/jira/browse/HADOOP-12980
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Xiaoyu Yao
>Assignee: Yiqun Lin
>Priority: Major
>
> This ticket is opened to document RPC scheduler, callqueue and handler
> related keys.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-12980) Document RPC scheduler/callqueue configuration keys
[ https://issues.apache.org/jira/browse/HADOOP-12980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin reassigned HADOOP-12980: -- Assignee: Ryan Wu (was: Yiqun Lin) > Document RPC scheduler/callqueue configuration keys > --- > > Key: HADOOP-12980 > URL: https://issues.apache.org/jira/browse/HADOOP-12980 > Project: Hadoop Common > Issue Type: Sub-task > Components: documentation >Reporter: Xiaoyu Yao >Assignee: Ryan Wu >Priority: Major > > This ticket is opened to document RPC scheduler, callqueue and handler > related keys. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15768) Do not use Time#now to calculate the rpc process time duration
[
https://issues.apache.org/jira/browse/HADOOP-15768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-15768:
---
Description:
For the rpc process time calculation, we are using a not-recommended way: Using
{{Time#now}} to calculate for this.
{code}
// Invoke the protocol method
long startTime = Time.now();
int qTime = (int) (startTime-receivedTime);
int processingTime = (int) (Time.now() - startTime);
...
server.updateMetrics(detailedMetricsName, qTime, processingTime, false);
{code}
Actually we should use {{Time#monotonicNow()}} instead. This JIRA will fix
these across RpcEngine impl classes.
was:
For the rpc queue time calculation, we are using a not-recommended way: Using
{{Time#now}} to calculate for this.
{code}
// Invoke the protocol method
long startTime = Time.now();
int qTime = (int) (startTime-receivedTime);
server.updateMetrics(detailedMetricsName, qTime, processingTime, false);
{code}
Actually we should use {{Time#monotonicNow()}} instead. This JIRA will fix
these across RpcEngine impl classes.
> Do not use Time#now to calculate the rpc process time duration
> --
>
> Key: HADOOP-15768
> URL: https://issues.apache.org/jira/browse/HADOOP-15768
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Assignee: Ryan Wu
>Priority: Minor
>
> For the rpc process time calculation, we are using a not-recommended way:
> Using {{Time#now}} to calculate for this.
> {code}
> // Invoke the protocol method
> long startTime = Time.now();
> int qTime = (int) (startTime-receivedTime);
>
> int processingTime = (int) (Time.now() - startTime);
> ...
> server.updateMetrics(detailedMetricsName, qTime, processingTime, false);
> {code}
> Actually we should use {{Time#monotonicNow()}} instead. This JIRA will fix
> these across RpcEngine impl classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15768) Do not use Time#now to calculate the rpc process time duration
[
https://issues.apache.org/jira/browse/HADOOP-15768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-15768:
---
Summary: Do not use Time#now to calculate the rpc process time duration
(was: Do not use Time#now to calculate the rpc queue time duration)
> Do not use Time#now to calculate the rpc process time duration
> --
>
> Key: HADOOP-15768
> URL: https://issues.apache.org/jira/browse/HADOOP-15768
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Assignee: Ryan Wu
>Priority: Minor
>
> For the rpc queue time calculation, we are using a not-recommended way: Using
> {{Time#now}} to calculate for this.
> {code}
> // Invoke the protocol method
> long startTime = Time.now();
> int qTime = (int) (startTime-receivedTime);
>
> server.updateMetrics(detailedMetricsName, qTime, processingTime, false);
> {code}
> Actually we should use {{Time#monotonicNow()}} instead. This JIRA will fix
> these across RpcEngine impl classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15768) Do not use Time#now to calculate the rpc queue time duration
[
https://issues.apache.org/jira/browse/HADOOP-15768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-15768:
---
Description:
For the rpc queue time calculation, we are using a not-recommended way: Using
{{Time#now}} to calculate for this.
{code}
// Invoke the protocol method
long startTime = Time.now();
int qTime = (int) (startTime-receivedTime);
server.updateMetrics(detailedMetricsName, qTime, processingTime, false);
{code}
Actually we should use {{Time#monotonicNow()}} instead. This JIRA will fix
these across RpcEngine impl classes.
was:
For the rpc queue time calculation, we are using a not-recommended way: Using
{{Time#now}} to calculate for this.
{code}
// Invoke the protocol method
long startTime = Time.now();
int qTime = (int) (startTime-receivedTime);
server.updateMetrics(detailedMetricsName, qTime, processingTime, false);
{code}
We should use {{Time#monotonicNow()}} instead. This JIRA will fix these across
RpcEngine impl classes.
> Do not use Time#now to calculate the rpc queue time duration
>
>
> Key: HADOOP-15768
> URL: https://issues.apache.org/jira/browse/HADOOP-15768
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Assignee: Ryan Wu
>Priority: Minor
>
> For the rpc queue time calculation, we are using a not-recommended way: Using
> {{Time#now}} to calculate for this.
> {code}
> // Invoke the protocol method
> long startTime = Time.now();
> int qTime = (int) (startTime-receivedTime);
>
> server.updateMetrics(detailedMetricsName, qTime, processingTime, false);
> {code}
> Actually we should use {{Time#monotonicNow()}} instead. This JIRA will fix
> these across RpcEngine impl classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15768) Do not use Time#now to calculate the rpc queue time duration
Yiqun Lin created HADOOP-15768:
--
Summary: Do not use Time#now to calculate the rpc queue time
duration
Key: HADOOP-15768
URL: https://issues.apache.org/jira/browse/HADOOP-15768
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.1.1
Reporter: Yiqun Lin
Assignee: Ryan Wu
For the rpc queue time calculation, we are using a not-recommended way: Using
{{Time#now}} to calculate for this.
{code}
// Invoke the protocol method
long startTime = Time.now();
int qTime = (int) (startTime-receivedTime);
server.updateMetrics(detailedMetricsName, qTime, processingTime, false);
{code}
We should use {{Time#monotonicNow()}} instead. This JIRA will fix these across
RpcEngine impl classes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12980) Document RPC scheduler/callqueue configuration keys
[ https://issues.apache.org/jira/browse/HADOOP-12980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618719#comment-16618719 ] Yiqun Lin commented on HADOOP-12980: Current doc of these settings are missing and that makes hard for users to learn and use something like FCQ. Recently we are using FCQ and wanting to improves document work. [~xyao], can I take over this? Also I see this JIRA hasn't been updated for a long time. > Document RPC scheduler/callqueue configuration keys > --- > > Key: HADOOP-12980 > URL: https://issues.apache.org/jira/browse/HADOOP-12980 > Project: Hadoop Common > Issue Type: Sub-task > Components: documentation >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao >Priority: Major > > This ticket is opened to document RPC scheduler, callqueue and handler > related keys. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15742) Log if ipc backoff is enabled in CallQueueManager
[
https://issues.apache.org/jira/browse/HADOOP-15742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-15742:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.2.0
Status: Resolved (was: Patch Available)
Committed this to trunk.
Thanks [~jianliang.wu] for the contribution and also thanks additional reviews,
:).
> Log if ipc backoff is enabled in CallQueueManager
> -
>
> Key: HADOOP-15742
> URL: https://issues.apache.org/jira/browse/HADOOP-15742
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Assignee: Ryan Wu
>Priority: Minor
> Fix For: 3.2.0
>
> Attachments: HADOOP-15742.001.patch, HADOOP-15742.002.patch,
> HADOOP-15742.003.patch
>
>
> Currently we don't log the info of ipc backoff. It will look good to print
> this as well so that makes users know if we enable this.
> {code:java}
> public CallQueueManager(Class> backingClass,
> Class schedulerClass,
> boolean clientBackOffEnabled, int maxQueueSize, String namespace,
> Configuration conf) {
> int priorityLevels = parseNumLevels(namespace, conf);
> this.scheduler = createScheduler(schedulerClass, priorityLevels,
> namespace, conf);
> BlockingQueue bq = createCallQueueInstance(backingClass,
> priorityLevels, maxQueueSize, namespace, conf);
> this.clientBackOffEnabled = clientBackOffEnabled;
> this.putRef = new AtomicReference>(bq);
> this.takeRef = new AtomicReference>(bq);
> LOG.info("Using callQueue: " + backingClass + " queueCapacity: " +
> maxQueueSize + " scheduler: " + schedulerClass);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15742) Log if ipc backoff is enabled in CallQueueManager
[
https://issues.apache.org/jira/browse/HADOOP-15742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617124#comment-16617124
]
Yiqun Lin commented on HADOOP-15742:
LGTM, +1. Will hold off the commit until at the end of today in case there are
some other comments.
> Log if ipc backoff is enabled in CallQueueManager
> -
>
> Key: HADOOP-15742
> URL: https://issues.apache.org/jira/browse/HADOOP-15742
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Assignee: Ryan Wu
>Priority: Minor
> Attachments: HADOOP-15742.001.patch, HADOOP-15742.002.patch,
> HADOOP-15742.003.patch
>
>
> Currently we don't log the info of ipc backoff. It will look good to print
> this as well so that makes users know if we enable this.
> {code:java}
> public CallQueueManager(Class> backingClass,
> Class schedulerClass,
> boolean clientBackOffEnabled, int maxQueueSize, String namespace,
> Configuration conf) {
> int priorityLevels = parseNumLevels(namespace, conf);
> this.scheduler = createScheduler(schedulerClass, priorityLevels,
> namespace, conf);
> BlockingQueue bq = createCallQueueInstance(backingClass,
> priorityLevels, maxQueueSize, namespace, conf);
> this.clientBackOffEnabled = clientBackOffEnabled;
> this.putRef = new AtomicReference>(bq);
> this.takeRef = new AtomicReference>(bq);
> LOG.info("Using callQueue: " + backingClass + " queueCapacity: " +
> maxQueueSize + " scheduler: " + schedulerClass);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15742) Log if ipc backoff is enabled in CallQueueManager
[
https://issues.apache.org/jira/browse/HADOOP-15742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613579#comment-16613579
]
Yiqun Lin commented on HADOOP-15742:
Other than checkstyle issue, +1 for others.
> Log if ipc backoff is enabled in CallQueueManager
> -
>
> Key: HADOOP-15742
> URL: https://issues.apache.org/jira/browse/HADOOP-15742
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Assignee: Ryan Wu
>Priority: Minor
> Attachments: HADOOP-15742.001.patch, HADOOP-15742.002.patch
>
>
> Currently we don't log the info of ipc backoff. It will look good to print
> this as well so that makes users know if we enable this.
> {code:java}
> public CallQueueManager(Class> backingClass,
> Class schedulerClass,
> boolean clientBackOffEnabled, int maxQueueSize, String namespace,
> Configuration conf) {
> int priorityLevels = parseNumLevels(namespace, conf);
> this.scheduler = createScheduler(schedulerClass, priorityLevels,
> namespace, conf);
> BlockingQueue bq = createCallQueueInstance(backingClass,
> priorityLevels, maxQueueSize, namespace, conf);
> this.clientBackOffEnabled = clientBackOffEnabled;
> this.putRef = new AtomicReference>(bq);
> this.takeRef = new AtomicReference>(bq);
> LOG.info("Using callQueue: " + backingClass + " queueCapacity: " +
> maxQueueSize + " scheduler: " + schedulerClass);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-15742) Log if ipc backoff is enabled in CallQueueManager
[
https://issues.apache.org/jira/browse/HADOOP-15742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613159#comment-16613159
]
Yiqun Lin edited comment on HADOOP-15742 at 9/13/18 7:57 AM:
-
[~jianliang.wu]， could you use string interpolation way as [~ste...@apache.org]
mentioned? Update this like following:
{code:java}
LOG.info("Using callQueue: {}, queueCapacity: {}, " +
"scheduler: {}, ipcBackoff: {}.", schedulerClass, backingClass,
maxQueueSize, schedulerClass, clientBackOffEnabled);
{code}
was (Author: linyiqun):
[~jianliang.wu]， could you use string interpolation way as [~ste...@apache.org]
mentioned? Update this like following:
{code}
LOG.info("Using callQueue: {}, queueCapacity: {}, " +
" scheduler: {}, ipcBackoff: {}.", schedulerClass, backingClass,
maxQueueSize, schedulerClass, clientBackOffEnabled);
{code}
> Log if ipc backoff is enabled in CallQueueManager
> -
>
> Key: HADOOP-15742
> URL: https://issues.apache.org/jira/browse/HADOOP-15742
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Assignee: Ryan Wu
>Priority: Minor
> Attachments: HADOOP-15742.001.patch
>
>
> Currently we don't log the info of ipc backoff. It will look good to print
> this as well so that makes users know if we enable this.
> {code:java}
> public CallQueueManager(Class> backingClass,
> Class schedulerClass,
> boolean clientBackOffEnabled, int maxQueueSize, String namespace,
> Configuration conf) {
> int priorityLevels = parseNumLevels(namespace, conf);
> this.scheduler = createScheduler(schedulerClass, priorityLevels,
> namespace, conf);
> BlockingQueue bq = createCallQueueInstance(backingClass,
> priorityLevels, maxQueueSize, namespace, conf);
> this.clientBackOffEnabled = clientBackOffEnabled;
> this.putRef = new AtomicReference>(bq);
> this.takeRef = new AtomicReference>(bq);
> LOG.info("Using callQueue: " + backingClass + " queueCapacity: " +
> maxQueueSize + " scheduler: " + schedulerClass);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15742) Log if ipc backoff is enabled in CallQueueManager
[
https://issues.apache.org/jira/browse/HADOOP-15742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613159#comment-16613159
]
Yiqun Lin commented on HADOOP-15742:
[~jianliang.wu]， could you use string interpolation way as [~ste...@apache.org]
mentioned? Update this like following:
{code}
LOG.info("Using callQueue: {}, queueCapacity: {}, " +
" scheduler: {}, ipcBackoff: {}.", schedulerClass, backingClass,
maxQueueSize, schedulerClass, clientBackOffEnabled);
{code}
> Log if ipc backoff is enabled in CallQueueManager
> -
>
> Key: HADOOP-15742
> URL: https://issues.apache.org/jira/browse/HADOOP-15742
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Assignee: Ryan Wu
>Priority: Minor
> Attachments: HADOOP-15742.001.patch
>
>
> Currently we don't log the info of ipc backoff. It will look good to print
> this as well so that makes users know if we enable this.
> {code:java}
> public CallQueueManager(Class> backingClass,
> Class schedulerClass,
> boolean clientBackOffEnabled, int maxQueueSize, String namespace,
> Configuration conf) {
> int priorityLevels = parseNumLevels(namespace, conf);
> this.scheduler = createScheduler(schedulerClass, priorityLevels,
> namespace, conf);
> BlockingQueue bq = createCallQueueInstance(backingClass,
> priorityLevels, maxQueueSize, namespace, conf);
> this.clientBackOffEnabled = clientBackOffEnabled;
> this.putRef = new AtomicReference>(bq);
> this.takeRef = new AtomicReference>(bq);
> LOG.info("Using callQueue: " + backingClass + " queueCapacity: " +
> maxQueueSize + " scheduler: " + schedulerClass);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15742) Log if ipc backoff is enabled in CallQueueManager
[
https://issues.apache.org/jira/browse/HADOOP-15742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-15742:
---
Status: Patch Available (was: Open)
> Log if ipc backoff is enabled in CallQueueManager
> -
>
> Key: HADOOP-15742
> URL: https://issues.apache.org/jira/browse/HADOOP-15742
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Assignee: Ryan Wu
>Priority: Minor
> Attachments: HADOOP-15742.001.patch
>
>
> Currently we don't log the info of ipc backoff. It will look good to print
> this as well so that makes users know if we enable this.
> {code:java}
> public CallQueueManager(Class> backingClass,
> Class schedulerClass,
> boolean clientBackOffEnabled, int maxQueueSize, String namespace,
> Configuration conf) {
> int priorityLevels = parseNumLevels(namespace, conf);
> this.scheduler = createScheduler(schedulerClass, priorityLevels,
> namespace, conf);
> BlockingQueue bq = createCallQueueInstance(backingClass,
> priorityLevels, maxQueueSize, namespace, conf);
> this.clientBackOffEnabled = clientBackOffEnabled;
> this.putRef = new AtomicReference>(bq);
> this.takeRef = new AtomicReference>(bq);
> LOG.info("Using callQueue: " + backingClass + " queueCapacity: " +
> maxQueueSize + " scheduler: " + schedulerClass);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15742) Log if ipc backoff is enabled in CallQueueManager
[
https://issues.apache.org/jira/browse/HADOOP-15742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16612033#comment-16612033
]
Yiqun Lin commented on HADOOP-15742:
{quote}Is this going to flood the logs if @ info?.
{quote}
Hi [~ste...@apache.org], this won't flood the log, we just add ipc backoff info
in existing info. This only print once when construct the call queue manager
{noformat}
LOG.info("Using callQueue: " + backingClass + " queueCapacity: " +
maxQueueSize + " scheduler: " + schedulerClass);
{noformat}
> Log if ipc backoff is enabled in CallQueueManager
> -
>
> Key: HADOOP-15742
> URL: https://issues.apache.org/jira/browse/HADOOP-15742
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Assignee: Ryan Wu
>Priority: Minor
>
> Currently we don't log the info of ipc backoff. It will look good to print
> this as well so that makes users know if we enable this.
> {code:java}
> public CallQueueManager(Class> backingClass,
> Class schedulerClass,
> boolean clientBackOffEnabled, int maxQueueSize, String namespace,
> Configuration conf) {
> int priorityLevels = parseNumLevels(namespace, conf);
> this.scheduler = createScheduler(schedulerClass, priorityLevels,
> namespace, conf);
> BlockingQueue bq = createCallQueueInstance(backingClass,
> priorityLevels, maxQueueSize, namespace, conf);
> this.clientBackOffEnabled = clientBackOffEnabled;
> this.putRef = new AtomicReference>(bq);
> this.takeRef = new AtomicReference>(bq);
> LOG.info("Using callQueue: " + backingClass + " queueCapacity: " +
> maxQueueSize + " scheduler: " + schedulerClass);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15742) Log if ipc backoff is enabled in CallQueueManager
[
https://issues.apache.org/jira/browse/HADOOP-15742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-15742:
---
Summary: Log if ipc backoff is enabled in CallQueueManager (was: Log if
the ipc backoff is enabled in CallQueueManager)
> Log if ipc backoff is enabled in CallQueueManager
> -
>
> Key: HADOOP-15742
> URL: https://issues.apache.org/jira/browse/HADOOP-15742
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.1.1
>Reporter: Yiqun Lin
>Assignee: Ryan Wu
>Priority: Minor
>
> Currently we don't log the info of ipc backoff. It will look good to print
> this as well so that makes users know if we enable this.
> {code:java}
> public CallQueueManager(Class> backingClass,
> Class schedulerClass,
> boolean clientBackOffEnabled, int maxQueueSize, String namespace,
> Configuration conf) {
> int priorityLevels = parseNumLevels(namespace, conf);
> this.scheduler = createScheduler(schedulerClass, priorityLevels,
> namespace, conf);
> BlockingQueue bq = createCallQueueInstance(backingClass,
> priorityLevels, maxQueueSize, namespace, conf);
> this.clientBackOffEnabled = clientBackOffEnabled;
> this.putRef = new AtomicReference>(bq);
> this.takeRef = new AtomicReference>(bq);
> LOG.info("Using callQueue: " + backingClass + " queueCapacity: " +
> maxQueueSize + " scheduler: " + schedulerClass);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15742) Log if the ipc backoff is enabled in CallQueueManager
Yiqun Lin created HADOOP-15742:
--
Summary: Log if the ipc backoff is enabled in CallQueueManager
Key: HADOOP-15742
URL: https://issues.apache.org/jira/browse/HADOOP-15742
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 3.1.1
Reporter: Yiqun Lin
Assignee: Ryan Wu
Currently we don't log the info of ipc backoff. It will look good to print this
as well so that makes users know if we enable this.
{code:java}
public CallQueueManager(Class> backingClass,
Class schedulerClass,
boolean clientBackOffEnabled, int maxQueueSize, String namespace,
Configuration conf) {
int priorityLevels = parseNumLevels(namespace, conf);
this.scheduler = createScheduler(schedulerClass, priorityLevels,
namespace, conf);
BlockingQueue bq = createCallQueueInstance(backingClass,
priorityLevels, maxQueueSize, namespace, conf);
this.clientBackOffEnabled = clientBackOffEnabled;
this.putRef = new AtomicReference>(bq);
this.takeRef = new AtomicReference>(bq);
LOG.info("Using callQueue: " + backingClass + " queueCapacity: " +
maxQueueSize + " scheduler: " + schedulerClass);
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15611) Log more details for FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin reassigned HADOOP-15611: -- Assignee: Ryan Wu > Log more details for FairCallQueue > -- > > Key: HADOOP-15611 > URL: https://issues.apache.org/jira/browse/HADOOP-15611 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.1.0 >Reporter: Ryan Wu >Assignee: Ryan Wu >Priority: Minor > Fix For: 3.2.0 > > Attachments: HADOOP-15611.001.patch, HADOOP-15611.002.patch, > HADOOP-15611.003.patch, HADOOP-15611.004.patch > > > In the usage of the FairCallQueue, we find there missing some Key log. Only a > few logs are printed, it makes us hard to learn and debug this feature. > At least, following places can print more logs. > * DecayRpcScheduler#decayCurrentCounts > * WeightedRoundRobinMultiplexer#moveToNextQueue -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15611) Log more details for FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-15611: --- Fix Version/s: (was: 3.2) 3.2.0 > Log more details for FairCallQueue > -- > > Key: HADOOP-15611 > URL: https://issues.apache.org/jira/browse/HADOOP-15611 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.1.0 >Reporter: Ryan Wu >Priority: Minor > Fix For: 3.2.0 > > Attachments: HADOOP-15611.001.patch, HADOOP-15611.002.patch, > HADOOP-15611.003.patch, HADOOP-15611.004.patch > > > In the usage of the FairCallQueue, we find there missing some Key log. Only a > few logs are printed, it makes us hard to learn and debug this feature. > At least, following places can print more logs. > * DecayRpcScheduler#decayCurrentCounts > * WeightedRoundRobinMultiplexer#moveToNextQueue -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15611) Log more details for FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-15611: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.2 Target Version/s: 3.2.0 (was: 3.2.0, 3.1.1) Status: Resolved (was: Patch Available) Committed this to trunk. Thanks [~jianliang.wu] for the contribution! > Log more details for FairCallQueue > -- > > Key: HADOOP-15611 > URL: https://issues.apache.org/jira/browse/HADOOP-15611 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.1.0 >Reporter: Ryan Wu >Priority: Minor > Fix For: 3.2 > > Attachments: HADOOP-15611.001.patch, HADOOP-15611.002.patch, > HADOOP-15611.003.patch, HADOOP-15611.004.patch > > > In the usage of the FairCallQueue, we find there missing some Key log. Only a > few logs are printed, it makes us hard to learn and debug this feature. > At least, following places can print more logs. > * DecayRpcScheduler#decayCurrentCounts > * WeightedRoundRobinMultiplexer#moveToNextQueue -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15611) Log more details for FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558125#comment-16558125 ] Yiqun Lin commented on HADOOP-15611: LGTM, +1. Committing this. > Log more details for FairCallQueue > -- > > Key: HADOOP-15611 > URL: https://issues.apache.org/jira/browse/HADOOP-15611 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.1.0 >Reporter: Ryan Wu >Priority: Minor > Attachments: HADOOP-15611.001.patch, HADOOP-15611.002.patch, > HADOOP-15611.003.patch, HADOOP-15611.004.patch > > > In the usage of the FairCallQueue, we find there missing some Key log. Only a > few logs are printed, it makes us hard to learn and debug this feature. > At least, following places can print more logs. > * DecayRpcScheduler#decayCurrentCounts > * WeightedRoundRobinMultiplexer#moveToNextQueue -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15611) Log more details for FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-15611: --- Summary: Log more details for FairCallQueue (was: Improve log in FairCallQueue) > Log more details for FairCallQueue > -- > > Key: HADOOP-15611 > URL: https://issues.apache.org/jira/browse/HADOOP-15611 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.1.0 >Reporter: Ryan Wu >Priority: Minor > Attachments: HADOOP-15611.001.patch, HADOOP-15611.002.patch, > HADOOP-15611.003.patch, HADOOP-15611.004.patch > > > In the usage of the FairCallQueue, we find there missing some Key log. Only a > few logs are printed, it makes us hard to learn and debug this feature. > At least, following places can print more logs. > * DecayRpcScheduler#decayCurrentCounts > * WeightedRoundRobinMultiplexer#moveToNextQueue -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-15611) Improve log in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553932#comment-16553932
]
Yiqun Lin edited comment on HADOOP-15611 at 7/24/18 9:14 AM:
-
Revisiting this, I suggest we can add one additional log to print detail decay
info for each user in the loop of {{DecayRpcScheduler#decayCurrentCounts}}
was (Author: linyiqun):
LGTM, +1. Commit this shortly, :).
> Improve log in FairCallQueue
>
>
> Key: HADOOP-15611
> URL: https://issues.apache.org/jira/browse/HADOOP-15611
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.1.0
>Reporter: Ryan Wu
>Priority: Minor
> Attachments: HADOOP-15611.001.patch, HADOOP-15611.002.patch,
> HADOOP-15611.003.patch
>
>
> In the usage of the FairCallQueue, we find there missing some Key log. Only a
> few logs are printed, it makes us hard to learn and debug this feature.
> At least, following places can print more logs.
> * DecayRpcScheduler#decayCurrentCounts
> * WeightedRoundRobinMultiplexer#moveToNextQueue
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15611) Improve log in FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553697#comment-16553697 ] Yiqun Lin commented on HADOOP-15611: Almost looks great to me now. [~jianliang.wu], would you mind adding some output logs tested in your local and then we can see if the log is we wanted. +1 once addressed. > Improve log in FairCallQueue > > > Key: HADOOP-15611 > URL: https://issues.apache.org/jira/browse/HADOOP-15611 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.1.0 >Reporter: Ryan Wu >Priority: Minor > Attachments: HADOOP-15611.001.patch, HADOOP-15611.002.patch, > HADOOP-15611.003.patch > > > In the usage of the FairCallQueue, we find there missing some Key log. Only a > few logs are printed, it makes us hard to learn and debug this feature. > At least, following places can print more logs. > * DecayRpcScheduler#decayCurrentCounts > * WeightedRoundRobinMultiplexer#moveToNextQueue -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15611) Improve log in FairCallQueue
[
https://issues.apache.org/jira/browse/HADOOP-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552253#comment-16552253
]
Yiqun Lin commented on HADOOP-15611:
Thanks [~jianliang.wu] for attaching the patch! Some minor comments from me:
* Use DEBUG log level to print for {{Start decay current counts.}}, and update
{{Start}} to {{Start to}}.
* {{"The decayedCount for this user "+entry.getKey().toString()+" is zero"}}
can be updated as {{"The decayed count for the user {} is zero and being
cleaned.",
* We need to add a space before {{totalRawCallCount}} in {{"After decaying the
stored counts, totalDecayedCount: {}," + "totalRawCallCount: {}.}}
* Remove following unrelated change:
{code}
if (totalCallSnapshot > 0) {
proportion = (double) occurrences / totalCallSnapshot;
}
-
// Start with low priority levels, since they will be most common
{code}
* {{"Moving to next queue, next queue index : {}," + "next queue weight : {},
the requestsLeft : {}."}} can be updated as {{"Moving to next queue from queue
index {} to index {}, number of requests left for current queue: {}."
BTW, you should also format these lines.
> Improve log in FairCallQueue
>
>
> Key: HADOOP-15611
> URL: https://issues.apache.org/jira/browse/HADOOP-15611
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.1.0
>Reporter: Ryan Wu
>Priority: Minor
> Attachments: HADOOP-15611.001.patch
>
>
> In the usage of the FairCallQueue, we find there missing some Key log. Only a
> few logs are printed, it makes us hard to learn and debug this feature.
> At least, following places can print more logs.
> * DecayRpcScheduler#decayCurrentCounts
> * WeightedRoundRobinMultiplexer#moveToNextQueue
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15611) Improve log in FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546080#comment-16546080 ] Yiqun Lin commented on HADOOP-15611: Anyone who can assign this JIRA to [~jianliang.wu]? He is willing to make contribution on this, :). > Improve log in FairCallQueue > > > Key: HADOOP-15611 > URL: https://issues.apache.org/jira/browse/HADOOP-15611 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.1.0 >Reporter: Ryan Wu >Priority: Minor > > In the usage of the FairCallQueue, we find there missing some Key log. Only a > few logs are printed, it makes us hard to learn and debug this feature. > At least, following places can print more logs. > * DecayRpcScheduler#decayCurrentCounts > * WeightedRoundRobinMultiplexer#moveToNextQueue -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15611) Improve log in FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-15611: --- Description: In the usage of the FairCallQueue, we find there missing some Key log. Only a few logs are printed, it makes us hard to learn and debug this feature. At least, following places can print more logs. * DecayRpcScheduler#decayCurrentCounts * WeightedRoundRobinMultiplexer#moveToNextQueue > Improve log in FairCallQueue > > > Key: HADOOP-15611 > URL: https://issues.apache.org/jira/browse/HADOOP-15611 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.1.0 >Reporter: Ryan Wu >Priority: Minor > > In the usage of the FairCallQueue, we find there missing some Key log. Only a > few logs are printed, it makes us hard to learn and debug this feature. > At least, following places can print more logs. > * DecayRpcScheduler#decayCurrentCounts > * WeightedRoundRobinMultiplexer#moveToNextQueue -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-9640) RPC Congestion Control with FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-9640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526157#comment-16526157 ] Yiqun Lin edited comment on HADOOP-9640 at 6/28/18 9:39 AM: Hi all, Any progress on this issue? This looks like a good feature and can be used in production env, any documentation where we can learn from that how to configure using this? Appreciate for this, :). was (Author: linyiqun): Hi all, Any progress on this issue? This looks like a good feature and can be used in production env, any documentation where we can learn from that how to configure using this? Appropriate for this, :). > RPC Congestion Control with FairCallQueue > - > > Key: HADOOP-9640 > URL: https://issues.apache.org/jira/browse/HADOOP-9640 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.2.0, 3.0.0-alpha1 >Reporter: Xiaobo Peng >Assignee: Chris Li >Priority: Major > Labels: hdfs, qos, rpc > Attachments: FairCallQueue-PerformanceOnCluster.pdf, > MinorityMajorityPerformance.pdf, NN-denial-of-service-updated-plan.pdf, > faircallqueue.patch, faircallqueue2.patch, faircallqueue3.patch, > faircallqueue4.patch, faircallqueue5.patch, faircallqueue6.patch, > faircallqueue7_with_runtime_swapping.patch, > rpc-congestion-control-draft-plan.pdf > > > For an easy-to-read summary see: > http://www.ebaytechblog.com/2014/08/21/quality-of-service-in-hadoop/ > Several production Hadoop cluster incidents occurred where the Namenode was > overloaded and failed to respond. > We can improve quality of service for users during namenode peak loads by > replacing the FIFO call queue with a [Fair Call > Queue|https://issues.apache.org/jira/secure/attachment/12616864/NN-denial-of-service-updated-plan.pdf]. > (this plan supersedes rpc-congestion-control-draft-plan). > Excerpted from the communication of one incident, “The map task of a user was > creating huge number of small files in the user directory. Due to the heavy > load on NN, the JT also was unable to communicate with NN...The cluster > became responsive only once the job was killed.” > Excerpted from the communication of another incident, “Namenode was > overloaded by GetBlockLocation requests (Correction: should be getFileInfo > requests. the job had a bug that called getFileInfo for a nonexistent file in > an endless loop). All other requests to namenode were also affected by this > and hence all jobs slowed down. Cluster almost came to a grinding > halt…Eventually killed jobtracker to kill all jobs that are running.” > Excerpted from HDFS-945, “We've seen defective applications cause havoc on > the NameNode, for e.g. by doing 100k+ 'listStatus' on very large directories > (60k files) etc.” -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-9640) RPC Congestion Control with FairCallQueue
[ https://issues.apache.org/jira/browse/HADOOP-9640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526157#comment-16526157 ] Yiqun Lin commented on HADOOP-9640: --- Hi all, Any progress on this issue? This looks like a good feature and can be used in production env, any documentation where we can learn from that how to configure using this? Appropriate for this, :). > RPC Congestion Control with FairCallQueue > - > > Key: HADOOP-9640 > URL: https://issues.apache.org/jira/browse/HADOOP-9640 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.2.0, 3.0.0-alpha1 >Reporter: Xiaobo Peng >Assignee: Chris Li >Priority: Major > Labels: hdfs, qos, rpc > Attachments: FairCallQueue-PerformanceOnCluster.pdf, > MinorityMajorityPerformance.pdf, NN-denial-of-service-updated-plan.pdf, > faircallqueue.patch, faircallqueue2.patch, faircallqueue3.patch, > faircallqueue4.patch, faircallqueue5.patch, faircallqueue6.patch, > faircallqueue7_with_runtime_swapping.patch, > rpc-congestion-control-draft-plan.pdf > > > For an easy-to-read summary see: > http://www.ebaytechblog.com/2014/08/21/quality-of-service-in-hadoop/ > Several production Hadoop cluster incidents occurred where the Namenode was > overloaded and failed to respond. > We can improve quality of service for users during namenode peak loads by > replacing the FIFO call queue with a [Fair Call > Queue|https://issues.apache.org/jira/secure/attachment/12616864/NN-denial-of-service-updated-plan.pdf]. > (this plan supersedes rpc-congestion-control-draft-plan). > Excerpted from the communication of one incident, “The map task of a user was > creating huge number of small files in the user directory. Due to the heavy > load on NN, the JT also was unable to communicate with NN...The cluster > became responsive only once the job was killed.” > Excerpted from the communication of another incident, “Namenode was > overloaded by GetBlockLocation requests (Correction: should be getFileInfo > requests. the job had a bug that called getFileInfo for a nonexistent file in > an endless loop). All other requests to namenode were also affected by this > and hence all jobs slowed down. Cluster almost came to a grinding > halt…Eventually killed jobtracker to kill all jobs that are running.” > Excerpted from HDFS-945, “We've seen defective applications cause havoc on > the NameNode, for e.g. by doing 100k+ 'listStatus' on very large directories > (60k files) etc.” -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15415) copyBytes hangs when the configuration file is corrupted
[
https://issues.apache.org/jira/browse/HADOOP-15415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-15415:
---
Attachment: HADOOP-15415.001.patch
> copyBytes hangs when the configuration file is corrupted
>
>
> Key: HADOOP-15415
> URL: https://issues.apache.org/jira/browse/HADOOP-15415
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 0.23.0, 2.5.0, 3.1.0
>Reporter: John Doe
>Assignee: Yiqun Lin
>Priority: Minor
> Attachments: HADOOP-15415.001.patch
>
>
> The third parameter, buffSize, is read from the configuration files or
> user-specified.
> When the configuration file is corrupted or the user configures with a wrong
> value, i.e., 0, the bytesRead will always be 0, making the while loop's
> condition always true, hanging IOUtils.copyBytes() endlessly.
> Here is the snippet of the code. There are four copyBytes in the following
> code. The 3rd and 4th copyBytes calls the 1st one. The 1st one calls the 2nd
> one. Hang happens in the while loop of the second copyBytes function.
>
> {code:java}
> //1st copyBytes
> public static void copyBytes(InputStream in, OutputStream out, int
> buffSize, boolean close)
> throws IOException {
> try {
> copyBytes(in, out, buffSize);
> if(close) {
> out.close();
> out = null;
> in.close();
> in = null;
> }
> } finally {
> if(close) {
> closeStream(out);
> closeStream(in);
> }
> }
> }
>
> //2nd copyBytes
> public static void copyBytes(InputStream in, OutputStream out, int
> buffSize)
> throws IOException {
> PrintStream ps = out instanceof PrintStream ? (PrintStream)out : null;
> byte buf[] = new byte[buffSize];
> int bytesRead = in.read(buf);
> while (bytesRead >= 0) {
> out.write(buf, 0, bytesRead);
> if ((ps != null) && ps.checkError()) {
> throw new IOException("Unable to write to output stream.");
> }
> bytesRead = in.read(buf);
> }
> }
> //3rd copyBytes
> public static void copyBytes(InputStream in, OutputStream out,
> Configuration conf)
> throws IOException {
> copyBytes(in, out, conf.getInt("io.file.buffer.size", 4096), true);
> }
>
> //4th copyBytes
> public static void copyBytes(InputStream in, OutputStream out,
> Configuration conf, boolean close)
> throws IOException {
> copyBytes(in, out, conf.getInt("io.file.buffer.size", 4096), close);
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15415) copyBytes hangs when the configuration file is corrupted
[
https://issues.apache.org/jira/browse/HADOOP-15415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-15415:
---
Status: Patch Available (was: Open)
> copyBytes hangs when the configuration file is corrupted
>
>
> Key: HADOOP-15415
> URL: https://issues.apache.org/jira/browse/HADOOP-15415
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 2.5.0, 0.23.0, 3.1.0
>Reporter: John Doe
>Assignee: Yiqun Lin
>Priority: Minor
> Attachments: HADOOP-15415.001.patch
>
>
> The third parameter, buffSize, is read from the configuration files or
> user-specified.
> When the configuration file is corrupted or the user configures with a wrong
> value, i.e., 0, the bytesRead will always be 0, making the while loop's
> condition always true, hanging IOUtils.copyBytes() endlessly.
> Here is the snippet of the code. There are four copyBytes in the following
> code. The 3rd and 4th copyBytes calls the 1st one. The 1st one calls the 2nd
> one. Hang happens in the while loop of the second copyBytes function.
>
> {code:java}
> //1st copyBytes
> public static void copyBytes(InputStream in, OutputStream out, int
> buffSize, boolean close)
> throws IOException {
> try {
> copyBytes(in, out, buffSize);
> if(close) {
> out.close();
> out = null;
> in.close();
> in = null;
> }
> } finally {
> if(close) {
> closeStream(out);
> closeStream(in);
> }
> }
> }
>
> //2nd copyBytes
> public static void copyBytes(InputStream in, OutputStream out, int
> buffSize)
> throws IOException {
> PrintStream ps = out instanceof PrintStream ? (PrintStream)out : null;
> byte buf[] = new byte[buffSize];
> int bytesRead = in.read(buf);
> while (bytesRead >= 0) {
> out.write(buf, 0, bytesRead);
> if ((ps != null) && ps.checkError()) {
> throw new IOException("Unable to write to output stream.");
> }
> bytesRead = in.read(buf);
> }
> }
> //3rd copyBytes
> public static void copyBytes(InputStream in, OutputStream out,
> Configuration conf)
> throws IOException {
> copyBytes(in, out, conf.getInt("io.file.buffer.size", 4096), true);
> }
>
> //4th copyBytes
> public static void copyBytes(InputStream in, OutputStream out,
> Configuration conf, boolean close)
> throws IOException {
> copyBytes(in, out, conf.getInt("io.file.buffer.size", 4096), close);
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15415) copyBytes hangs when the configuration file is corrupted
[
https://issues.apache.org/jira/browse/HADOOP-15415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483596#comment-16483596
]
Yiqun Lin commented on HADOOP-15415:
I'd like to fix this, :).
> copyBytes hangs when the configuration file is corrupted
>
>
> Key: HADOOP-15415
> URL: https://issues.apache.org/jira/browse/HADOOP-15415
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 0.23.0, 2.5.0, 3.1.0
>Reporter: John Doe
>Priority: Minor
>
> The third parameter, buffSize, is read from the configuration files or
> user-specified.
> When the configuration file is corrupted or the user configures with a wrong
> value, i.e., 0, the bytesRead will always be 0, making the while loop's
> condition always true, hanging IOUtils.copyBytes() endlessly.
> Here is the snippet of the code. There are four copyBytes in the following
> code. The 3rd and 4th copyBytes calls the 1st one. The 1st one calls the 2nd
> one. Hang happens in the while loop of the second copyBytes function.
>
> {code:java}
> //1st copyBytes
> public static void copyBytes(InputStream in, OutputStream out, int
> buffSize, boolean close)
> throws IOException {
> try {
> copyBytes(in, out, buffSize);
> if(close) {
> out.close();
> out = null;
> in.close();
> in = null;
> }
> } finally {
> if(close) {
> closeStream(out);
> closeStream(in);
> }
> }
> }
>
> //2nd copyBytes
> public static void copyBytes(InputStream in, OutputStream out, int
> buffSize)
> throws IOException {
> PrintStream ps = out instanceof PrintStream ? (PrintStream)out : null;
> byte buf[] = new byte[buffSize];
> int bytesRead = in.read(buf);
> while (bytesRead >= 0) {
> out.write(buf, 0, bytesRead);
> if ((ps != null) && ps.checkError()) {
> throw new IOException("Unable to write to output stream.");
> }
> bytesRead = in.read(buf);
> }
> }
> //3rd copyBytes
> public static void copyBytes(InputStream in, OutputStream out,
> Configuration conf)
> throws IOException {
> copyBytes(in, out, conf.getInt("io.file.buffer.size", 4096), true);
> }
>
> //4th copyBytes
> public static void copyBytes(InputStream in, OutputStream out,
> Configuration conf, boolean close)
> throws IOException {
> copyBytes(in, out, conf.getInt("io.file.buffer.size", 4096), close);
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15415) copyBytes hangs when the configuration file is corrupted
[
https://issues.apache.org/jira/browse/HADOOP-15415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin reassigned HADOOP-15415:
--
Assignee: Yiqun Lin
> copyBytes hangs when the configuration file is corrupted
>
>
> Key: HADOOP-15415
> URL: https://issues.apache.org/jira/browse/HADOOP-15415
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 0.23.0, 2.5.0, 3.1.0
>Reporter: John Doe
>Assignee: Yiqun Lin
>Priority: Minor
>
> The third parameter, buffSize, is read from the configuration files or
> user-specified.
> When the configuration file is corrupted or the user configures with a wrong
> value, i.e., 0, the bytesRead will always be 0, making the while loop's
> condition always true, hanging IOUtils.copyBytes() endlessly.
> Here is the snippet of the code. There are four copyBytes in the following
> code. The 3rd and 4th copyBytes calls the 1st one. The 1st one calls the 2nd
> one. Hang happens in the while loop of the second copyBytes function.
>
> {code:java}
> //1st copyBytes
> public static void copyBytes(InputStream in, OutputStream out, int
> buffSize, boolean close)
> throws IOException {
> try {
> copyBytes(in, out, buffSize);
> if(close) {
> out.close();
> out = null;
> in.close();
> in = null;
> }
> } finally {
> if(close) {
> closeStream(out);
> closeStream(in);
> }
> }
> }
>
> //2nd copyBytes
> public static void copyBytes(InputStream in, OutputStream out, int
> buffSize)
> throws IOException {
> PrintStream ps = out instanceof PrintStream ? (PrintStream)out : null;
> byte buf[] = new byte[buffSize];
> int bytesRead = in.read(buf);
> while (bytesRead >= 0) {
> out.write(buf, 0, bytesRead);
> if ((ps != null) && ps.checkError()) {
> throw new IOException("Unable to write to output stream.");
> }
> bytesRead = in.read(buf);
> }
> }
> //3rd copyBytes
> public static void copyBytes(InputStream in, OutputStream out,
> Configuration conf)
> throws IOException {
> copyBytes(in, out, conf.getInt("io.file.buffer.size", 4096), true);
> }
>
> //4th copyBytes
> public static void copyBytes(InputStream in, OutputStream out,
> Configuration conf, boolean close)
> throws IOException {
> copyBytes(in, out, conf.getInt("io.file.buffer.size", 4096), close);
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15391) Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules
[ https://issues.apache.org/jira/browse/HADOOP-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-15391: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.1 3.2.0 Status: Resolved (was: Patch Available) Committed to trunk and branch-3.1. Thanks [~ajisakaa] for the review. > Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake modules > - > > Key: HADOOP-15391 > URL: https://issues.apache.org/jira/browse/HADOOP-15391 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Major > Fix For: 3.2.0, 3.1.1 > > Attachments: HADOOP-15391.001.patch, screen-shot-fixed.jpg, > screen-shot.jpg > > > The documentation pages for hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake render error (see screen-shot attached or > [here|http://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html]). > The reason of this is that the css file is missing in these modules. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15391) Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules
[ https://issues.apache.org/jira/browse/HADOOP-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442023#comment-16442023 ] Yiqun Lin commented on HADOOP-15391: Thanks for the review, [~ajisakaa]. The Jenkins looks good. Will commit shortly, :). > Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake modules > - > > Key: HADOOP-15391 > URL: https://issues.apache.org/jira/browse/HADOOP-15391 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Major > Attachments: HADOOP-15391.001.patch, screen-shot-fixed.jpg, > screen-shot.jpg > > > The documentation pages for hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake render error (see screen-shot attached or > [here|http://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html]). > The reason of this is that the css file is missing in these modules. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15391) Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules
[ https://issues.apache.org/jira/browse/HADOOP-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-15391: --- Attachment: HADOOP-15391.001.patch > Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake modules > - > > Key: HADOOP-15391 > URL: https://issues.apache.org/jira/browse/HADOOP-15391 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Major > Attachments: HADOOP-15391.001.patch, screen-shot-fixed.jpg, > screen-shot.jpg > > > The documentation pages for hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake render error (see screen-shot attached or > [here|http://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html]). > The reason of this is that the css file is missing in these modules. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15391) Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules
[ https://issues.apache.org/jira/browse/HADOOP-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-15391: --- Attachment: (was: HADOOP-15391.001.patch) > Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake modules > - > > Key: HADOOP-15391 > URL: https://issues.apache.org/jira/browse/HADOOP-15391 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Major > Attachments: HADOOP-15391.001.patch, screen-shot-fixed.jpg, > screen-shot.jpg > > > The documentation pages for hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake render error (see screen-shot attached or > [here|http://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html]). > The reason of this is that the css file is missing in these modules. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15391) Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules
[ https://issues.apache.org/jira/browse/HADOOP-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-15391: --- Attachment: screen-shot-fixed.jpg > Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake modules > - > > Key: HADOOP-15391 > URL: https://issues.apache.org/jira/browse/HADOOP-15391 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Major > Attachments: HADOOP-15391.001.patch, screen-shot-fixed.jpg, > screen-shot.jpg > > > The documentation pages for hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake render error (see screen-shot attached or > [here|http://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html]). > The reason of this is that the css file is missing in these modules. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15391) Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules
[ https://issues.apache.org/jira/browse/HADOOP-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-15391: --- Status: Patch Available (was: Open) > Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake modules > - > > Key: HADOOP-15391 > URL: https://issues.apache.org/jira/browse/HADOOP-15391 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Major > Attachments: HADOOP-15391.001.patch, screen-shot.jpg > > > The documentation pages for hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake render error (see screen-shot attached or > [here|http://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html]). > The reason of this is that the css file is missing in these modules. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15391) Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules
[ https://issues.apache.org/jira/browse/HADOOP-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-15391: --- Attachment: HADOOP-15391.001.patch > Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake modules > - > > Key: HADOOP-15391 > URL: https://issues.apache.org/jira/browse/HADOOP-15391 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Major > Attachments: HADOOP-15391.001.patch, screen-shot.jpg > > > The documentation pages for hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake render error (see screen-shot attached or > [here|http://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html]). > The reason of this is that the css file is missing in these modules. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15391) Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules
[ https://issues.apache.org/jira/browse/HADOOP-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440560#comment-16440560 ] Yiqun Lin commented on HADOOP-15391: Attach the patch for adding the css file and attach the new screen-shot by fixed. > Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake modules > - > > Key: HADOOP-15391 > URL: https://issues.apache.org/jira/browse/HADOOP-15391 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Major > Attachments: HADOOP-15391.001.patch, screen-shot.jpg > > > The documentation pages for hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake render error (see screen-shot attached or > [here|http://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html]). > The reason of this is that the css file is missing in these modules. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15391) Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules
[ https://issues.apache.org/jira/browse/HADOOP-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-15391: --- Attachment: screen-shot.jpg > Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake modules > - > > Key: HADOOP-15391 > URL: https://issues.apache.org/jira/browse/HADOOP-15391 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Major > Attachments: screen-shot.jpg > > > The documentation pages for hadoop-aws, hadoop-aliyun, hadoop-azure and > hadoop-azure-datalake render error (see screen-shot attached or > [here|http://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html]). > The reason of this is that the css file is missing in these modules. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15391) Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules
Yiqun Lin created HADOOP-15391: -- Summary: Add missing css file in hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake modules Key: HADOOP-15391 URL: https://issues.apache.org/jira/browse/HADOOP-15391 Project: Hadoop Common Issue Type: Bug Components: documentation Affects Versions: 3.0.1 Reporter: Yiqun Lin Assignee: Yiqun Lin The documentation pages for hadoop-aws, hadoop-aliyun, hadoop-azure and hadoop-azure-datalake render error (see screen-shot attached or [here|http://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html]). The reason of this is that the css file is missing in these modules. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15296) Fix a wrong link for RBF in the top page
[ https://issues.apache.org/jira/browse/HADOOP-15296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-15296: --- Affects Version/s: 3.0.0 Target Version/s: 3.1.0, 3.2.0, 3.0.2 Fix Version/s: 3.0.2 3.2.0 3.1.0 Committed to trunk, branch-3.1 and branch-3.0. Thanks [~tasanuma0829] for the contribution! > Fix a wrong link for RBF in the top page > > > Key: HADOOP-15296 > URL: https://issues.apache.org/jira/browse/HADOOP-15296 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0 >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Minor > Fix For: 3.1.0, 3.2.0, 3.0.2 > > Attachments: HADOOP-15296.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15296) Fix a wrong link for RBF in the top page
[ https://issues.apache.org/jira/browse/HADOOP-15296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-15296: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Fix a wrong link for RBF in the top page > > > Key: HADOOP-15296 > URL: https://issues.apache.org/jira/browse/HADOOP-15296 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0 >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Minor > Fix For: 3.1.0, 3.2.0, 3.0.2 > > Attachments: HADOOP-15296.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15296) Fix a wrong link for RBF in the top page
[ https://issues.apache.org/jira/browse/HADOOP-15296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390873#comment-16390873 ] Yiqun Lin commented on HADOOP-15296: LGTM, +1, committing... > Fix a wrong link for RBF in the top page > > > Key: HADOOP-15296 > URL: https://issues.apache.org/jira/browse/HADOOP-15296 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Minor > Attachments: HADOOP-15296.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14839) DistCp log output should contain copied and deleted files and directories
[
https://issues.apache.org/jira/browse/HADOOP-14839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-14839:
---
Attachment: HADOOP-14839-branch-2.002.patch
> DistCp log output should contain copied and deleted files and directories
> -
>
> Key: HADOOP-14839
> URL: https://issues.apache.org/jira/browse/HADOOP-14839
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools/distcp
>Affects Versions: 2.7.1
>Reporter: Konstantin Shaposhnikov
>Assignee: Yiqun Lin
> Fix For: 3.0.0-beta1
>
> Attachments: HADOOP-14839.006.patch, HADOOP-14839-branch-2.001.patch,
> HADOOP-14839-branch-2.002.patch, HDFS-10234.001.patch, HDFS-10234.002.patch,
> HDFS-10234.003.patch, HDFS-10234.004.patch, HDFS-10234.005.patch
>
>
> DistCp log output (specified via {{-log}} command line option) currently
> contains only skipped and failed (when failures are ignored via {{-i}}) files.
> It will be more useful if it also contains copied and deleted files and
> created directories.
> This should be fixed in
> https://github.com/apache/hadoop/blob/branch-2.7.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14839) DistCp log output should contain copied and deleted files and directories
[
https://issues.apache.org/jira/browse/HADOOP-14839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155276#comment-16155276
]
Yiqun Lin commented on HADOOP-14839:
The failure test {{TestOptionsParser}} is related and other two can be passed
in my local.
Attach the updated patch of branch-2.
> DistCp log output should contain copied and deleted files and directories
> -
>
> Key: HADOOP-14839
> URL: https://issues.apache.org/jira/browse/HADOOP-14839
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools/distcp
>Affects Versions: 2.7.1
>Reporter: Konstantin Shaposhnikov
>Assignee: Yiqun Lin
> Fix For: 3.0.0-beta1
>
> Attachments: HADOOP-14839.006.patch, HADOOP-14839-branch-2.001.patch,
> HDFS-10234.001.patch, HDFS-10234.002.patch, HDFS-10234.003.patch,
> HDFS-10234.004.patch, HDFS-10234.005.patch
>
>
> DistCp log output (specified via {{-log}} command line option) currently
> contains only skipped and failed (when failures are ignored via {{-i}}) files.
> It will be more useful if it also contains copied and deleted files and
> created directories.
> This should be fixed in
> https://github.com/apache/hadoop/blob/branch-2.7.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14839) DistCp log output should contain copied and deleted files and directories
[
https://issues.apache.org/jira/browse/HADOOP-14839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-14839:
---
Attachment: HADOOP-14839-branch-2.001.patch
> DistCp log output should contain copied and deleted files and directories
> -
>
> Key: HADOOP-14839
> URL: https://issues.apache.org/jira/browse/HADOOP-14839
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools/distcp
>Affects Versions: 2.7.1
>Reporter: Konstantin Shaposhnikov
>Assignee: Yiqun Lin
> Fix For: 3.0.0-beta1
>
> Attachments: HADOOP-14839.006.patch, HADOOP-14839-branch-2.001.patch,
> HDFS-10234.001.patch, HDFS-10234.002.patch, HDFS-10234.003.patch,
> HDFS-10234.004.patch, HDFS-10234.005.patch
>
>
> DistCp log output (specified via {{-log}} command line option) currently
> contains only skipped and failed (when failures are ignored via {{-i}}) files.
> It will be more useful if it also contains copied and deleted files and
> created directories.
> This should be fixed in
> https://github.com/apache/hadoop/blob/branch-2.7.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-14839) DistCp log output should contain copied and deleted files and directories
[
https://issues.apache.org/jira/browse/HADOOP-14839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin reopened HADOOP-14839:
> DistCp log output should contain copied and deleted files and directories
> -
>
> Key: HADOOP-14839
> URL: https://issues.apache.org/jira/browse/HADOOP-14839
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools/distcp
>Affects Versions: 2.7.1
>Reporter: Konstantin Shaposhnikov
>Assignee: Yiqun Lin
> Fix For: 3.0.0-beta1
>
> Attachments: HADOOP-14839.006.patch, HADOOP-14839-branch-2.001.patch,
> HDFS-10234.001.patch, HDFS-10234.002.patch, HDFS-10234.003.patch,
> HDFS-10234.004.patch, HDFS-10234.005.patch
>
>
> DistCp log output (specified via {{-log}} command line option) currently
> contains only skipped and failed (when failures are ignored via {{-i}}) files.
> It will be more useful if it also contains copied and deleted files and
> created directories.
> This should be fixed in
> https://github.com/apache/hadoop/blob/branch-2.7.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14839) DistCp log output should contain copied and deleted files and directories
[
https://issues.apache.org/jira/browse/HADOOP-14839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-14839:
---
Status: Patch Available (was: Reopened)
> DistCp log output should contain copied and deleted files and directories
> -
>
> Key: HADOOP-14839
> URL: https://issues.apache.org/jira/browse/HADOOP-14839
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools/distcp
>Affects Versions: 2.7.1
>Reporter: Konstantin Shaposhnikov
>Assignee: Yiqun Lin
> Fix For: 3.0.0-beta1
>
> Attachments: HADOOP-14839.006.patch, HADOOP-14839-branch-2.001.patch,
> HDFS-10234.001.patch, HDFS-10234.002.patch, HDFS-10234.003.patch,
> HDFS-10234.004.patch, HDFS-10234.005.patch
>
>
> DistCp log output (specified via {{-log}} command line option) currently
> contains only skipped and failed (when failures are ignored via {{-i}}) files.
> It will be more useful if it also contains copied and deleted files and
> created directories.
> This should be fixed in
> https://github.com/apache/hadoop/blob/branch-2.7.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14839) DistCp log output should contain copied and deleted files and directories
[
https://issues.apache.org/jira/browse/HADOOP-14839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155065#comment-16155065
]
Yiqun Lin commented on HADOOP-14839:
Thanks Xiaoyu, attach the patch for branch-2.
> DistCp log output should contain copied and deleted files and directories
> -
>
> Key: HADOOP-14839
> URL: https://issues.apache.org/jira/browse/HADOOP-14839
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools/distcp
>Affects Versions: 2.7.1
>Reporter: Konstantin Shaposhnikov
>Assignee: Yiqun Lin
> Fix For: 3.0.0-beta1
>
> Attachments: HADOOP-14839.006.patch, HDFS-10234.001.patch,
> HDFS-10234.002.patch, HDFS-10234.003.patch, HDFS-10234.004.patch,
> HDFS-10234.005.patch
>
>
> DistCp log output (specified via {{-log}} command line option) currently
> contains only skipped and failed (when failures are ignored via {{-i}}) files.
> It will be more useful if it also contains copied and deleted files and
> created directories.
> This should be fixed in
> https://github.com/apache/hadoop/blob/branch-2.7.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14839) DistCp log output should contain copied and deleted files and directories
[
https://issues.apache.org/jira/browse/HADOOP-14839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-14839:
---
Attachment: HADOOP-14839.006.patch
> DistCp log output should contain copied and deleted files and directories
> -
>
> Key: HADOOP-14839
> URL: https://issues.apache.org/jira/browse/HADOOP-14839
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools/distcp
>Affects Versions: 2.7.1
>Reporter: Konstantin Shaposhnikov
>Assignee: Yiqun Lin
> Attachments: HADOOP-14839.006.patch, HDFS-10234.001.patch,
> HDFS-10234.002.patch, HDFS-10234.003.patch, HDFS-10234.004.patch,
> HDFS-10234.005.patch
>
>
> DistCp log output (specified via {{-log}} command line option) currently
> contains only skipped and failed (when failures are ignored via {{-i}}) files.
> It will be more useful if it also contains copied and deleted files and
> created directories.
> This should be fixed in
> https://github.com/apache/hadoop/blob/branch-2.7.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14839) DistCp log output should contain copied and deleted files and directories
[
https://issues.apache.org/jira/browse/HADOOP-14839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154722#comment-16154722
]
Yiqun Lin commented on HADOOP-14839:
Thanks [~xyao] for catching this, Update the updated patch.
> DistCp log output should contain copied and deleted files and directories
> -
>
> Key: HADOOP-14839
> URL: https://issues.apache.org/jira/browse/HADOOP-14839
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools/distcp
>Affects Versions: 2.7.1
>Reporter: Konstantin Shaposhnikov
>Assignee: Yiqun Lin
> Attachments: HDFS-10234.001.patch, HDFS-10234.002.patch,
> HDFS-10234.003.patch, HDFS-10234.004.patch, HDFS-10234.005.patch
>
>
> DistCp log output (specified via {{-log}} command line option) currently
> contains only skipped and failed (when failures are ignored via {{-i}}) files.
> It will be more useful if it also contains copied and deleted files and
> created directories.
> This should be fixed in
> https://github.com/apache/hadoop/blob/branch-2.7.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14568) GenericTestUtils#waitFor missing parameter verification
[ https://issues.apache.org/jira/browse/HADOOP-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14568: --- Attachment: HADOOP-14568.003.patch Thanks [~shahrs87] for the review. Will back-port this to 2.8 after patch is confirmed and getting binding +1 from others. Attach the updated patch to fix checkstyle warning. > GenericTestUtils#waitFor missing parameter verification > --- > > Key: HADOOP-14568 > URL: https://issues.apache.org/jira/browse/HADOOP-14568 > Project: Hadoop Common > Issue Type: Bug > Components: test >Affects Versions: 3.0.0-alpha3 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Attachments: HADOOP-14568.001.patch, HADOOP-14568.002.patch, > HADOOP-14568.003.patch > > > GenericTestUtils#waitFor missed the parameters verification. This will lead > the method ran error or meanless. This point was suggested from this > [comment|https://issues.apache.org/jira/browse/HDFS-11711?focusedCommentId=16039292=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16039292]. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14568) GenericTestUtils#waitFor missing parameter verification
[ https://issues.apache.org/jira/browse/HADOOP-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14568: --- Attachment: HADOOP-14568.002.patch Thanks [~ste...@apache.org] for the review. Attach the updated patch to address your comment. > GenericTestUtils#waitFor missing parameter verification > --- > > Key: HADOOP-14568 > URL: https://issues.apache.org/jira/browse/HADOOP-14568 > Project: Hadoop Common > Issue Type: Bug > Components: test >Affects Versions: 3.0.0-alpha3 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Attachments: HADOOP-14568.001.patch, HADOOP-14568.002.patch > > > GenericTestUtils#waitFor missed the parameters verification. This will lead > the method ran error or meanless. This point was suggested from this > [comment|https://issues.apache.org/jira/browse/HDFS-11711?focusedCommentId=16039292=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16039292]. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14568) GenericTestUtils#waitFor missing parameter verification
[
https://issues.apache.org/jira/browse/HADOOP-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HADOOP-14568:
---
Attachment: HADOOP-14568.001.patch
Reupload patch to use {{Preconditions#checkArgument}} to check parameters.
> GenericTestUtils#waitFor missing parameter verification
> ---
>
> Key: HADOOP-14568
> URL: https://issues.apache.org/jira/browse/HADOOP-14568
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0-alpha3
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: HADOOP-14568.001.patch
>
>
> GenericTestUtils#waitFor missed the parameters verification. This will lead
> the method ran error or meanless. This point was suggested from this
> [comment|https://issues.apache.org/jira/browse/HDFS-11711?focusedCommentId=16039292=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16039292].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14568) GenericTestUtils#waitFor missing parameter verification
[ https://issues.apache.org/jira/browse/HADOOP-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14568: --- Attachment: (was: HADOOP-14568.001.patch) > GenericTestUtils#waitFor missing parameter verification > --- > > Key: HADOOP-14568 > URL: https://issues.apache.org/jira/browse/HADOOP-14568 > Project: Hadoop Common > Issue Type: Bug > Components: test >Affects Versions: 3.0.0-alpha3 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > > GenericTestUtils#waitFor missed the parameters verification. This will lead > the method ran error or meanless. This point was suggested from this > [comment|https://issues.apache.org/jira/browse/HDFS-11711?focusedCommentId=16039292=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16039292]. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14568) GenericTestUtils#waitFor missing parameter verification
[ https://issues.apache.org/jira/browse/HADOOP-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14568: --- Status: Patch Available (was: Open) > GenericTestUtils#waitFor missing parameter verification > --- > > Key: HADOOP-14568 > URL: https://issues.apache.org/jira/browse/HADOOP-14568 > Project: Hadoop Common > Issue Type: Bug > Components: test >Affects Versions: 3.0.0-alpha3 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Attachments: HADOOP-14568.001.patch > > > GenericTestUtils#waitFor missed the parameters verification. This will lead > the method ran error or meanless. This point was suggested from this > [comment|https://issues.apache.org/jira/browse/HDFS-11711?focusedCommentId=16039292=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16039292]. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14568) GenericTestUtils#waitFor missing parameter verification
[ https://issues.apache.org/jira/browse/HADOOP-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14568: --- Attachment: HADOOP-14568.001.patch Attach the initial patch. Adding the null checking and time validation checking. > GenericTestUtils#waitFor missing parameter verification > --- > > Key: HADOOP-14568 > URL: https://issues.apache.org/jira/browse/HADOOP-14568 > Project: Hadoop Common > Issue Type: Bug > Components: test >Affects Versions: 3.0.0-alpha3 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Attachments: HADOOP-14568.001.patch > > > GenericTestUtils#waitFor missed the parameters verification. This will lead > the method ran error or meanless. This point was suggested from this > [comment|https://issues.apache.org/jira/browse/HDFS-11711?focusedCommentId=16039292=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16039292]. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14568) GenericTestUtils#waitFor missing parameter verification
Yiqun Lin created HADOOP-14568: -- Summary: GenericTestUtils#waitFor missing parameter verification Key: HADOOP-14568 URL: https://issues.apache.org/jira/browse/HADOOP-14568 Project: Hadoop Common Issue Type: Bug Components: test Affects Versions: 3.0.0-alpha3 Reporter: Yiqun Lin Assignee: Yiqun Lin GenericTestUtils#waitFor missed the parameters verification. This will lead the method ran error or meanless. This point was suggested from this [comment|https://issues.apache.org/jira/browse/HDFS-11711?focusedCommentId=16039292=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16039292]. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14337) Fix warnings from Spotbugs in hadoop-hdfs
[
https://issues.apache.org/jira/browse/HADOOP-14337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15980718#comment-15980718
]
Yiqun Lin edited comment on HADOOP-14337 at 4/24/17 4:31 AM:
-
{quote}
BTW, the last line says 3 unchanged, does that mean there is still 3 warnings
unfixed in hadoop-hdfs-project/hadoop-hdfs?
{quote}
>From my understanding of this, that should be this meaning. But I think this
>is not correct, :).
was (Author: linyiqun):
{quote}
BTW, the last line says 3 unchanged, does that mean there is still 3 warnings
unfixed in hadoop-hdfs-project/hadoop-hdfs?
{quote}
>From my understanding of this, that should be this meaning.
> Fix warnings from Spotbugs in hadoop-hdfs
> -
>
> Key: HADOOP-14337
> URL: https://issues.apache.org/jira/browse/HADOOP-14337
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha3
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: HADOOP-14337.001.patch, HADOOP-14337.002.patch,
> HADOOP-14337.003.patch
>
>
> There are totally 12 findbugs issues generated after switching from Findbugs
> to Spotbugs across the project in HADOOP-14316. This JIRA focus on cleaning
> up the part of warnings under scope of HDFS(mainly contained in hadoop-hdfs
> and hadoop-hdfs-client).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14337) Fix warnings from Spotbugs in hadoop-hdfs
[
https://issues.apache.org/jira/browse/HADOOP-14337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15980718#comment-15980718
]
Yiqun Lin commented on HADOOP-14337:
{quote}
BTW, the last line says 3 unchanged, does that mean there is still 3 warnings
unfixed in hadoop-hdfs-project/hadoop-hdfs?
{quote}
>From my understanding of this, that should be this meaning.
> Fix warnings from Spotbugs in hadoop-hdfs
> -
>
> Key: HADOOP-14337
> URL: https://issues.apache.org/jira/browse/HADOOP-14337
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha3
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: HADOOP-14337.001.patch, HADOOP-14337.002.patch,
> HADOOP-14337.003.patch
>
>
> There are totally 12 findbugs issues generated after switching from Findbugs
> to Spotbugs across the project in HADOOP-14316. This JIRA focus on cleaning
> up the part of warnings under scope of HDFS(mainly contained in hadoop-hdfs
> and hadoop-hdfs-client).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14337) Fix warnings from Spotbugs in hadoop-hdfs
[ https://issues.apache.org/jira/browse/HADOOP-14337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15980649#comment-15980649 ] Yiqun Lin edited comment on HADOOP-14337 at 4/24/17 2:32 AM: - Hi [~ajisakaa], would you mind have a review on this? BTW, I am a little confused that I have fixed all the findbugs issues under the HDFS but the Jenkins still told me there were 3 warnings remained, :(. Thanks. was (Author: linyiqun): Hi [~ajisakaa], would you mind have a review on this? BTW, I am a little confused that I have fixed all the findbugs issues under the HDFS but the Jenkins still tole me there were 3 warnings remained, :(. Thanks. > Fix warnings from Spotbugs in hadoop-hdfs > - > > Key: HADOOP-14337 > URL: https://issues.apache.org/jira/browse/HADOOP-14337 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.0.0-alpha3 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Attachments: HADOOP-14337.001.patch, HADOOP-14337.002.patch, > HADOOP-14337.003.patch > > > There are totally 12 findbugs issues generated after switching from Findbugs > to Spotbugs across the project in HADOOP-14316. This JIRA focus on cleaning > up the part of warnings under scope of HDFS(mainly contained in hadoop-hdfs > and hadoop-hdfs-client). -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14337) Fix warnings from Spotbugs in hadoop-hdfs
[ https://issues.apache.org/jira/browse/HADOOP-14337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15980649#comment-15980649 ] Yiqun Lin commented on HADOOP-14337: Hi [~ajisakaa], would you mind have a review on this? BTW, I am a little confused that I have fixed all the findbugs issues under the HDFS but the Jenkins still tole me there were 3 warnings remained, :(. Thanks. > Fix warnings from Spotbugs in hadoop-hdfs > - > > Key: HADOOP-14337 > URL: https://issues.apache.org/jira/browse/HADOOP-14337 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.0.0-alpha3 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Attachments: HADOOP-14337.001.patch, HADOOP-14337.002.patch, > HADOOP-14337.003.patch > > > There are totally 12 findbugs issues generated after switching from Findbugs > to Spotbugs across the project in HADOOP-14316. This JIRA focus on cleaning > up the part of warnings under scope of HDFS(mainly contained in hadoop-hdfs > and hadoop-hdfs-client). -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14337) Fix warnings from Spotbugs in hadoop-hdfs
[ https://issues.apache.org/jira/browse/HADOOP-14337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14337: --- Attachment: HADOOP-14337.003.patch Attach the new patch to fix the remaining failure tests. > Fix warnings from Spotbugs in hadoop-hdfs > - > > Key: HADOOP-14337 > URL: https://issues.apache.org/jira/browse/HADOOP-14337 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.0.0-alpha3 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Attachments: HADOOP-14337.001.patch, HADOOP-14337.002.patch, > HADOOP-14337.003.patch > > > There are totally 12 findbugs issues generated after switching from Findbugs > to Spotbugs across the project in HADOOP-14316. This JIRA focus on cleaning > up the part of warnings under scope of HDFS(mainly contained in hadoop-hdfs > and hadoop-hdfs-client). -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14337) Fix warnings from Spotbugs in hadoop-hdfs
[ https://issues.apache.org/jira/browse/HADOOP-14337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14337: --- Attachment: HADOOP-14337.002.patch Attach the new patch to fix remaining findbugs issues. > Fix warnings from Spotbugs in hadoop-hdfs > - > > Key: HADOOP-14337 > URL: https://issues.apache.org/jira/browse/HADOOP-14337 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.0.0-alpha3 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Attachments: HADOOP-14337.001.patch, HADOOP-14337.002.patch > > > There are totally 12 findbugs issues generated after switching from Findbugs > to Spotbugs across the project in HADOOP-14316. This JIRA focus on cleaning > up the part of warnings under scope of HDFS(mainly contained in hadoop-hdfs > and hadoop-hdfs-client). -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14337) Fix warnings from Spotbugs in hadoop-hdfs
[ https://issues.apache.org/jira/browse/HADOOP-14337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14337: --- Description: There are totally 12 findbugs issues generated after switching from Findbugs to Spotbugs across the project in HADOOP-14316. This JIRA focus on cleaning up the part of warnings under scope of HDFS(mainly contained in hadoop-hdfs and hadoop-hdfs-client). (was: There are totally 10 findbugs issues generated after switching from Findbugs to Spotbugs across the project in HADOOP-14316. This JIRA focus on cleaning up the part of warnings under scope of HDFS.) > Fix warnings from Spotbugs in hadoop-hdfs > - > > Key: HADOOP-14337 > URL: https://issues.apache.org/jira/browse/HADOOP-14337 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.0.0-alpha3 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Attachments: HADOOP-14337.001.patch > > > There are totally 12 findbugs issues generated after switching from Findbugs > to Spotbugs across the project in HADOOP-14316. This JIRA focus on cleaning > up the part of warnings under scope of HDFS(mainly contained in hadoop-hdfs > and hadoop-hdfs-client). -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14337) Fix warnings from Spotbugs in hadoop-hdfs
[ https://issues.apache.org/jira/browse/HADOOP-14337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14337: --- Status: Patch Available (was: Open) > Fix warnings from Spotbugs in hadoop-hdfs > - > > Key: HADOOP-14337 > URL: https://issues.apache.org/jira/browse/HADOOP-14337 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.0.0-alpha3 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Attachments: HADOOP-14337.001.patch > > > There are totally 10 findbugs issues generated after switching from Findbugs > to Spotbugs across the project in HADOOP-14316. This JIRA focus on cleaning > up the part of warnings under scope of HDFS. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14337) Fix warnings from Spotbugs in hadoop-hdfs
[ https://issues.apache.org/jira/browse/HADOOP-14337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14337: --- Attachment: HADOOP-14337.001.patch Attach the initial patch. Kindly review. > Fix warnings from Spotbugs in hadoop-hdfs > - > > Key: HADOOP-14337 > URL: https://issues.apache.org/jira/browse/HADOOP-14337 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.0.0-alpha3 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Attachments: HADOOP-14337.001.patch > > > There are totally 10 findbugs issues generated after switching from Findbugs > to Spotbugs across the project in HADOOP-14316. This JIRA focus on cleaning > up the part of warnings under scope of HDFS. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14133) Fix javac warnings in Hadoop Common that caused by deprecated FileSystem APIs
[
https://issues.apache.org/jira/browse/HADOOP-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888212#comment-15888212
]
Yiqun Lin commented on HADOOP-14133:
The failed test {{TestRawLocalFileSystemContract}} is related. Will delay to
attach new patch in case someone has comment on this.
> Fix javac warnings in Hadoop Common that caused by deprecated FileSystem APIs
> -
>
> Key: HADOOP-14133
> URL: https://issues.apache.org/jira/browse/HADOOP-14133
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
>Priority: Minor
> Attachments: HADOOP-14133.001.patch
>
>
> This JIRA is part of HADOOP-14106. This JIRA foucs on fixing javadoc warnings
> under HADOOP-COMMON module.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14133) Fix javac warnings in Hadoop Common that caused by deprecated FileSystem APIs
[ https://issues.apache.org/jira/browse/HADOOP-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14133: --- Summary: Fix javac warnings in Hadoop Common that caused by deprecated FileSystem APIs (was: Fix javadoc warnings in Hadoop Common that caused by deprecated FileSystem APIs) > Fix javac warnings in Hadoop Common that caused by deprecated FileSystem APIs > - > > Key: HADOOP-14133 > URL: https://issues.apache.org/jira/browse/HADOOP-14133 > Project: Hadoop Common > Issue Type: Bug >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Minor > Attachments: HADOOP-14133.001.patch > > > This JIRA is part of HADOOP-14106. This JIRA foucs on fixing javadoc warnings > under HADOOP-COMMON module. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14133) Fix javadoc warnings in Hadoop Common that caused by deprecated FileSystem APIs
[ https://issues.apache.org/jira/browse/HADOOP-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14133: --- Summary: Fix javadoc warnings in Hadoop Common that caused by deprecated FileSystem APIs (was: Fix javadoc warnings in HADOOP-COMMON that caused by deprecated FileSystem APIs) > Fix javadoc warnings in Hadoop Common that caused by deprecated FileSystem > APIs > --- > > Key: HADOOP-14133 > URL: https://issues.apache.org/jira/browse/HADOOP-14133 > Project: Hadoop Common > Issue Type: Bug >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Minor > Attachments: HADOOP-14133.001.patch > > > This JIRA is part of HADOOP-14106. This JIRA foucs on fixing javadoc warnings > under HADOOP-COMMON module. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14133) Fix javadoc warnings in HADOOP-COMMON that caused by deprecated FileSystem APIs
[ https://issues.apache.org/jira/browse/HADOOP-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14133: --- Status: Patch Available (was: Open) Attach the initial patch. The patch seems very big. Maybe there is missing something to update. > Fix javadoc warnings in HADOOP-COMMON that caused by deprecated FileSystem > APIs > --- > > Key: HADOOP-14133 > URL: https://issues.apache.org/jira/browse/HADOOP-14133 > Project: Hadoop Common > Issue Type: Bug >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Minor > Attachments: HADOOP-14133.001.patch > > > This JIRA is part of HADOOP-14106. This JIRA foucs on fixing javadoc warnings > under HADOOP-COMMON module. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14133) Fix javadoc warnings in HADOOP-COMMON that caused by deprecated FileSystem APIs
[ https://issues.apache.org/jira/browse/HADOOP-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14133: --- Attachment: HADOOP-14133.001.patch > Fix javadoc warnings in HADOOP-COMMON that caused by deprecated FileSystem > APIs > --- > > Key: HADOOP-14133 > URL: https://issues.apache.org/jira/browse/HADOOP-14133 > Project: Hadoop Common > Issue Type: Bug >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Minor > Attachments: HADOOP-14133.001.patch > > > This JIRA is part of HADOOP-14106. This JIRA foucs on fixing javadoc warnings > under HADOOP-COMMON module. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14133) Fix javadoc warnings in HADOOP-COMMON that caused by deprecated FileSystem APIs
[ https://issues.apache.org/jira/browse/HADOOP-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14133: --- Description: This JIRA is part of HADOOP-14106. This JIRA foucs on fixing javadoc warnings under HADOOP-COMMON module. (was: This JIRA is part of HADOOP-14106.) > Fix javadoc warnings in HADOOP-COMMON that caused by deprecated FileSystem > APIs > --- > > Key: HADOOP-14133 > URL: https://issues.apache.org/jira/browse/HADOOP-14133 > Project: Hadoop Common > Issue Type: Bug >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Minor > > This JIRA is part of HADOOP-14106. This JIRA foucs on fixing javadoc warnings > under HADOOP-COMMON module. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Moved] (HADOOP-14133) Fix javadoc warnings in HADOOP-COMMON that caused by deprecated FileSystem APIs
[ https://issues.apache.org/jira/browse/HADOOP-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin moved YARN-6250 to HADOOP-14133: -- Target Version/s: 3.0.0-alpha3 (was: 3.0.0-alpha3) Key: HADOOP-14133 (was: YARN-6250) Project: Hadoop Common (was: Hadoop YARN) > Fix javadoc warnings in HADOOP-COMMON that caused by deprecated FileSystem > APIs > --- > > Key: HADOOP-14133 > URL: https://issues.apache.org/jira/browse/HADOOP-14133 > Project: Hadoop Common > Issue Type: Bug >Reporter: Yiqun Lin >Assignee: Yiqun Lin >Priority: Minor > > This JIRA is part of HADOOP-14106. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14119) Remove unused imports from GzipCodec.java
[ https://issues.apache.org/jira/browse/HADOOP-14119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14119: --- Target Version/s: 2.9.0, 3.0.0-alpha3 (was: 2.9.0, 3.0.0-alpha2) > Remove unused imports from GzipCodec.java > - > > Key: HADOOP-14119 > URL: https://issues.apache.org/jira/browse/HADOOP-14119 > Project: Hadoop Common > Issue Type: Bug >Reporter: Akira Ajisaka >Assignee: Yiqun Lin >Priority: Minor > Labels: newbie > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: HADOOP-14119.001.patch > > > There are some unused imports after HADOOP-14097. Let's fix this. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14119) Remove unused imports from GzipCodec.java
[ https://issues.apache.org/jira/browse/HADOOP-14119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14119: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3 2.9.0 Target Version/s: 3.0.0-alpha2, 2.9.0 (was: 2.9.0, 3.0.0-alpha2) Status: Resolved (was: Patch Available) Committed this to trunk and branch-2. Thanks [~ajisakaa] for the review! > Remove unused imports from GzipCodec.java > - > > Key: HADOOP-14119 > URL: https://issues.apache.org/jira/browse/HADOOP-14119 > Project: Hadoop Common > Issue Type: Bug >Reporter: Akira Ajisaka >Assignee: Yiqun Lin >Priority: Minor > Labels: newbie > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: HADOOP-14119.001.patch > > > There are some unused imports after HADOOP-14097. Let's fix this. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14119) Remove unused imports from GzipCodec.java
[ https://issues.apache.org/jira/browse/HADOOP-14119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14119: --- Target Version/s: 3.0.0-alpha2, 2.9.0 (was: 2.9.0, 3.0.0-alpha2) Status: Patch Available (was: Open) Attach a patch to make a clean up. It looks strange that Jenkins not shows the checkstyle issues. > Remove unused imports from GzipCodec.java > - > > Key: HADOOP-14119 > URL: https://issues.apache.org/jira/browse/HADOOP-14119 > Project: Hadoop Common > Issue Type: Bug >Reporter: Akira Ajisaka >Assignee: Yiqun Lin >Priority: Minor > Labels: newbie > Attachments: HADOOP-14119.001.patch > > > There are some unused imports after HADOOP-14097. Let's fix this. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14119) Remove unused imports from GzipCodec.java
[ https://issues.apache.org/jira/browse/HADOOP-14119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-14119: --- Attachment: HADOOP-14119.001.patch > Remove unused imports from GzipCodec.java > - > > Key: HADOOP-14119 > URL: https://issues.apache.org/jira/browse/HADOOP-14119 > Project: Hadoop Common > Issue Type: Bug >Reporter: Akira Ajisaka >Assignee: Yiqun Lin >Priority: Minor > Labels: newbie > Attachments: HADOOP-14119.001.patch > > > There are some unused imports after HADOOP-14097. Let's fix this. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
