Re: [Hadoop-3.3 Release update]- branch-3.3 has created

[Brahma Reddy Battula]

Hi All,

we are down to two blockers issues now (YARN-10194 and YARN-9848) which are
in patch available state.Hopefully we can out the RC soon.

thanks to @Prabhu Joseph 
,@masakate,@akira and @Wei-Chiu
Chuang   and others for helping resloving the
blockers.



On Tue, Apr 14, 2020 at 10:49 PM Brahma Reddy Battula 
wrote:

>
> @Prabhu Joseph 
> >>> Have committed the YARN blocker YARN-10219 to trunk and cherry-picked
> to branch-3.3. Right now, there are two blocker Jiras - YARN-10233 and
> HADOOP-16982
> which i will help to review and commit. Thanks.
>
> Looks you committed YARN-10219. Noted YARN-10233 and HADOOP-16982 as a
> blockers. (without YARN-10233 we have given so many releases,it's not newly
> introduced.).. Thanks
>
> @Vinod Kumar Vavilapalli  ,@adam Antal,
>
> I noted YARN-9848 as a blocker as you mentioned above.
>
> @All,
>
> Currently following four blockers are pending for 3.3.0 RC.
>
> HADOOP-16963,YARN-10233,HADOOP-16982 and YARN-9848.
>
>
>
> On Tue, Apr 14, 2020 at 8:11 PM Vinod Kumar Vavilapalli <
> vino...@apache.org> wrote:
>
>> Looks like a really bad bug to me.
>>
>> +1 for revert and +1 for making that a 3.3.0 blocker. I think should also
>> revert it in a 3.2 maintenance release too.
>>
>> Thanks
>> +Vinod
>>
>> > On Apr 14, 2020, at 5:03 PM, Adam Antal 
>> wrote:
>> >
>> > Hi everyone,
>> >
>> > Sorry for coming a bit late with this, but there's also one jira that
>> can
>> > have potential impact on clusters and we should talk about it.
>> >
>> > Steven Rand found this problem earlier and commented to
>> > https://issues.apache.org/jira/browse/YARN-4946.
>> > The bug has impact on the RM state store: the RM does not delete apps -
>> see
>> > more details in his comment here:
>> >
>> https://issues.apache.org/jira/browse/YARN-4946?focusedCommentId=16898599=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16898599
>> > .
>> > (FYI He also created https://issues.apache.org/jira/browse/YARN-9848
>> with
>> > the revert task).
>> >
>> > It might not be an actual blocker, but since there wasn't any consensus
>> > about a follow up action, I thought we should decide how to proceed
>> before
>> > release 3.3.0.
>> >
>> > Regards,
>> > Adam
>> >
>> > On Tue, Apr 14, 2020 at 9:35 AM Prabhu Joseph <
>> prabhujose.ga...@gmail.com>
>> > wrote:
>> >
>> >> Thanks Brahma for the update.
>> >>
>> >> Have committed the YARN blocker YARN-10219 to trunk and cherry-picked
>> to
>> >> branch-3.3. Right now, there are two blocker Jiras - YARN-10233 and
>> >> HADOOP-16982
>> >> which i will help to review and commit. Thanks.
>> >>
>> >> [image: Screen Shot 2020-04-14 at 1.01.51 PM.png]
>> >>
>> >> project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in (Blocker,
>> >> Critical) AND resolution = Unresolved AND "Target Version/s" = 3.3.0
>> ORDER
>> >> BY priority DESC
>> >>
>> >>
>> >> On Sun, Apr 12, 2020 at 12:19 AM Brahma Reddy Battula <
>> bra...@apache.org>
>> >> wrote:
>> >>
>> >>> *Pending for 3.3.0 Release:*
>> >>>
>> >>> One Blocker(HADOOP-16963) confirmation and following jira's are open
>> as
>> >>> these needs to merged to other branches(I am tracking the same,
>> Ideally
>> >>> this can be closed and can raise seperate jira's to track).
>> >>>
>> >>>
>> >>> 1–4 of 4Refresh results
>> >>> <
>> >>>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(%22Hadoop%20HDFS%22)%20AND%20resolution%20%3D%20Unresolved%20AND%20(cf%5B12310320%5D%20%3D%203.3.0%20OR%20fixVersion%20%3D%203.3.0)%20ORDER%20BY%20priority%20DESC#
>> 
>> >>> Columns
>> >>> Patch InfoKeyTSummaryAssigneeReporterP
>> StatusResolutionUpdatedDueCreated
>> >>>  HDFS-14353
>> >>>  [image: Sub-task]
>> >>> 
>> >>>
>> >>> HDFS-8031  Erasure
>> >>> Coding:
>> >>> metrics xmitsInProgress become to negative.
>> >>> 
>> >>> maobaolong
>> >>> <
>> https://issues.apache.org/jira/secure/ViewProfile.jspa?name=maobaolong>
>> >>> maobaolong
>> >>> <
>> https://issues.apache.org/jira/secure/ViewProfile.jspa?name=maobaolong>
>> >>> [image:
>> >>> Major] REOPENED *Unresolved* 11/Apr/20   11/Mar/19 Actions
>> >>> <
>> >>>
>> https://issues.apache.org/jira/rest/api/1.0/issues/13220750/ActionsAndOperations?atl_token=A5KQ-2QAV-T4JA-FDED_194e108bac53dceebb1b88ae92ef65a9eba913b0_lin
>> 
>> >>>  HDFS-14788
>> >>>  [image:
>> Improvement]
>> >>> 
>> >>>
>> >>> Use dynamic regex filter to ignore copy of source files in Distcp
>> >>> 
>> >>> Mukund Thakur
>> >>> <
>> >>>
>> https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mukund-thakur
>> 
>> >>> Mukund
>> 

Re: Hadoop 3.1.2 missing from download site?

[Akira Ajisaka]

Filed https://issues.apache.org/jira/browse/HADOOP-16993

On Fri, Apr 17, 2020 at 9:15 AM Akira Ajisaka  wrote:

> Hi Arpit,
>
> Thank you for your report.
> Old releases are moved to the archive site (
> https://archive.apache.org/dist/hadoop/common/) on purpose. I'll remove
> the Hadoop 3.1.2 download link.
>
> Thanks,
> Akira
>
> On Fri, Apr 17, 2020 at 6:54 AM Arpit Agarwal
>  wrote:
>
>> The Hadoop 3.1.2 download links seem to be broken on
>> https://hadoop.apache.org/releases.html
>>
>> Was it removed on purpose?
>>
>>
>>
>> -
>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>>
>>

[jira] [Created] (HADOOP-16993) Hadoop 3.2.1 download link is broken

[Akira Ajisaka (Jira)]

Akira Ajisaka created HADOOP-16993:
--

 Summary: Hadoop 3.2.1 download link is broken
 Key: HADOOP-16993
 URL: https://issues.apache.org/jira/browse/HADOOP-16993
 Project: Hadoop Common
  Issue Type: Bug
  Components: website
Reporter: Arpit Agarwal
Assignee: Akira Ajisaka


Remove broken Hadoop 3.1.2 download links from the website.
https://hadoop.apache.org/releases.html



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: Hadoop 3.1.2 missing from download site?

[Akira Ajisaka]

Hi Arpit,

Thank you for your report.
Old releases are moved to the archive site (
https://archive.apache.org/dist/hadoop/common/) on purpose. I'll remove the
Hadoop 3.1.2 download link.

Thanks,
Akira

On Fri, Apr 17, 2020 at 6:54 AM Arpit Agarwal 
wrote:

> The Hadoop 3.1.2 download links seem to be broken on
> https://hadoop.apache.org/releases.html
>
> Was it removed on purpose?
>
>
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>

[jira] [Created] (HADOOP-16992) Update download links

[Arpit Agarwal (Jira)]

Arpit Agarwal created HADOOP-16992:
--

 Summary: Update download links
 Key: HADOOP-16992
 URL: https://issues.apache.org/jira/browse/HADOOP-16992
 Project: Hadoop Common
  Issue Type: Improvement
  Components: website
Reporter: Arpit Agarwal


The download lists for signatures/checksums/KEYS should be updated from 
dist.apache.org to https://downloads.apache.org/hadoop/ozone/.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Hadoop 3.1.2 missing from download site?

[Arpit Agarwal]

The Hadoop 3.1.2 download links seem to be broken on 
https://hadoop.apache.org/releases.html

Was it removed on purpose?



-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [DISCUSS] Making 2.10 the last minor 2.x release

[Jonathan Hung]

Source code has been deleted from branch-2. Thanks Akira for taking this up!

Jonathan Hung


On Thu, Apr 16, 2020 at 11:40 AM Jonathan Hung  wrote:

> Makes sense. I've cherry-picked the commits in branch-2 that were missed
> in branch-2.10.
>
> Jonathan Hung
>
>
> On Wed, Apr 15, 2020 at 2:25 AM Akira Ajisaka  wrote:
>
>> Hi folks,
>>
>> I am still seeing some changes are being committed to branch-2.
>> I'd like to delete the source code from branch-2 to avoid mistakes.
>> https://issues.apache.org/jira/browse/HADOOP-16988
>>
>> -Akira
>>
>> On Wed, Jan 1, 2020 at 2:38 AM Ayush Saxena  wrote:
>>
>>> Hi Jim,
>>> Thanx for catching, I have configured the build to run on branch-2.10.
>>>
>>> -Ayush
>>>
>>> On Tue, 31 Dec 2019 at 22:50, Jim Brennan <
>>> james.bren...@verizonmedia.com> wrote:
>>>
 It looks like QBT tests are still being run on branch-2 (
 https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/),
 and they are not very helpful at this point.
 Can we change the QBT tests to run against branch-2.10 instead?

 Jim

 On Mon, Dec 23, 2019 at 7:44 PM Akira Ajisaka 
 wrote:

> Thank you, Ayush.
>
> I understand we should keep branch-2 as is, as well as master.
>
> -Akira
>
>
> On Mon, Dec 23, 2019 at 9:14 PM Ayush Saxena 
> wrote:
>
> > Hi Akira
> > Seems there was an INFRA ticket for that. INFRA-19581,
> > But the INFRA people closed as wont do and yes, the branch is
> protected,
> > we can’t delete it directly.
> >
> > Ref: https://issues.apache.org/jira/browse/INFRA-19581
> >
> > -Ayush
> >
> > On 23-Dec-2019, at 5:03 PM, Akira Ajisaka 
> wrote:
> >
> > Thank you for your work, Jonathan.
> >
> > I found branch-2 has been unintentionally pushed again. Would you
> remove
> > it?
> > I think the branch should be protected if possible.
> >
> > -Akira
> >
> > On Tue, Dec 10, 2019 at 5:17 AM Jonathan Hung 
> > wrote:
> >
> > It's done. The new commit chain is: trunk -> branch-3.2 ->
> branch-3.1 ->
> >
> > branch-2.10 -> branch-2.9 -> branch-2.8 (branch-2 no longer exists,
> please
> >
> > don't try to commit to it)
> >
> >
> > Completed procedure:
> >
> >
> >   - Verified everything in old branch-2.10 was in old branch-2
> >
> >   - Delete old branch-2.10
> >
> >   - Rename branch-2 to (new) branch-2.10
> >
> >   - Set version in new branch-2.10 to 2.10.1-SNAPSHOT
> >
> >   - Renamed fix versions from 2.11.0 to 2.10.1
> >
> >   - Removed 2.11.0 as a version in HADOOP/YARN/HDFS/MAPREDUCE
> >
> >
> >
> > Jonathan Hung
> >
> >
> >
> > On Wed, Dec 4, 2019 at 10:55 AM Jonathan Hung 
> >
> > wrote:
> >
> >
> > FYI, starting the rename process, beginning with INFRA-19521.
> >
> >
> > Jonathan Hung
> >
> >
> >
> > On Wed, Nov 27, 2019 at 12:15 PM Konstantin Shvachko <
> >
> > shv.had...@gmail.com>
> >
> > wrote:
> >
> >
> > Hey guys,
> >
> >
> > I think we diverged a bit from the initial topic of this discussion,
> >
> > which is removing branch-2.10, and changing the version of branch-2
> from
> >
> > 2.11.0-SNAPSHOT to 2.10.1-SNAPSHOT.
> >
> > Sounds like the subject line for this thread "Making 2.10 the last
> minor
> >
> > 2.x release" confused people.
> >
> > It is in fact a wider matter that can be discussed when somebody
> >
> > actually
> >
> > proposes to release 2.11, which I understand nobody does at the
> moment.
> >
> >
> > So if anybody objects removing branch-2.10 please make an argument.
> >
> > Otherwise we should go ahead and just do it next week.
> >
> > I see people still struggling to keep branch-2 and branch-2.10 in
> sync.
> >
> >
> > Thanks,
> >
> > --Konstantin
> >
> >
> > On Thu, Nov 21, 2019 at 3:49 PM Jonathan Hung 
> >
> > wrote:
> >
> >
> > Thanks for the detailed thoughts, everyone.
> >
> >
> > Eric (Badger), my understanding is the same as yours re. minor vs
> patch
> >
> > releases. As for putting features into minor/patch releases, if we
> >
> > keep the
> >
> > convention of putting new features only into minor releases, my
> >
> > assumption
> >
> > is still that it's unlikely people will want to get them into
> branch-2
> >
> > (based on the 2.10.0 release process). For the java 11 issue, we
> >
> > haven't
> >
> > even really removed support for java 7 in branch-2 (much less java
> 8),
> >
> > so I
> >
> > feel moving to java 11 would go along with a move to branch 3. And as
> >
> > you
> >

Re: [DISCUSS] Making 2.10 the last minor 2.x release

[Jonathan Hung]

Makes sense. I've cherry-picked the commits in branch-2 that were missed in
branch-2.10.

Jonathan Hung


On Wed, Apr 15, 2020 at 2:25 AM Akira Ajisaka  wrote:

> Hi folks,
>
> I am still seeing some changes are being committed to branch-2.
> I'd like to delete the source code from branch-2 to avoid mistakes.
> https://issues.apache.org/jira/browse/HADOOP-16988
>
> -Akira
>
> On Wed, Jan 1, 2020 at 2:38 AM Ayush Saxena  wrote:
>
>> Hi Jim,
>> Thanx for catching, I have configured the build to run on branch-2.10.
>>
>> -Ayush
>>
>> On Tue, 31 Dec 2019 at 22:50, Jim Brennan 
>> wrote:
>>
>>> It looks like QBT tests are still being run on branch-2 (
>>> https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/),
>>> and they are not very helpful at this point.
>>> Can we change the QBT tests to run against branch-2.10 instead?
>>>
>>> Jim
>>>
>>> On Mon, Dec 23, 2019 at 7:44 PM Akira Ajisaka 
>>> wrote:
>>>
 Thank you, Ayush.

 I understand we should keep branch-2 as is, as well as master.

 -Akira


 On Mon, Dec 23, 2019 at 9:14 PM Ayush Saxena 
 wrote:

 > Hi Akira
 > Seems there was an INFRA ticket for that. INFRA-19581,
 > But the INFRA people closed as wont do and yes, the branch is
 protected,
 > we can’t delete it directly.
 >
 > Ref: https://issues.apache.org/jira/browse/INFRA-19581
 >
 > -Ayush
 >
 > On 23-Dec-2019, at 5:03 PM, Akira Ajisaka 
 wrote:
 >
 > Thank you for your work, Jonathan.
 >
 > I found branch-2 has been unintentionally pushed again. Would you
 remove
 > it?
 > I think the branch should be protected if possible.
 >
 > -Akira
 >
 > On Tue, Dec 10, 2019 at 5:17 AM Jonathan Hung 
 > wrote:
 >
 > It's done. The new commit chain is: trunk -> branch-3.2 -> branch-3.1
 ->
 >
 > branch-2.10 -> branch-2.9 -> branch-2.8 (branch-2 no longer exists,
 please
 >
 > don't try to commit to it)
 >
 >
 > Completed procedure:
 >
 >
 >   - Verified everything in old branch-2.10 was in old branch-2
 >
 >   - Delete old branch-2.10
 >
 >   - Rename branch-2 to (new) branch-2.10
 >
 >   - Set version in new branch-2.10 to 2.10.1-SNAPSHOT
 >
 >   - Renamed fix versions from 2.11.0 to 2.10.1
 >
 >   - Removed 2.11.0 as a version in HADOOP/YARN/HDFS/MAPREDUCE
 >
 >
 >
 > Jonathan Hung
 >
 >
 >
 > On Wed, Dec 4, 2019 at 10:55 AM Jonathan Hung 
 >
 > wrote:
 >
 >
 > FYI, starting the rename process, beginning with INFRA-19521.
 >
 >
 > Jonathan Hung
 >
 >
 >
 > On Wed, Nov 27, 2019 at 12:15 PM Konstantin Shvachko <
 >
 > shv.had...@gmail.com>
 >
 > wrote:
 >
 >
 > Hey guys,
 >
 >
 > I think we diverged a bit from the initial topic of this discussion,
 >
 > which is removing branch-2.10, and changing the version of branch-2
 from
 >
 > 2.11.0-SNAPSHOT to 2.10.1-SNAPSHOT.
 >
 > Sounds like the subject line for this thread "Making 2.10 the last
 minor
 >
 > 2.x release" confused people.
 >
 > It is in fact a wider matter that can be discussed when somebody
 >
 > actually
 >
 > proposes to release 2.11, which I understand nobody does at the
 moment.
 >
 >
 > So if anybody objects removing branch-2.10 please make an argument.
 >
 > Otherwise we should go ahead and just do it next week.
 >
 > I see people still struggling to keep branch-2 and branch-2.10 in
 sync.
 >
 >
 > Thanks,
 >
 > --Konstantin
 >
 >
 > On Thu, Nov 21, 2019 at 3:49 PM Jonathan Hung 
 >
 > wrote:
 >
 >
 > Thanks for the detailed thoughts, everyone.
 >
 >
 > Eric (Badger), my understanding is the same as yours re. minor vs
 patch
 >
 > releases. As for putting features into minor/patch releases, if we
 >
 > keep the
 >
 > convention of putting new features only into minor releases, my
 >
 > assumption
 >
 > is still that it's unlikely people will want to get them into branch-2
 >
 > (based on the 2.10.0 release process). For the java 11 issue, we
 >
 > haven't
 >
 > even really removed support for java 7 in branch-2 (much less java 8),
 >
 > so I
 >
 > feel moving to java 11 would go along with a move to branch 3. And as
 >
 > you
 >
 > mentioned, if people really want to use java 11 on branch-2, we can
 >
 > always
 >
 > revive branch-2. But for now I think the convenience of not needing to
 >
 > port
 >
 > to both branch-2 and branch-2.10 (and below) outweighs the cost of
 >
 > potentially needing to revive branch-2.
 >
 >
 > Jonathan Hung
 >
 >
 >

[jira] [Created] (HADOOP-16991) Remove RetryInvocation INFO logging from ozone CLI output

[Hanisha Koneru (Jira)]

Hanisha Koneru created HADOOP-16991:
---

 Summary: Remove RetryInvocation INFO logging from ozone CLI output
 Key: HADOOP-16991
 URL: https://issues.apache.org/jira/browse/HADOOP-16991
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Nilotpal Nandi
Assignee: Hanisha Koneru


In OM HA failover proxy provider, RetryInvocationHandler logs error message 
when client tries contacting non-leader OM. This error message can be 
suppressed as the failover would happen to leader OM.

{code:java}
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ozone.om.exceptions.OMNotLeaderException):
 OM:om2 is not the leader. Suggested leader is OM:om3. at 
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.createNotLeaderException(OzoneManagerProtocolServerSideTranslatorPB.java:186)
 at 
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitReadRequestToOM(OzoneManagerProtocolServerSideTranslatorPB.java:174)
 at 
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.processRequest(OzoneManagerProtocolServerSideTranslatorPB.java:110)
 at 
org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:72)
 at 
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:98)
 at 
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
 at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025) at 
org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876) at 
org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822) at 
java.security.AccessController.doPrivileged(Native Method) at 
javax.security.auth.Subject.doAs(Subject.java:422) at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
 at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682), while invoking 
$Proxy16.submitRequest over nodeId=om2,nodeAddress=om2:9862 after 1 failover 
attempts. Trying to failover immediately.
{code}




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1471/

[Apr 15, 2020 5:24:04 AM] (snemeth) YARN-10234. FS-CS converter: don't enable 
auto-create queue property for
[Apr 15, 2020 8:09:37 AM] (pjoseph) YARN-10233. Fix YARN UI2 Daemon Logs
[Apr 15, 2020 11:18:44 AM] (snemeth) YARN-. 
TestFSSchedulerConfigurationStore: Extend from
[Apr 15, 2020 3:08:19 PM] (vinayakumarb) HADOOP-16985. Handle release package 
related issues (#1957)
[Apr 15, 2020 5:31:58 PM] (ayushsaxena) HDFS-15277. Parent directory in the 
explorer does not support all path




-1 overall


The following subsystems voted -1:
asflicense findbugs pathlen unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

XML :

   Parsing Error(s): 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml
 

FindBugs :

   
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
 
   org.apache.hadoop.yarn.server.webapp.WebServiceClient.sslFactory should 
be package protected At WebServiceClient.java: At WebServiceClient.java:[line 
42] 

FindBugs :

   module:hadoop-cloud-storage-project/hadoop-cos 
   Redundant nullcheck of dir, which is known to be non-null in 
org.apache.hadoop.fs.cosn.BufferPool.createDir(String) Redundant null check at 
BufferPool.java:is known to be non-null in 
org.apache.hadoop.fs.cosn.BufferPool.createDir(String) Redundant null check at 
BufferPool.java:[line 66] 
   org.apache.hadoop.fs.cosn.CosNInputStream$ReadBuffer.getBuffer() may 
expose internal representation by returning CosNInputStream$ReadBuffer.buffer 
At CosNInputStream.java:by returning CosNInputStream$ReadBuffer.buffer At 
CosNInputStream.java:[line 87] 
   Found reliance on default encoding in 
org.apache.hadoop.fs.cosn.CosNativeFileSystemStore.storeFile(String, File, 
byte[]):in org.apache.hadoop.fs.cosn.CosNativeFileSystemStore.storeFile(String, 
File, byte[]): new String(byte[]) At CosNativeFileSystemStore.java:[line 199] 
   Found reliance on default encoding in 
org.apache.hadoop.fs.cosn.CosNativeFileSystemStore.storeFileWithRetry(String, 
InputStream, byte[], long):in 
org.apache.hadoop.fs.cosn.CosNativeFileSystemStore.storeFileWithRetry(String, 
InputStream, byte[], long): new String(byte[]) At 
CosNativeFileSystemStore.java:[line 178] 
   org.apache.hadoop.fs.cosn.CosNativeFileSystemStore.uploadPart(File, 
String, String, int) may fail to clean up java.io.InputStream Obligation to 
clean up resource created at CosNativeFileSystemStore.java:fail to clean up 
java.io.InputStream Obligation to clean up resource created at 
CosNativeFileSystemStore.java:[line 252] is not discharged 

Failed junit tests :

   hadoop.hdfs.server.namenode.ha.TestConfiguredFailoverProxyProvider 
   hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate 
   hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy 
   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks 
   hadoop.hdfs.TestDecommissionWithStriped 
   hadoop.hdfs.TestFileChecksumCompositeCrc 
   hadoop.yarn.applications.distributedshell.TestDistributedShell 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1471/artifact/out/diff-compile-cc-root.txt
  [8.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1471/artifact/out/diff-compile-javac-root.txt
  [428K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1471/artifact/out/diff-checkstyle-root.txt
  [16M]

   pathlen:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1471/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1471/artifact/out/diff-patch-pylint.txt
  [24K]

   shellcheck:

   

Apache Hadoop qbt Report: branch2.10+JDK7 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/657/

No changes




-1 overall


The following subsystems voted -1:
asflicense findbugs hadolint pathlen unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

XML :

   Parsing Error(s): 
   
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/empty-configuration.xml
 
   hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml 
   hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/public/crossdomain.xml 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml
 

FindBugs :

   module:hadoop-common-project/hadoop-minikdc 
   Possible null pointer dereference in 
org.apache.hadoop.minikdc.MiniKdc.delete(File) due to return value of called 
method Dereferenced at 
MiniKdc.java:org.apache.hadoop.minikdc.MiniKdc.delete(File) due to return value 
of called method Dereferenced at MiniKdc.java:[line 515] 

FindBugs :

   module:hadoop-common-project/hadoop-auth 
   
org.apache.hadoop.security.authentication.server.MultiSchemeAuthenticationHandler.authenticate(HttpServletRequest,
 HttpServletResponse) makes inefficient use of keySet iterator instead of 
entrySet iterator At MultiSchemeAuthenticationHandler.java:of keySet iterator 
instead of entrySet iterator At MultiSchemeAuthenticationHandler.java:[line 
192] 

FindBugs :

   module:hadoop-common-project/hadoop-common 
   org.apache.hadoop.crypto.CipherSuite.setUnknownValue(int) 
unconditionally sets the field unknownValue At CipherSuite.java:unknownValue At 
CipherSuite.java:[line 44] 
   org.apache.hadoop.crypto.CryptoProtocolVersion.setUnknownValue(int) 
unconditionally sets the field unknownValue At 
CryptoProtocolVersion.java:unknownValue At CryptoProtocolVersion.java:[line 67] 
   Possible null pointer dereference in 
org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(File) due to return value of 
called method Dereferenced at 
FileUtil.java:org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(File) due to 
return value of called method Dereferenced at FileUtil.java:[line 118] 
   Possible null pointer dereference in 
org.apache.hadoop.fs.RawLocalFileSystem.handleEmptyDstDirectoryOnWindows(Path, 
File, Path, File) due to return value of called method Dereferenced at 
RawLocalFileSystem.java:org.apache.hadoop.fs.RawLocalFileSystem.handleEmptyDstDirectoryOnWindows(Path,
 File, Path, File) due to return value of called method Dereferenced at 
RawLocalFileSystem.java:[line 383] 
   Useless condition:lazyPersist == true at this point At 
CommandWithDestination.java:[line 502] 
   org.apache.hadoop.io.DoubleWritable.compareTo(DoubleWritable) 
incorrectly handles double value At DoubleWritable.java: At 
DoubleWritable.java:[line 78] 
   org.apache.hadoop.io.DoubleWritable$Comparator.compare(byte[], int, int, 
byte[], int, int) incorrectly handles double value At DoubleWritable.java:int) 
incorrectly handles double value At DoubleWritable.java:[line 97] 
   org.apache.hadoop.io.FloatWritable.compareTo(FloatWritable) incorrectly 
handles float value At FloatWritable.java: At FloatWritable.java:[line 71] 
   org.apache.hadoop.io.FloatWritable$Comparator.compare(byte[], int, int, 
byte[], int, int) incorrectly handles float value At FloatWritable.java:int) 
incorrectly handles float value At FloatWritable.java:[line 89] 
   Possible null pointer dereference in 
org.apache.hadoop.io.IOUtils.listDirectory(File, FilenameFilter) due to return 
value of called method Dereferenced at 
IOUtils.java:org.apache.hadoop.io.IOUtils.listDirectory(File, FilenameFilter) 
due to return value of called method Dereferenced at IOUtils.java:[line 389] 
   Possible bad parsing of shift operation in 
org.apache.hadoop.io.file.tfile.Utils$Version.hashCode() At 
Utils.java:operation in 
org.apache.hadoop.io.file.tfile.Utils$Version.hashCode() At Utils.java:[line 
398] 
   
org.apache.hadoop.metrics2.lib.DefaultMetricsFactory.setInstance(MutableMetricsFactory)
 unconditionally sets the field mmfImpl At DefaultMetricsFactory.java:mmfImpl 
At DefaultMetricsFactory.java:[line 49] 
   
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.setMiniClusterMode(boolean) 
unconditionally sets the field miniClusterMode At 
DefaultMetricsSystem.java:miniClusterMode At DefaultMetricsSystem.java:[line 
92] 
   Useless object stored in variable seqOs of method 
org.apache.hadoop.security.token.delegation.ZKDelegationTokenSecretManager.addOrUpdateToken(AbstractDelegationTokenIdentifier,
 AbstractDelegationTokenSecretManager$DelegationTokenInformation, boolean) At 
ZKDelegationTokenSecretManager.java:seqOs of method 

[jira] [Resolved] (HADOOP-16979) S3Guard auth mode should be set to false by default in integration tests.

[Gabor Bota (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-16979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Bota resolved HADOOP-16979.
-
Resolution: Fixed

> S3Guard auth mode should be set to false by default in integration tests.
> -
>
> Key: HADOOP-16979
> URL: https://issues.apache.org/jira/browse/HADOOP-16979
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: fs/s3, test
>Affects Versions: 3.2.0, 3.3.0
>Reporter: Mukund Thakur
>Assignee: Mukund Thakur
>Priority: Minor
>
> As per the doc 
> [https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md#testing-s3a-with-s3guard-enabled]
> , it says if s3Guard profile is enabled all tests runs in "non-authoritative" 
> mode. But as per the code 
> [https://github.com/apache/hadoop/blob/8d49229c3764a205f21750225005a2c9a8124ab9/hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestUtils.java#L485]
> the default value is set to true.  So either we fix the doc or code. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org