[jira] [Created] (HADOOP-18150) Fix ITestAuditManagerDisabled after S3A audit logging was enabled in HADOOP-18091

[Mehakmeet Singh (Jira)]

Mehakmeet Singh created HADOOP-18150:


 Summary: Fix ITestAuditManagerDisabled after S3A audit logging was 
enabled in HADOOP-18091
 Key: HADOOP-18150
 URL: https://issues.apache.org/jira/browse/HADOOP-18150
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs/s3
Affects Versions: 3.3.2
Reporter: Mehakmeet Singh
Assignee: Mehakmeet Singh


In HADOOP-18094, we disabled audit logging by default and introduced an IT 
which verifies that by default the audit logging is disabled. After 
HADOOP-18091, we switched the audit logging back on, but that test remained, 
which now fails since by default we have the audit logging enabled now.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: branch-3.2+JDK8 on Linux/x86_64

[Apache Jenkins Server]

For more details, see 
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/

[Feb 28, 2022 1:41:47 AM] (Akira Ajisaka) HADOOP-17112. S3A committers can't 
handle whitespace in paths. (#2953)
[Feb 28, 2022 1:54:06 AM] (Akira Ajisaka) HADOOP-16717. Remove GenericsUtil 
isLog4jLogger dependency on Log4jLoggerAdapter. Contributed by Xieming Li.
[Mar 1, 2022 5:06:39 PM] (Szilard Nemeth) YARN-11075. Explicitly declare 
serialVersionUID in LogMutation class. Contributed by Benjamin Teke




-1 overall


The following subsystems voted -1:
asflicense blanks hadolint pathlen unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

XML :

   Parsing Error(s): 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml
 

Failed junit tests :

   hadoop.hdfs.server.datanode.TestBPOfferService 
   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer 
   hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2 
   hadoop.mapred.uploader.TestFrameworkUploader 
   hadoop.yarn.sls.TestSLSRunner 
  

   cc:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/results-compile-cc-root.txt
 [48K]

   javac:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/results-compile-javac-root.txt
 [336K]

   blanks:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/blanks-eol.txt
 [13M]
  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/blanks-tabs.txt
 [2.0M]

   checkstyle:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/results-checkstyle-root.txt
 [14M]

   hadolint:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/results-hadolint.txt
 [8.0K]

   pathlen:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/results-pathlen.txt
 [16K]

   pylint:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/results-pylint.txt
 [148K]

   shellcheck:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/results-shellcheck.txt
 [20K]

   xml:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/xml.txt
 [16K]

   javadoc:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/results-javadoc-javadoc-root.txt
 [1.7M]

   unit:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 [532K]
  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
 [16K]
  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-uploader.txt
 [12K]
  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/patch-unit-hadoop-tools_hadoop-sls.txt
 [12K]

   asflicense:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/36/artifact/out/results-asflicense.txt
 [4.0K]

Powered by Apache Yetus 0.14.0-SNAPSHOT   https://yetus.apache.org

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Invitation to old meetup sent by mistake

[Mara Ruvalcaba]

Dear Hadoop community,

sorry for the old message that was just sent!!!

We will share the next meetup info as soon as possible.


--
Mara Ruvalcaba
COO, SG Software Guru & Nearshore Link
USA: 512 296 2884
MX: 55 5239 5502


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [DISCUSS] Race condition in ProtobufRpcEngine2

[Vinayakumar B]

Call.toString() was used only while sending the response, in the response
processor thread. At this point, header would be already read and
processed. So no chance of race conditions.

This race condition was not introduced by the HADOOP-17046
, instead existing code
was never intended to be used for logging purposes by multiple threads.
Though the locking mechanism proposed in HADOOP-18143
 solves the problems,
it adds overhead in both memory as well as performance.

This is a critical path for RPC processing, and adding a
locking/synchronization in this place would affect the overall performance
(may not be huge).

I would advise to remove logging and locking both altogether and leave the
code as is in Reader thread.

If logging is required, then add it in the response processor thread or the
handler threads.

-Vinay


On Mon, Feb 28, 2022 at 10:02 PM Ayush Saxena  wrote:

> Hey Andras,
> I had a quick look at HADOOP-18143, the methods in question in
> ProtobufRpcEngine2 are identical to the ones in ProtobufRpcEngine. So, I am
> not very sure how the  race condition doesn't happen in  ProtobufRpcEngine.
> I have to debug and spend some more time, considering that I have
> reverted HADOOP-18082 for now to unblock YARN. Though the issue would still
> be there as you said, but will give us some time to analyse.
>
> Thanks
> -Ayush
>
> On Mon, 28 Feb 2022 at 21:26, Gyori Andras 
> wrote:
>
>> Hey everyone!
>>
>> We have started seeing test failures in YARN PRs for a while. We have
>> identified the problematic commit, which is HADOOP-18082
>> , however, this
>> change
>> just revealed the race condition lying in ProtobufRpcEngine2 introduced in
>> HADOOP-17046 . We
>> have
>> also fixed the underlying issue via a locking mechanism, presented in
>> HADOOP-18143 , but
>> since it is out of our area of expertise, we can neither verify nor
>> guarantee that it will not cause some subtle issues in the RPC system.
>> As we think it is a core part of Hadoop, we would use feedback from
>> someone
>> who is proficient in this part.
>>
>> Regards:
>> Andras
>>
>

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC5

[Chao Sun]

Thanks all!

With 3 binding votes and 2 non-binding votes (including mine), this vote
now passes. As the next step, I'll publish the release and send an
announcement email once it's done.

Best,
Chao

On Fri, Feb 25, 2022 at 10:58 PM Masatake Iwasaki <
iwasak...@oss.nttdata.co.jp> wrote:

> +1 (binding)
>
> * verified signature and checksum of the source tarball.
> * built the source code on Rocky Linux 8 (x86_64) and OpenJDK 8
>by `mvn install -DskipTests -Pnative -Pdist`.
> * launched pseudo distributed cluster with Kerberos security enabled and
> ran sample MR jobs.
> * launched HA enabled 3-nodes docker cluster and ran sample MR jobs.
> * launched pseudo distributed cluster and `spark-shell --master yarn`
>with spark-3.2.1-bin-without-hadoop and ran some tutorial code.
> * built site documentation by `mvn site site:stage` and skimmed the
> contents.
>
> Thanks,
> Masatake Iwasaki
>
> On 2022/02/22 14:17, Chao Sun wrote:
> > Hi all,
> >
> > Here's Hadoop 3.3.2 release candidate #5:
> >
> > The RC is available at:
> http://people.apache.org/~sunchao/hadoop-3.3.2-RC5
> > The RC tag is at:
> > https://github.com/apache/hadoop/releases/tag/release-3.3.2-RC5
> > The Maven artifacts are staged at:
> > https://repository.apache.org/content/repositories/orgapachehadoop-1335
> >
> > You can find my public key at:
> > https://downloads.apache.org/hadoop/common/KEYS
> >
> > CHANGELOG is the only difference between this and RC4. Therefore, the
> tests
> > I've done in RC4 are still valid:
> > - Ran all the unit tests
> > - Started a single node HDFS cluster and tested a few simple commands
> > - Ran all the tests in Spark using the RC5 artifacts
> >
> > Please evaluate the RC and vote, thanks!
> >
> > Best,
> > Chao
> >
>

[jira] [Resolved] (HADOOP-18128) outputstream.md typo issue

[Hui Fei (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-18128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hui Fei resolved HADOOP-18128.
--
Fix Version/s: 3.4.0
   Resolution: Fixed

> outputstream.md typo issue
> --
>
> Key: HADOOP-18128
> URL: https://issues.apache.org/jira/browse/HADOOP-18128
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: leo sun
>Assignee: leo sun
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: image-2022-02-17-10-53-01-704.png
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> There is a typo issue in outputstream.md on the branch – trunk
> !image-2022-02-17-10-53-01-704.png!



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64

[Apache Jenkins Server]

For more details, see 
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/797/

[Mar 1, 2022 11:48:10 AM] (noreply) HADOOP-18075. ABFS: Fix failure caused by 
listFiles() in ITestAbfsRestOperationException  (#4040)


[Error replacing 'FILE' - Workspace is not accessible]

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64

[Apache Jenkins Server]

For more details, see 
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/589/

No changes


[Error replacing 'FILE' - Workspace is not accessible]

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org