[jira] [Resolved] (HADOOP-16507) S3Guard fsck: Add option to configure severity (level) for the scan
[
https://issues.apache.org/jira/browse/HADOOP-16507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Thakur resolved HADOOP-16507.
Resolution: Won't Fix
> S3Guard fsck: Add option to configure severity (level) for the scan
> ---
>
> Key: HADOOP-16507
> URL: https://issues.apache.org/jira/browse/HADOOP-16507
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.0
>Reporter: Gabor Bota
>Assignee: Mukund Thakur
>Priority: Major
>
> There's the severity of Violation (inconsistency) defined in
> {{org.apache.hadoop.fs.s3a.s3guard.S3GuardFsck.Violation}}.
> This flag is only for defining the severity of the Violation, but not used to
> filter the scan for issue severity.
> The task to do: Use the severity level to define which issue should be logged
> and/or fixed during the scan.
> Note: the best way to avoid possible code duplication would be to not even
> add the consistency violation pair to the list of violations during the scan.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[ANNOUNCE] Mukund Thakur is a new Apache Hadoop Committer
Hi all, It's my pleasure to announce that Mukund Thakur has been elected as a committer on the Apache Hadoop project recognizing his continued contributions to the project. He has accepted the invitation. His involvement in the hadoop object store modules, hadoop common and distcp tool is a great addition to the Hadoop project. His reviews are comprehensive with high standards to "approve a pull request". The granting of committership to him will enable better productivity. Please join me in congratulating him. Welcome aboard, Mukund! Mingliang Liu (on behalf of the Apache Hadoop PMC)
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/405/ [Feb 1, 2021 4:42:44 AM] (noreply) HADOOP-17424. Replace HTrace with No-Op tracer (#2645) [Feb 1, 2021 9:51:00 AM] (noreply) HADOOP-17354. Move Jenkinsfile outside of the root directory. (#2647) [Feb 1, 2021 9:56:59 AM] (noreply) HDFS-15791. Possible Resource Leak in FSImageFormatProtobuf. (#2652) [Feb 1, 2021 3:28:29 PM] (Xiaoqiao He) HDFS-15792. ClasscastException while loading FSImage. Contributed by Renukaprasad C. [Feb 1, 2021 5:40:01 PM] (noreply) HADOOP-17482: Remove Commons Logger from FileSystem Class (#2633) -1 overall The following subsystems voted -1: asflicense findbugs pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml findbugs : module:hadoop-yarn-project/hadoop-yarn Possible null pointer dereference of LegacyMappingRuleToJson.userGroupMappingRules in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.converter.LegacyMappingRuleToJson.convert() Dereferenced at LegacyMappingRuleToJson.java:LegacyMappingRuleToJson.userGroupMappingRules in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.converter.LegacyMappingRuleToJson.convert() Dereferenced at LegacyMappingRuleToJson.java:[line 151] findbugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server Possible null pointer dereference of LegacyMappingRuleToJson.userGroupMappingRules in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.converter.LegacyMappingRuleToJson.convert() Dereferenced at LegacyMappingRuleToJson.java:LegacyMappingRuleToJson.userGroupMappingRules in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.converter.LegacyMappingRuleToJson.convert() Dereferenced at LegacyMappingRuleToJson.java:[line 151] findbugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Possible null pointer dereference of LegacyMappingRuleToJson.userGroupMappingRules in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.converter.LegacyMappingRuleToJson.convert() Dereferenced at LegacyMappingRuleToJson.java:LegacyMappingRuleToJson.userGroupMappingRules in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.converter.LegacyMappingRuleToJson.convert() Dereferenced at LegacyMappingRuleToJson.java:[line 151] findbugs : module:root Possible null pointer dereference of LegacyMappingRuleToJson.userGroupMappingRules in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.converter.LegacyMappingRuleToJson.convert() Dereferenced at LegacyMappingRuleToJson.java:LegacyMappingRuleToJson.userGroupMappingRules in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.converter.LegacyMappingRuleToJson.convert() Dereferenced at LegacyMappingRuleToJson.java:[line 151] Failed junit tests : hadoop.hdfs.server.balancer.TestBalancer hadoop.yarn.applications.distributedshell.TestDSWithMultipleNodeManager hadoop.tools.dynamometer.TestDynamometerInfra hadoop.tools.dynamometer.TestDynamometerInfra cc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/405/artifact/out/diff-compile-cc-root.txt [116K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/405/artifact/out/diff-compile-javac-root.txt [560K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/405/artifact/out/diff-checkstyle-root.txt [16M] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/405/artifact/out/pathlen.txt [12K] pylint:
[jira] [Created] (HADOOP-17512) Remove the enable/disable flag for ABFSRemoteListIterator
Bilahari T H created HADOOP-17512: - Summary: Remove the enable/disable flag for ABFSRemoteListIterator Key: HADOOP-17512 URL: https://issues.apache.org/jira/browse/HADOOP-17512 Project: Hadoop Common Issue Type: Improvement Components: fs/azure Affects Versions: 3.4.0 Reporter: Bilahari T H -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17509) Parallelize building of dependencies
[ https://issues.apache.org/jira/browse/HADOOP-17509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-17509. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Committed to trunk. Thank you [~gautham]. > Parallelize building of dependencies > > > Key: HADOOP-17509 > URL: https://issues.apache.org/jira/browse/HADOOP-17509 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.3.0 >Reporter: Gautham Banasandra >Assignee: Gautham Banasandra >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Need to use make -j$(nproc) to parallelize building of Protocol buffers and > Intel ISA - LÂ dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17511) Add an Audit plugin point for S3A auditing/context
Steve Loughran created HADOOP-17511: --- Summary: Add an Audit plugin point for S3A auditing/context Key: HADOOP-17511 URL: https://issues.apache.org/jira/browse/HADOOP-17511 Project: Hadoop Common Issue Type: Sub-task Affects Versions: 3.3.1 Reporter: Steve Loughran Assignee: Steve Loughran Add a way for auditing tools to correlate S3 object calls with Hadoop FS API calls. Initially just to log/forward to an auditing service. Later: let us attach them as parameters in S3 requests, such as opentrace headeers or (my initial idea: http referrer header -where it will get into the log) Challenges * ensuring the audit span is created for every public entry point. That will have to include those used in s3guard tools, some defacto public APIs * and not re-entered for active spans. s3A code must not call back into the FS API points * Propagation across worker threads -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
