[jira] [Commented] (HADOOP-19033) S3A: disable checksum validation
[ https://issues.apache.org/jira/browse/HADOOP-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809304#comment-17809304 ] Akira Ajisaka commented on HADOOP-19033: Thanks a lot [~ste...@apache.org] [~slfan1989]! > S3A: disable checksum validation > > > Key: HADOOP-19033 > URL: https://issues.apache.org/jira/browse/HADOOP-19033 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Fix For: 3.5.0 > > > AWS v2 sdk turns on client-side checksum validation; this kills performance > Given we are using TLS to download from AWS s3, there's implicit channel > checksumming going on on, that's along with the IPv4 TCP checksumming. > We don't need it, all it does is slow us down. > proposed: disable in DefaultS3ClientFactory > I don't want to add an option to enable it as it only complicates life (yet > another config option), but I am open to persuasion -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19033) S3A: disable checksum validation
[ https://issues.apache.org/jira/browse/HADOOP-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808617#comment-17808617 ] Akira Ajisaka commented on HADOOP-19033: This issue seems critical. Hi [~slfan1989], would you backport it to branch-3.4 and 3.4.0 to include in Hadoop 3.4.0 release? > S3A: disable checksum validation > > > Key: HADOOP-19033 > URL: https://issues.apache.org/jira/browse/HADOOP-19033 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > > AWS v2 sdk turns on client-side checksum validation; this kills performance > Given we are using TLS to download from AWS s3, there's implicit channel > checksumming going on on, that's along with the IPv4 TCP checksumming. > We don't need it, all it does is slow us down. > proposed: disable in DefaultS3ClientFactory > I don't want to add an option to enable it as it only complicates life (yet > another config option), but I am open to persuasion -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18912) upgrade snappy-java to 1.1.10.4 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18912: --- Fix Version/s: 3.3.9 > upgrade snappy-java to 1.1.10.4 due to CVE > -- > > Key: HADOOP-18912 > URL: https://issues.apache.org/jira/browse/HADOOP-18912 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9 > > > follow up to HADOOP-18782 > https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18850) Enable dual-layer server-side encryption with AWS KMS keys (DSSE-KMS)
Akira Ajisaka created HADOOP-18850: -- Summary: Enable dual-layer server-side encryption with AWS KMS keys (DSSE-KMS) Key: HADOOP-18850 URL: https://issues.apache.org/jira/browse/HADOOP-18850 Project: Hadoop Common Issue Type: Sub-task Components: fs/s3, security Reporter: Akira Ajisaka Add support for DSSE-KMS https://docs.aws.amazon.com/AmazonS3/latest/userguide/specifying-dsse-encryption.html -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18747) AWS SDK V2 - sigv2 support
[ https://issues.apache.org/jira/browse/HADOOP-18747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17727358#comment-17727358 ] Akira Ajisaka commented on HADOOP-18747: Hi [~ahmarsu], is this feature really required after migrating to AWS SDK v2? User can still use some existing version of hadoop-aws module to use AWS SDK v1 that supports sigV2. > AWS SDK V2 - sigv2 support > -- > > Key: HADOOP-18747 > URL: https://issues.apache.org/jira/browse/HADOOP-18747 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Ahmar Suhail >Priority: Major > > AWS SDK V2 does not support sigV2 signing. However, the S3 client supports > configurable signers so a custom sigV2 signer can be implemented and > configured. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17326) mvn verify fails due to duplicate entry in the shaded jar
[
https://issues.apache.org/jira/browse/HADOOP-17326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka resolved HADOOP-17326.
Resolution: Cannot Reproduce
Closing. I successfully ran {{mvn verify}} in both trunk and branch-3.3 on my
local.
Please feel free to reopen this if it's still failing in some environment.
> mvn verify fails due to duplicate entry in the shaded jar
> -
>
> Key: HADOOP-17326
> URL: https://issues.apache.org/jira/browse/HADOOP-17326
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.2.2, 3.3.1, 3.4.0
>Reporter: Wei-Chiu Chuang
>Priority: Blocker
>
> Found this when I was chasing a separate shading error with [~smeng].
> In trunk:
> run mvn verify under hadoop-client-module/
> {noformat}
> [INFO]
>
> [INFO] Reactor Summary for Apache Hadoop Client Modules 3.4.0-SNAPSHOT:
> [INFO]
> [INFO] Apache Hadoop Client Aggregator SUCCESS [ 2.607
> s]
> [INFO] Apache Hadoop Client API ... SUCCESS [03:16
> min]
> [INFO] Apache Hadoop Client Runtime ... SUCCESS [01:30
> min]
> [INFO] Apache Hadoop Client Test Minicluster .. FAILURE [04:44
> min]
> [INFO] Apache Hadoop Client Packaging Invariants .. SKIPPED
> [INFO] Apache Hadoop Client Packaging Invariants for Test . SKIPPED
> [INFO] Apache Hadoop Client Packaging Integration Tests ... SKIPPED
> [INFO] Apache Hadoop Client Modules ... SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 09:34 min
> [INFO] Finished at: 2020-10-23T16:38:53-07:00
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> hadoop-client-minicluster: Error creating shaded jar: duplicate entry:
> META-INF/services/org.apache.hadoop.shaded.com.fasterxml.jackson.core.JsonFactory
> -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> swit
> {noformat}
> This is reproducible in trunk and branch-3.3. However, not reproducible in
> branch-3.1.
> (branch-3.3 has a different error:
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> hadoop-client-minicluster: Error creating shaded jar: duplicate entry:
> META-INF/services/org.apache.hadoop.shaded.javax.ws.rs.ext.MessageBodyReader
> -> [Help 1])
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17326) mvn verify fails due to duplicate entry in the shaded jar
[
https://issues.apache.org/jira/browse/HADOOP-17326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643931#comment-17643931
]
Akira Ajisaka commented on HADOOP-17326:
Is this issue still valid? It's set to blocker but there's no progress for more
than 2 years.
> mvn verify fails due to duplicate entry in the shaded jar
> -
>
> Key: HADOOP-17326
> URL: https://issues.apache.org/jira/browse/HADOOP-17326
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.2.2, 3.3.1, 3.4.0
>Reporter: Wei-Chiu Chuang
>Priority: Blocker
>
> Found this when I was chasing a separate shading error with [~smeng].
> In trunk:
> run mvn verify under hadoop-client-module/
> {noformat}
> [INFO]
>
> [INFO] Reactor Summary for Apache Hadoop Client Modules 3.4.0-SNAPSHOT:
> [INFO]
> [INFO] Apache Hadoop Client Aggregator SUCCESS [ 2.607
> s]
> [INFO] Apache Hadoop Client API ... SUCCESS [03:16
> min]
> [INFO] Apache Hadoop Client Runtime ... SUCCESS [01:30
> min]
> [INFO] Apache Hadoop Client Test Minicluster .. FAILURE [04:44
> min]
> [INFO] Apache Hadoop Client Packaging Invariants .. SKIPPED
> [INFO] Apache Hadoop Client Packaging Invariants for Test . SKIPPED
> [INFO] Apache Hadoop Client Packaging Integration Tests ... SKIPPED
> [INFO] Apache Hadoop Client Modules ... SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 09:34 min
> [INFO] Finished at: 2020-10-23T16:38:53-07:00
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> hadoop-client-minicluster: Error creating shaded jar: duplicate entry:
> META-INF/services/org.apache.hadoop.shaded.com.fasterxml.jackson.core.JsonFactory
> -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> swit
> {noformat}
> This is reproducible in trunk and branch-3.3. However, not reproducible in
> branch-3.1.
> (branch-3.3 has a different error:
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project
> hadoop-client-minicluster: Error creating shaded jar: duplicate entry:
> META-INF/services/org.apache.hadoop.shaded.javax.ws.rs.ext.MessageBodyReader
> -> [Help 1])
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17177) Java 17 support
[ https://issues.apache.org/jira/browse/HADOOP-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17639969#comment-17639969 ] Akira Ajisaka commented on HADOOP-17177: Hi [~richa...@gmail.com] now there's no Hadoop version compatible with Java 17. > Java 17 support > --- > > Key: HADOOP-17177 > URL: https://issues.apache.org/jira/browse/HADOOP-17177 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Priority: Major > > Umbrella JIRA to support Java 17 LTS. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18532) Update command usage in FileSystemShell.md
[ https://issues.apache.org/jira/browse/HADOOP-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18532: --- Summary: Update command usage in FileSystemShell.md (was: fix typos in FileSystemShell) > Update command usage in FileSystemShell.md > -- > > Key: HADOOP-18532 > URL: https://issues.apache.org/jira/browse/HADOOP-18532 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.3.4 >Reporter: guophilipse >Assignee: guophilipse >Priority: Trivial > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9 > > > Fixt typos in FileSystemShell.md -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18532) fix typos in FileSystemShell
[ https://issues.apache.org/jira/browse/HADOOP-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-18532. Fix Version/s: 3.4.0 3.3.9 Resolution: Fixed Committed to trunk and branch-3.3. > fix typos in FileSystemShell > > > Key: HADOOP-18532 > URL: https://issues.apache.org/jira/browse/HADOOP-18532 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.3.4 >Reporter: guophilipse >Assignee: guophilipse >Priority: Trivial > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9 > > > Fixt typos in FileSystemShell.md -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18532) fix typos in FileSystemShell
[ https://issues.apache.org/jira/browse/HADOOP-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-18532: -- Assignee: guophilipse > fix typos in FileSystemShell > > > Key: HADOOP-18532 > URL: https://issues.apache.org/jira/browse/HADOOP-18532 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.3.4 >Reporter: guophilipse >Assignee: guophilipse >Priority: Trivial > Labels: pull-request-available > > Fixt typos in FileSystemShell.md -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18532) fix typos in FileSystemShell
[ https://issues.apache.org/jira/browse/HADOOP-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18532: --- Issue Type: Bug (was: Improvement) > fix typos in FileSystemShell > > > Key: HADOOP-18532 > URL: https://issues.apache.org/jira/browse/HADOOP-18532 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.3.4 >Reporter: guophilipse >Priority: Trivial > Labels: pull-request-available > > Fixt typos in FileSystemShell.md -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-8728) Display (fs -text) shouldn't hard-depend on Writable serialized sequence files.
[ https://issues.apache.org/jira/browse/HADOOP-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-8728: -- Fix Version/s: 3.4.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Display (fs -text) shouldn't hard-depend on Writable serialized sequence > files. > --- > > Key: HADOOP-8728 > URL: https://issues.apache.org/jira/browse/HADOOP-8728 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 2.6.0 >Reporter: Harsh J >Assignee: Ashutosh Gupta >Priority: Minor > Labels: BB2015-05-TBR, pull-request-available > Fix For: 3.4.0 > > Attachments: HADOOP-8728-002.patch, HADOOP-8728-003.patch, > HADOOP-8728.patch > > > The Display command (fs -text) currently reads only Writable-based > SequenceFiles. This isn't necessary to do, and prevents reading > non-Writable-based serialization in SequenceFiles from the shell. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18472) Upgrade to snakeyaml 1.33
[ https://issues.apache.org/jira/browse/HADOOP-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-18472. Fix Version/s: 3.4.0 3.2.5 3.3.9 Resolution: Fixed Committed to trunk, branch-3.3, and branch-3.2. > Upgrade to snakeyaml 1.33 > - > > Key: HADOOP-18472 > URL: https://issues.apache.org/jira/browse/HADOOP-18472 > Project: Hadoop Common > Issue Type: Improvement >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.2.5, 3.3.9 > > > Recent snakeyaml fixes missed a use case. Relates to HADOOP-18443 > [https://bitbucket.org/snakeyaml/snakeyaml/wiki/Changes] > * Fix > [#553|https://bitbucket.org/snakeyaml/snakeyaml/issues/553/loaderoptionssetcodepointlimit-not-honored]: > LoaderOptions.setCodePointLimit() not honored by loadAll() (thanks to Robert > Patrick) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18440) Replace Google Analytics with ASF Matomo in website
[ https://issues.apache.org/jira/browse/HADOOP-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615583#comment-17615583 ] Akira Ajisaka commented on HADOOP-18440: Thanks [~martijnvisser] for your comment. I raised the same issue in Kyuubi and have been fixed by the latest several commits in https://github.com/apache/incubator-kyuubi-website/commits/master [~groot] I think you can follow the above changes in Kyuubi as both Kyuubi and Hadoop use Hugo for building website. Thanks. > Replace Google Analytics with ASF Matomo in website > --- > > Key: HADOOP-18440 > URL: https://issues.apache.org/jira/browse/HADOOP-18440 > Project: Hadoop Common > Issue Type: Task > Components: site >Reporter: Akira Ajisaka >Assignee: Ashutosh Gupta >Priority: Major > > We currently use Google Analytics > (https://github.com/apache/hadoop-site/blob/asf-site/layouts/partials/footer.html#L37) > in the website but it's not recommended. > Also we should link the Privacy Policy to > https://privacy.apache.org/policies/privacy-policy-public.html instead of > Hadoop's one https://hadoop.apache.org/privacy_policy.html > For more details, please check: > https://privacy.apache.org/faq/committers.html and > https://www.apache.org/foundation/marks/pmcs.html#navigation -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-8728) Display (fs -text) shouldn't hard-depend on Writable serialized sequence files.
[ https://issues.apache.org/jira/browse/HADOOP-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-8728: - Assignee: (was: Akira Ajisaka) > Display (fs -text) shouldn't hard-depend on Writable serialized sequence > files. > --- > > Key: HADOOP-8728 > URL: https://issues.apache.org/jira/browse/HADOOP-8728 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 2.6.0 >Reporter: Harsh J >Priority: Minor > Labels: BB2015-05-TBR > Attachments: HADOOP-8728-002.patch, HADOOP-8728-003.patch, > HADOOP-8728.patch > > > The Display command (fs -text) currently reads only Writable-based > SequenceFiles. This isn't necessary to do, and prevents reading > non-Writable-based serialization in SequenceFiles from the shell. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-9946) NumAllSinks metrics shows lower value than NumActiveSinks
[
https://issues.apache.org/jira/browse/HADOOP-9946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka reassigned HADOOP-9946:
-
Assignee: (was: Akira Ajisaka)
> NumAllSinks metrics shows lower value than NumActiveSinks
> -
>
> Key: HADOOP-9946
> URL: https://issues.apache.org/jira/browse/HADOOP-9946
> Project: Hadoop Common
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.1.0-beta
>Reporter: Akira Ajisaka
>Priority: Major
> Labels: BB2015-05-TBR, metrics
> Attachments: HADOOP-9946, HADOOP-9946.02.patch, HADOOP-9946.patch
>
>
> In my hadoop cluster (trunk), output metrics file is as follows:
> {code}
> $ less /tmp/nodemanager-metrics.out
> 1377894554661 metricssystem.MetricsSystem: Context=metricssystem,
> Hostname=hadoop, NumActiveSources=8, NumAllSources=8, NumActiveSinks=1,
> NumAllSinks=0, Sink_fileNumOps=0, ...
> {code}
> NumAllSinks should be equal to or greater than NumActiveSinks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17405) Upgrade Yetus to 0.13.0
[ https://issues.apache.org/jira/browse/HADOOP-17405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-17405. Assignee: (was: Akira Ajisaka) Resolution: Duplicate The Yetus version is already 0.14.0. > Upgrade Yetus to 0.13.0 > --- > > Key: HADOOP-17405 > URL: https://issues.apache.org/jira/browse/HADOOP-17405 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Akira Ajisaka >Priority: Major > > After HADOOP-17262 and HADOOP-17297, Hadoop is using a non-release version of > Apache Yetus. It should be upgraded to 0.13.0 when released. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-11733) mvn clean does not clean /tmp
[ https://issues.apache.org/jira/browse/HADOOP-11733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-11733: -- Assignee: (was: Akira Ajisaka) > mvn clean does not clean /tmp > - > > Key: HADOOP-11733 > URL: https://issues.apache.org/jira/browse/HADOOP-11733 > Project: Hadoop Common > Issue Type: Bug > Components: build, test >Affects Versions: 2.6.0 > Environment: All >Reporter: Tony Reix >Priority: Minor > > When Hadoop tests are run, files and directories are created in /tmp . > Many (or all) of them are not cleaned when running "mvn clean". > This generates an issue when tests were previously run (by mistake) as root > and then run again as a non-root user, since directories (like: .hdfs-nfs > hadoop-root hsperfdata_root) cannot be written by second test run. > List of files/directories that seem to be created in /tmp when testing Hadoop: >Jetty* *test *deferred .hdfs-nfs hadoop-root hsperfdata_root > and that should be cleaned after tests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-15877) Upgrade ZooKeeper version to 3.5.4-beta and Curator version to 4.0.1
[ https://issues.apache.org/jira/browse/HADOOP-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-15877. Assignee: (was: Akira Ajisaka) Resolution: Invalid Now ZooKeeper version is 3.6.3 and Curator version is 5.2.0. > Upgrade ZooKeeper version to 3.5.4-beta and Curator version to 4.0.1 > > > Key: HADOOP-15877 > URL: https://issues.apache.org/jira/browse/HADOOP-15877 > Project: Hadoop Common > Issue Type: Improvement > Components: ha >Reporter: Akira Ajisaka >Priority: Major > > A long-term option to fix YARN-8937. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18200) Update ZooKeeper to 3.6.4 when released
[ https://issues.apache.org/jira/browse/HADOOP-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-18200: -- Assignee: (was: Akira Ajisaka) > Update ZooKeeper to 3.6.4 when released > --- > > Key: HADOOP-18200 > URL: https://issues.apache.org/jira/browse/HADOOP-18200 > Project: Hadoop Common > Issue Type: Improvement > Components: build, ha >Reporter: Akira Ajisaka >Priority: Major > > Regarding ZOOKEEPER-4455, ZooKeeper 3.6.4 uses reload4j instead of log4j 1.x. > Let's upgrade when released. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16769) LocalDirAllocator to provide diagnostics when file creation fails
[ https://issues.apache.org/jira/browse/HADOOP-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-16769: --- Fix Version/s: 3.4.0 > LocalDirAllocator to provide diagnostics when file creation fails > - > > Key: HADOOP-16769 > URL: https://issues.apache.org/jira/browse/HADOOP-16769 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Reporter: Ramesh Kumar Thangarajan >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > Attachments: HADOOP-16769.1.patch, HADOOP-16769.3.patch, > HADOOP-16769.4.patch, HADOOP-16769.5.patch, HADOOP-16769.6.patch, > HADOOP-16769.7.patch, HADOOP-16769.8.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Log details of requested size and available capacity when file creation is > not successuful -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18443) Upgrade snakeyaml to 1.32
[ https://issues.apache.org/jira/browse/HADOOP-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18443: --- Fix Version/s: 3.2.5 Committed to branch-3.2. > Upgrade snakeyaml to 1.32 > - > > Key: HADOOP-18443 > URL: https://issues.apache.org/jira/browse/HADOOP-18443 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 3.3.3, 3.3.4 >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9, 3.2.5 > > > Upgrade snakeyaml to 1.32 to mitigate CVE-2022-25857 and > [CVE-2022-38752|https://github.com/advisories/GHSA-9w3m-gqgf-c4p9] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18443) Upgrade snakeyaml to 1.32
[ https://issues.apache.org/jira/browse/HADOOP-18443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18443: --- Fix Version/s: 3.4.0 3.3.9 Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk and branch-3.3. Thank you [~groot] for the updates. > Upgrade snakeyaml to 1.32 > - > > Key: HADOOP-18443 > URL: https://issues.apache.org/jira/browse/HADOOP-18443 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 3.3.3, 3.3.4 >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9 > > > Upgrade snakeyaml to 1.32 to mitigate CVE-2022-25857 and > [CVE-2022-38752|https://github.com/advisories/GHSA-9w3m-gqgf-c4p9] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18400) Fix file split duplicating records from a succeeding split when reading BZip2 text files
[ https://issues.apache.org/jira/browse/HADOOP-18400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18400: --- Fix Version/s: 3.4.0 3.3.9 Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk and branch-3.3. > Fix file split duplicating records from a succeeding split when reading > BZip2 text files > -- > > Key: HADOOP-18400 > URL: https://issues.apache.org/jira/browse/HADOOP-18400 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.3.3, 3.3.4 >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Critical > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9 > > > Fix data correctness issue with TextInputFormat that can occur when reading > BZip2 compressed text files. When a file split's range does not include the > start position of a BZip2 block, then it is expected to contain no records > (i.e. the split is empty). However, if it so happens that the end of this > split (exclusive) is at the start of a BZip2 block, then LineRecordReader > ends up returning all the records for that BZip2 block. This ends up > duplicating records read by a job because the next split would also end up > returning all the records for the same block (since its range would include > the start of that block). > This bug does not get triggered when the file split's range does include the > start of at least one block and ends just before the start of another block. > The reason for this has to do with when BZip2CompressionInputStream updates > its position when using the BYBLOCK READMODE. Using this read mode, the > stream's position while reading only gets updated when reading the first byte > past an end of a block marker. The bug is that if the stream, when > initialized, was adjusted to be at the end of one block, then we don't update > the position after we read the first byte of the next block. Rather, we keep > the position to be equal to the next block marker we've initialized to. If > the exclusive end position of the split is equal to stream's position, > LineRecordReader will continue to read lines until the position is updated > (an an additional record in the next block is read if needed). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18302) Remove WhiteBox in hadoop-common module.
[ https://issues.apache.org/jira/browse/HADOOP-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-18302. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Committed to trunk. > Remove WhiteBox in hadoop-common module. > > > Key: HADOOP-18302 > URL: https://issues.apache.org/jira/browse/HADOOP-18302 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.4.0, 3.3.9 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > WhiteBox is deprecated, try to remove this method in hadoop-common. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18440) Replace Google Analytics with ASF Matomo in website
[ https://issues.apache.org/jira/browse/HADOOP-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18440: --- Issue Type: Task (was: Improvement) > Replace Google Analytics with ASF Matomo in website > --- > > Key: HADOOP-18440 > URL: https://issues.apache.org/jira/browse/HADOOP-18440 > Project: Hadoop Common > Issue Type: Task > Components: site >Reporter: Akira Ajisaka >Priority: Major > > We currently use Google Analytics > (https://github.com/apache/hadoop-site/blob/asf-site/layouts/partials/footer.html#L37) > in the website but it's not recommended. > Also we should link the Privacy Policy to > https://privacy.apache.org/policies/privacy-policy-public.html instead of > Hadoop's one https://hadoop.apache.org/privacy_policy.html > For more details, please check: > https://privacy.apache.org/faq/committers.html and > https://www.apache.org/foundation/marks/pmcs.html#navigation -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18440) Replace Google Analytics with ASF Matomo in website
Akira Ajisaka created HADOOP-18440: -- Summary: Replace Google Analytics with ASF Matomo in website Key: HADOOP-18440 URL: https://issues.apache.org/jira/browse/HADOOP-18440 Project: Hadoop Common Issue Type: Improvement Components: site Reporter: Akira Ajisaka We currently use Google Analytics (https://github.com/apache/hadoop-site/blob/asf-site/layouts/partials/footer.html#L37) in the website but it's not recommended. Also we should link the Privacy Policy to https://privacy.apache.org/policies/privacy-policy-public.html instead of Hadoop's one https://hadoop.apache.org/privacy_policy.html For more details, please check: https://privacy.apache.org/faq/committers.html and https://www.apache.org/foundation/marks/pmcs.html#navigation -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18321) Fix when to read an additional record from a BZip2 text file split
[
https://issues.apache.org/jira/browse/HADOOP-18321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-18321:
---
Component/s: io
> Fix when to read an additional record from a BZip2 text file split
> --
>
> Key: HADOOP-18321
> URL: https://issues.apache.org/jira/browse/HADOOP-18321
> Project: Hadoop Common
> Issue Type: Bug
> Components: io
>Affects Versions: 3.3.3
>Reporter: groot
>Assignee: groot
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.9
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Fix data correctness issue with TextInputFormat that can occur when reading
> BZip2 compressed text files. When triggered this bug would cause a split to
> return the first record of the succeeding split that reads the next BZip2
> block, thereby duplicating that record.
> *When the bug is triggered?*
> The condition for the bug to occur requires that the flag
> "needAdditionalRecord" in CompressedSplitLineReader to be set to true by
> #fillBuffer at an inappropriate time: when we haven't read the remaining
> bytes of split. This can happen when the inDelimiter parameter is true while
> #fillBuffer is invoked while reading the next line. The inDelimiter parameter
> is true when either 1) the last byte of the buffer is a CR character ('\r')
> if using the default delimiters, or 2) the last bytes of the buffer are a
> common prefix of the delimiter if using a custom delimiter.
> This can occur in various edge cases, illustrated by five unit tests added in
> this change -- specifically the five that would fail without the fix are as
> listed below:
> #
> BaseTestLineRecordReaderBZip2.customDelimiter_lastRecordDelimiterStartsAtNextBlockStart
> # BaseTestLineRecordReaderBZip2.firstBlockEndsWithLF_secondBlockStartsWithCR
> # BaseTestLineRecordReaderBZip2.delimitedByCRSpanningThreeBlocks
> # BaseTestLineRecordReaderBZip2.usingCRDelimiterWithSmallestBufferSize
> #
> BaseTestLineRecordReaderBZip2.customDelimiter_lastThreeBytesInBlockAreDelimiter
> For background, the purpose of "needAdditionalRecord" field in
> CompressedSplitLineReader is to indicate to LineRecordReader via the
> #needAdditionalRecordAfterSplit method that an extra record lying beyond the
> split range should be included in the split. This complication arises due to
> a problem when splitting text files. When a split starts at a position
> greater than zero, we do not know whether the first line belongs to the last
> record in the prior split or is a new record. The solution done in Hadoop is
> to make splits that start at position greater than zero to always discard the
> first line and then have the prior split decide whether it should include the
> first line of the next split or not (as part of the last record or as a new
> record). This works well even in the case of a single line spanning multiple
> splits.
> *What is the fix?*
> The fix is to prevent ever setting "needAdditionalRecord" if the bytes filled
> to the buffer are not the bytes immediately outside the range of the split.
> When reading compressed data, CompressedSplitLineReader requires/assumes that
> the stream's #read method never returns bytes from more than one compression
> block at a time. This ensures that #fillBuffer gets invoked to read the first
> byte of the next block. This next block may or may not be part of the split
> we are reading. If we detect that the last bytes of the prior block maybe
> part of a delimiter, then we may decide that we should read an additional
> record, but we should only do that when this next block is not part of our
> split *and* we aren't filling the buffer again beyond our split range. This
> is because we are only concerned whether the we need to read the very first
> line of the next split as a separate record. If it going to be part of the
> last record, then we don't need to read an extra record, or in the special
> case of CR + LF (i.e. "\r\n"), if the LF is the first byte of the next split,
> it will be treated as an empty line, thus we don't need to include an extra
> record into the mix.
> Thus, to emphasize. It is when we read the first bytes outside our split
> range that matters. But the current logic doesn't take that into account in
> CompressedSplitLineReader. This is in contrast to UncompressedSplitLineReader
> which does.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18390) Fix out of sync import for HADOOP-18321
[ https://issues.apache.org/jira/browse/HADOOP-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18390: --- Fix Version/s: 3.3.9 Target Version/s: 3.4.0, 3.3.9 (was: 3.4.0) Affects Version/s: (was: 3.4.0) Backported to branch-3.3. > Fix out of sync import for HADOOP-18321 > --- > > Key: HADOOP-18390 > URL: https://issues.apache.org/jira/browse/HADOOP-18390 > Project: Hadoop Common > Issue Type: Bug >Reporter: groot >Assignee: groot >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9 > > Time Spent: 20m > Remaining Estimate: 0h > > Fix out of sync import for added as part of HADOOP-18321 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18321) Fix when to read an additional record from a BZip2 text file split
[
https://issues.apache.org/jira/browse/HADOOP-18321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-18321:
---
Fix Version/s: 3.3.9
Backported to branch-3.3.
> Fix when to read an additional record from a BZip2 text file split
> --
>
> Key: HADOOP-18321
> URL: https://issues.apache.org/jira/browse/HADOOP-18321
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.3
>Reporter: groot
>Assignee: groot
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.9
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Fix data correctness issue with TextInputFormat that can occur when reading
> BZip2 compressed text files. When triggered this bug would cause a split to
> return the first record of the succeeding split that reads the next BZip2
> block, thereby duplicating that record.
> *When the bug is triggered?*
> The condition for the bug to occur requires that the flag
> "needAdditionalRecord" in CompressedSplitLineReader to be set to true by
> #fillBuffer at an inappropriate time: when we haven't read the remaining
> bytes of split. This can happen when the inDelimiter parameter is true while
> #fillBuffer is invoked while reading the next line. The inDelimiter parameter
> is true when either 1) the last byte of the buffer is a CR character ('\r')
> if using the default delimiters, or 2) the last bytes of the buffer are a
> common prefix of the delimiter if using a custom delimiter.
> This can occur in various edge cases, illustrated by five unit tests added in
> this change -- specifically the five that would fail without the fix are as
> listed below:
> #
> BaseTestLineRecordReaderBZip2.customDelimiter_lastRecordDelimiterStartsAtNextBlockStart
> # BaseTestLineRecordReaderBZip2.firstBlockEndsWithLF_secondBlockStartsWithCR
> # BaseTestLineRecordReaderBZip2.delimitedByCRSpanningThreeBlocks
> # BaseTestLineRecordReaderBZip2.usingCRDelimiterWithSmallestBufferSize
> #
> BaseTestLineRecordReaderBZip2.customDelimiter_lastThreeBytesInBlockAreDelimiter
> For background, the purpose of "needAdditionalRecord" field in
> CompressedSplitLineReader is to indicate to LineRecordReader via the
> #needAdditionalRecordAfterSplit method that an extra record lying beyond the
> split range should be included in the split. This complication arises due to
> a problem when splitting text files. When a split starts at a position
> greater than zero, we do not know whether the first line belongs to the last
> record in the prior split or is a new record. The solution done in Hadoop is
> to make splits that start at position greater than zero to always discard the
> first line and then have the prior split decide whether it should include the
> first line of the next split or not (as part of the last record or as a new
> record). This works well even in the case of a single line spanning multiple
> splits.
> *What is the fix?*
> The fix is to prevent ever setting "needAdditionalRecord" if the bytes filled
> to the buffer are not the bytes immediately outside the range of the split.
> When reading compressed data, CompressedSplitLineReader requires/assumes that
> the stream's #read method never returns bytes from more than one compression
> block at a time. This ensures that #fillBuffer gets invoked to read the first
> byte of the next block. This next block may or may not be part of the split
> we are reading. If we detect that the last bytes of the prior block maybe
> part of a delimiter, then we may decide that we should read an additional
> record, but we should only do that when this next block is not part of our
> split *and* we aren't filling the buffer again beyond our split range. This
> is because we are only concerned whether the we need to read the very first
> line of the next split as a separate record. If it going to be part of the
> last record, then we don't need to read an extra record, or in the special
> case of CR + LF (i.e. "\r\n"), if the LF is the first byte of the next split,
> it will be treated as an empty line, thus we don't need to include an extra
> record into the mix.
> Thus, to emphasize. It is when we read the first bytes outside our split
> range that matters. But the current logic doesn't take that into account in
> CompressedSplitLineReader. This is in contrast to UncompressedSplitLineReader
> which does.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18390) Fix out of sync import for HADOOP-18321
[ https://issues.apache.org/jira/browse/HADOOP-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-18390. Fix Version/s: 3.4.0 Resolution: Fixed Merged the PR into trunk. Thank you [~groot]! > Fix out of sync import for HADOOP-18321 > --- > > Key: HADOOP-18390 > URL: https://issues.apache.org/jira/browse/HADOOP-18390 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.4.0 >Reporter: groot >Assignee: groot >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Fix out of sync import for added as part of HADOOP-18321 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18390) Fix out of sync import for HADOOP-18321
[ https://issues.apache.org/jira/browse/HADOOP-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18390: --- Target Version/s: 3.4.0 (was: 3.3.9) Affects Version/s: 3.4.0 (was: 3.3.3) Issue Type: Bug (was: Improvement) Priority: Minor (was: Major) > Fix out of sync import for HADOOP-18321 > --- > > Key: HADOOP-18390 > URL: https://issues.apache.org/jira/browse/HADOOP-18390 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.4.0 >Reporter: groot >Assignee: groot >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Fix out of sync import for added as part of HADOOP-18321 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18301) Upgrade commons-io to 2.11.0
[ https://issues.apache.org/jira/browse/HADOOP-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-18301. Fix Version/s: 3.4.0 Resolution: Fixed Merged the PR into trunk. Thank you [~groot] for your contribution. > Upgrade commons-io to 2.11.0 > > > Key: HADOOP-18301 > URL: https://issues.apache.org/jira/browse/HADOOP-18301 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.2.3, 3.3.3 >Reporter: groot >Assignee: groot >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Current version 2.8.0 is almost ~2 years old > Upgrading to new release to keep up for new features and bug fixes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18294) Ensure build folder exists before writing checksum file.ProtocRunner#writeChecksums
[ https://issues.apache.org/jira/browse/HADOOP-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-18294. Fix Version/s: 3.4.0 Resolution: Fixed Merged the PR into trunk. Thank you [~groot] for your contribution. > Ensure build folder exists before writing checksum > file.ProtocRunner#writeChecksums > --- > > Key: HADOOP-18294 > URL: https://issues.apache.org/jira/browse/HADOOP-18294 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.3.3 >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 2h > Remaining Estimate: 0h > > Ensure build folder exists before writing checksum > file.ProtocRunner#writeChecksums -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564792#comment-17564792 ] Akira Ajisaka edited comment on HADOOP-18033 at 7/11/22 4:36 AM: - [~vjasani] Currently I recommend downgrading to 2.12.7 in both trunk and branch-3.3. That way we don't need to treat HADOOP-15984 as a blocker for 3.4.0. (maybe HADOOP-15984 is still a blocker for 3.4.0 regardless of this issue as it blocks compile Hadoop with Java 11) was (Author: ajisakaa): [~vjasani] Currently I recommend downgrading to 2.12.7 in both trunk and branch-3.3. That way we don't need to treat HADOOP-15984 as a blocker for 3.4.0. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564792#comment-17564792 ] Akira Ajisaka commented on HADOOP-18033: [~vjasani] Currently I recommend downgrading to 2.12.7 in both trunk and branch-3.3. That way we don't need to treat HADOOP-15984 as a blocker for 3.4.0. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564780#comment-17564780 ] Akira Ajisaka commented on HADOOP-18033: Note: When reverting this issue, I recommend to use a separate JIRA because the change was released in 3.3.2 and 3.3.3. That way we can easily track what change is in the specific release. > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564761#comment-17564761 ] Akira Ajisaka edited comment on HADOOP-18033 at 7/11/22 2:28 AM: - bq. we might have to call out on the Jackson CVE The CVE is fixed in 2.12.6.1 or upper (https://github.com/FasterXML/jackson-databind/issues/2816), therefore we should change the version to 2.12.7 (the latest 2.12.x as of now). That way the vulnerability will be still fixed. was (Author: ajisakaa): bq. we might have to call out on the Jackson CVE The CVE is fixed in 2.12.6.1 or upper (https://github.com/FasterXML/jackson-databind/issues/2816), therefore we should change the version to 2.12.7 (the latest 2.12.x as of now). > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564761#comment-17564761 ] Akira Ajisaka commented on HADOOP-18033: bq. we might have to call out on the Jackson CVE The CVE is fixed in 2.12.6.1 or upper (https://github.com/FasterXML/jackson-databind/issues/2816), therefore we should change the version to 2.12.7 (the latest 2.12.x as of now). > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564664#comment-17564664 ] Akira Ajisaka edited comment on HADOOP-18033 at 7/10/22 10:29 AM: -- Thank you [~ayushtkn] [~pan3793] [~vjasani] for your discussion. I'm really sorry for not caching up with. In my past experience, Jersey 2.x upgrade takes a lot of time and I think it will cause some incompatible changes. Therefore I think we should revert the patch and update to 2.12.x latest to avoid the above issue for (at least) Hadoop 3.3.x. What do you think? Jackson 2.12.x would work because I think the change in https://github.com/FasterXML/jackson-jaxrs-providers/issues/134 caused the issue and it is only in Jackson 2.13.0 and upper. was (Author: ajisakaa): Thank you [~ayushtkn] [~pan3793] [~vjasani] for your discussion. I'm really sorry for not caching up with. In my past experience, Jersey 2.x upgrade takes a lot of time and I think it will cause some incompatible changes. Therefore I think we should revert the patch and update to 2.12.x latest to avoid the above issue for (at least) Hadoop 3.3.x. What do you think? > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18033) Upgrade fasterxml Jackson to 2.13.0
[ https://issues.apache.org/jira/browse/HADOOP-18033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564664#comment-17564664 ] Akira Ajisaka commented on HADOOP-18033: Thank you [~ayushtkn] [~pan3793] [~vjasani] for your discussion. I'm really sorry for not caching up with. In my past experience, Jersey 2.x upgrade takes a lot of time and I think it will cause some incompatible changes. Therefore I think we should revert the patch and update to 2.12.x latest to avoid the above issue for (at least) Hadoop 3.3.x. What do you think? > Upgrade fasterxml Jackson to 2.13.0 > --- > > Key: HADOOP-18033 > URL: https://issues.apache.org/jira/browse/HADOOP-18033 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 5.5h > Remaining Estimate: 0h > > Spark 3.2.0 depends on Jackson 2.12.3. Let's upgrade to 2.12.5 (2.12.x latest > as of now) or upper. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18277) Remove org.apache.hadoop.test#Whitebox Class
[ https://issues.apache.org/jira/browse/HADOOP-18277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564659#comment-17564659 ] Akira Ajisaka commented on HADOOP-18277: Hi [~slfan1989] [~weichiu], I disagree with replacing Whitebox with FieldUtils just to avoid the warning in HADOOP-18289 and HADOOP-18302. Please refactor the code itself instead of changing the library. > Remove org.apache.hadoop.test#Whitebox Class > > > Key: HADOOP-18277 > URL: https://issues.apache.org/jira/browse/HADOOP-18277 > Project: Hadoop Common > Issue Type: Improvement >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > > org.apache.hadoop.test#Whitebox is marked as deprecated, I personally feel > that this is unnecessary, which leads to a large number of junit test code > appearing deprecated, and a large number of Warnings appear in the > compilation, I checked the code and think the deprecated mark is unreasonable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18297) Upgrade dependency-check-maven to 7.1.1
[ https://issues.apache.org/jira/browse/HADOOP-18297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17562489#comment-17562489 ] Akira Ajisaka commented on HADOOP-18297: Hi [~dannycjones] No worries, when closing JIRA, we need to set "Fix versions" as well but setting the version is not simple. Please ask some committer when you need to close issue in JIRA. JIRA Guidelines in https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute has more details. > Upgrade dependency-check-maven to 7.1.1 > --- > > Key: HADOOP-18297 > URL: https://issues.apache.org/jira/browse/HADOOP-18297 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.3.3 >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The OWASP dependency-check-maven Plugin version has corrected various false > positives in 7.1.1. We can upgrade to it. > https://github.com/jeremylong/DependencyCheck/milestone/45?closed=1 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18297) Upgrade dependency-check-maven to 7.1.1
[ https://issues.apache.org/jira/browse/HADOOP-18297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-18297. Fix Version/s: 3.4.0 Resolution: Fixed Re-open and close with correct resolution. > Upgrade dependency-check-maven to 7.1.1 > --- > > Key: HADOOP-18297 > URL: https://issues.apache.org/jira/browse/HADOOP-18297 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.3.3 >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The OWASP dependency-check-maven Plugin version has corrected various false > positives in 7.1.1. We can upgrade to it. > https://github.com/jeremylong/DependencyCheck/milestone/45?closed=1 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-18297) Upgrade dependency-check-maven to 7.1.1
[ https://issues.apache.org/jira/browse/HADOOP-18297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reopened HADOOP-18297: > Upgrade dependency-check-maven to 7.1.1 > --- > > Key: HADOOP-18297 > URL: https://issues.apache.org/jira/browse/HADOOP-18297 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.3.3 >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > The OWASP dependency-check-maven Plugin version has corrected various false > positives in 7.1.1. We can upgrade to it. > https://github.com/jeremylong/DependencyCheck/milestone/45?closed=1 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18288) Total requests and total requests per sec served by RPC servers
[ https://issues.apache.org/jira/browse/HADOOP-18288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557163#comment-17557163 ] Akira Ajisaka commented on HADOOP-18288: Hi [~vjasani] [~tomscut] Reading your comments, now I'm fine with the change. Thank you for your contribution! > Total requests and total requests per sec served by RPC servers > --- > > Key: HADOOP-18288 > URL: https://issues.apache.org/jira/browse/HADOOP-18288 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > RPC Servers provide bunch of useful information like num of open connections, > slow requests, num of in-progress handlers, RPC processing time, queue time > etc, however so far it doesn't provide accumulation of all requests as well > as current snapshot of requests per second served by the server. Exposing > them would benefit from operational viewpoint in identifying how busy the > servers have been and how much load they are currently serving in the > presence of cluster wide high load. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18294) Ensure build folder exists before writing checksum file.ProtocRunner#writeChecksums
[ https://issues.apache.org/jira/browse/HADOOP-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556208#comment-17556208 ] Akira Ajisaka commented on HADOOP-18294: Hi [~groot] is there any detailed background for this issue? Is there any error message you faced? > Ensure build folder exists before writing checksum > file.ProtocRunner#writeChecksums > --- > > Key: HADOOP-18294 > URL: https://issues.apache.org/jira/browse/HADOOP-18294 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.3.3 >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Ensure build folder exists before writing checksum > file.ProtocRunner#writeChecksums -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18288) Total requests and total requests per sec served by RPC servers
[ https://issues.apache.org/jira/browse/HADOOP-18288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553861#comment-17553861 ] Akira Ajisaka commented on HADOOP-18288: I'm +1 for exposing total requests as a metric, but I'm not sure we should expose total metrics per second as a metric. I suppose it should be calculated by the metrics collector such as Prometheus. What do you think [~vjasani]? > Total requests and total requests per sec served by RPC servers > --- > > Key: HADOOP-18288 > URL: https://issues.apache.org/jira/browse/HADOOP-18288 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > RPC Servers provide bunch of useful information like num of open connections, > slow requests, num of in-progress handlers, RPC processing time, queue time > etc, however so far it doesn't provide accumulation of all requests as well > as current snapshot of requests per second served by the server. Exposing > them would benefit from operational viewpoint in identifying how busy the > servers have been and how much load they are currently serving in the > presence of cluster wide high load. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18277) Remove org.apache.hadoop.test#Whitebox Deprecated Annotation
[ https://issues.apache.org/jira/browse/HADOOP-18277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17550807#comment-17550807 ] Akira Ajisaka commented on HADOOP-18277: -1 I think the deprecation is necessary. Use of the Whitebox feature is a bad practice, and all the test code relying on Whitebox should be refactored. Please see HADOOP-14188 for the background. > Remove org.apache.hadoop.test#Whitebox Deprecated Annotation > > > Key: HADOOP-18277 > URL: https://issues.apache.org/jira/browse/HADOOP-18277 > Project: Hadoop Common > Issue Type: Improvement >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > > org.apache.hadoop.test#Whitebox is marked as deprecated, I personally feel > that this is unnecessary, which leads to a large number of junit test code > appearing deprecated, and a large number of Warnings appear in the > compilation, I checked the code and think the deprecated mark is unreasonable. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18268) Install Maven from Apache archives
[
https://issues.apache.org/jira/browse/HADOOP-18268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17546057#comment-17546057
]
Akira Ajisaka commented on HADOOP-18268:
Note that it's not using Apache Archives, but using CDN. But keeping the issue
title as-is for better tracking.
> Install Maven from Apache archives
> --
>
> Key: HADOOP-18268
> URL: https://issues.apache.org/jira/browse/HADOOP-18268
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.4.0
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The Jenkins CI for Hadoop is failing to build since it's unable to download
> and install maven -
> {code}
> 22:38:13 #11 [ 7/16] RUN pkg-resolver/install-maven.sh centos:7
> 22:38:13 #11
> sha256:8b1823a6197611693af5daa2888f195db76ae5e9d0765f799becc7e7d5f7b019
> 22:40:25 #11 131.5 curl: (7) Failed to connect to 2403:8940:3:1::f: Cannot
> assign requested address
> 22:40:25 #11 ERROR: executor failed running [/bin/bash --login -c
> pkg-resolver/install-maven.sh centos:7]: exit code: 7
> {code}
> Jenkins run -
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4370/4/console
> We need to switch to using Maven from Apache archives to prevent such issues.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18240) Upgrade Yetus to 0.14.0
[ https://issues.apache.org/jira/browse/HADOOP-18240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-18240. Fix Version/s: 3.4.0 3.2.4 3.3.4 2.10.3 Resolution: Fixed Committed to trunk, branch-3.3, branch-3.2, and branch-2.10. > Upgrade Yetus to 0.14.0 > --- > > Key: HADOOP-18240 > URL: https://issues.apache.org/jira/browse/HADOOP-18240 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Assignee: Ashutosh Gupta >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.2.4, 3.3.4, 2.10.3 > > Time Spent: 1h > Remaining Estimate: 0h > > Yetus 0.14.0 is released. Let's upgrade. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-17591) Fix the wrong CIDR range example in Proxy User documentation
[ https://issues.apache.org/jira/browse/HADOOP-17591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541844#comment-17541844 ] Akira Ajisaka edited comment on HADOOP-17591 at 5/25/22 6:23 AM: - Fixed by HADOOP-17592. Closing. was (Author: ajisakaa): Fixed by HADOOP-17952. Closing. > Fix the wrong CIDR range example in Proxy User documentation > > > Key: HADOOP-17591 > URL: https://issues.apache.org/jira/browse/HADOOP-17591 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.2.2 >Reporter: Kwangsun Noh >Priority: Trivial > Labels: newbie > > The CIDR range example on the Proxy user description page is wrong. > > In the Configurations section of the Proxy user page, CIDR 10.222.0.0/16 > means range 10.222.0.0-15. > > But It's not true. CIDR format 10.222.0.0/16 means 10.222.0.0-10.222.255.255 > > as-is : 10.222.0.0-15 > to-be : 10.222.0.0-10.222.255.255 > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17591) Fix the wrong CIDR range example in Proxy User documentation
[ https://issues.apache.org/jira/browse/HADOOP-17591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-17591. Resolution: Duplicate Fixed by HADOOP-17952. Closing. > Fix the wrong CIDR range example in Proxy User documentation > > > Key: HADOOP-17591 > URL: https://issues.apache.org/jira/browse/HADOOP-17591 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.2.2 >Reporter: Kwangsun Noh >Priority: Trivial > Labels: newbie > > The CIDR range example on the Proxy user description page is wrong. > > In the Configurations section of the Proxy user page, CIDR 10.222.0.0/16 > means range 10.222.0.0-15. > > But It's not true. CIDR format 10.222.0.0/16 means 10.222.0.0-10.222.255.255 > > as-is : 10.222.0.0-15 > to-be : 10.222.0.0-10.222.255.255 > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18140) Update default value of hadoop.ssl.enabled.protocols in the EncryptedShuffle doc
[ https://issues.apache.org/jira/browse/HADOOP-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-18140: -- Assignee: Samrat Deb > Update default value of hadoop.ssl.enabled.protocols in the EncryptedShuffle > doc > > > Key: HADOOP-18140 > URL: https://issues.apache.org/jira/browse/HADOOP-18140 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 2.9.0, 2.8.4 >Reporter: Wei-Chiu Chuang >Assignee: Samrat Deb >Priority: Trivial > Labels: newbie > > The default value of hadoop.ssl.enabled.protocols was updated several times > but our doc was never updated. > https://hadoop.apache.org/docs/r3.1.3/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18140) Update default value of hadoop.ssl.enabled.protocols in the EncryptedShuffle doc
[ https://issues.apache.org/jira/browse/HADOOP-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541415#comment-17541415 ] Akira Ajisaka commented on HADOOP-18140: Sure. Assigned. > Update default value of hadoop.ssl.enabled.protocols in the EncryptedShuffle > doc > > > Key: HADOOP-18140 > URL: https://issues.apache.org/jira/browse/HADOOP-18140 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 2.9.0, 2.8.4 >Reporter: Wei-Chiu Chuang >Assignee: Samrat Deb >Priority: Trivial > Labels: newbie > > The default value of hadoop.ssl.enabled.protocols was updated several times > but our doc was never updated. > https://hadoop.apache.org/docs/r3.1.3/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18224) Upgrade maven compiler plugin to 3.10.1
[
https://issues.apache.org/jira/browse/HADOOP-18224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka resolved HADOOP-18224.
Fix Version/s: 3.4.0
Resolution: Fixed
Merged the PR into trunk. Thanks!
> Upgrade maven compiler plugin to 3.10.1
> ---
>
> Key: HADOOP-18224
> URL: https://issues.apache.org/jira/browse/HADOOP-18224
> Project: Hadoop Common
> Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> Currently we are using maven-compiler-plugin 3.1 version, which is quite old
> (2013) and it's also pulling in vulnerable log4j dependency:
> {code:java}
> [INFO]
> org.apache.maven.plugins:maven-compiler-plugin:maven-plugin:3.1:runtime
> [INFO] org.apache.maven.plugins:maven-compiler-plugin:jar:3.1
> [INFO] org.apache.maven:maven-plugin-api:jar:2.0.9
> [INFO] org.apache.maven:maven-artifact:jar:2.0.9
> [INFO] org.codehaus.plexus:plexus-utils:jar:1.5.1
> [INFO] org.apache.maven:maven-core:jar:2.0.9
> [INFO] org.apache.maven:maven-settings:jar:2.0.9
> [INFO] org.apache.maven:maven-plugin-parameter-documenter:jar:2.0.9
> ...
> ...
> ...
> [INFO] log4j:log4j:jar:1.2.12
> [INFO] commons-logging:commons-logging-api:jar:1.1
> [INFO] com.google.collections:google-collections:jar:1.0
> [INFO] junit:junit:jar:3.8.2
> {code}
>
> We should upgrade to 3.10.1 (latest Mar, 2022) version of
> maven-compiler-plugin.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15984) Update jersey from 1.19 to 2.x
[
https://issues.apache.org/jira/browse/HADOOP-15984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539203#comment-17539203
]
Akira Ajisaka commented on HADOOP-15984:
{quote}Do you plan to merge this PR? or are you aware of any workaround.
{quote}
Currently I don't plan to merge this PR because I found it was very hard to fix
and I don't have enough time to do. Now I don't aware any workaround.
> Update jersey from 1.19 to 2.x
> --
>
> Key: HADOOP-15984
> URL: https://issues.apache.org/jira/browse/HADOOP-15984
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Akira Ajisaka
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> jersey-json 1.19 depends on Jackson 1.9.2. Let's upgrade.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18240) Upgrade Yetus to 0.14.0
[ https://issues.apache.org/jira/browse/HADOOP-18240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538329#comment-17538329 ] Akira Ajisaka commented on HADOOP-18240: Need to update * HADOOP_YETUS_VERSION in dev-support/bin/yetus-wrapper * YETUS_VERSION in dev-support/Jenkinsfile > Upgrade Yetus to 0.14.0 > --- > > Key: HADOOP-18240 > URL: https://issues.apache.org/jira/browse/HADOOP-18240 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Akira Ajisaka >Priority: Major > > Yetus 0.14.0 is released. Let's upgrade. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18240) Upgrade Yetus to 0.14.0
Akira Ajisaka created HADOOP-18240: -- Summary: Upgrade Yetus to 0.14.0 Key: HADOOP-18240 URL: https://issues.apache.org/jira/browse/HADOOP-18240 Project: Hadoop Common Issue Type: Improvement Components: build Reporter: Akira Ajisaka Yetus 0.14.0 is released. Let's upgrade. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18222) Prevent DelegationTokenSecretManagerMetrics from registering multiple times
[
https://issues.apache.org/jira/browse/HADOOP-18222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-18222:
---
Issue Type: Bug (was: Improvement)
Priority: Major (was: Minor)
> Prevent DelegationTokenSecretManagerMetrics from registering multiple times
>
>
> Key: HADOOP-18222
> URL: https://issues.apache.org/jira/browse/HADOOP-18222
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Hector Sandoval Chaverri
>Assignee: Hector Sandoval Chaverri
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.4
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> After committing HADOOP-18167, we received reports of the following error
> when ResourceManager is initialized:
> {noformat}
> Caused by: java.io.IOException: Problem starting http server
> at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:1389)
> at
> org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:475)
> ... 4 more
> Caused by: org.apache.hadoop.metrics2.MetricsException: Metrics source
> DelegationTokenSecretManagerMetrics already exists!
> at
> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:152)
> at
> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:125)
> at
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:229)
> at
> org.apache.hadoop.metrics2.MetricsSystem.register(MetricsSystem.java:71)
> at
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager$DelegationTokenSecretManagerMetrics.create(AbstractDelegationTokenSecretManager.java:878)
> at
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.(AbstractDelegationTokenSecretManager.java:152)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenManager$DelegationTokenSecretManager.(DelegationTokenManager.java:72)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenManager.(DelegationTokenManager.java:122)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.initTokenManager(DelegationTokenAuthenticationHandler.java:161)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.init(DelegationTokenAuthenticationHandler.java:130)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.initializeAuthHandler(AuthenticationFilter.java:194)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.initializeAuthHandler(DelegationTokenAuthenticationFilter.java:214)
> at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:180)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.init(DelegationTokenAuthenticationFilter.java:180)
> at
> org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter.init(RMAuthenticationFilter.java:53){noformat}
> This can happen if MetricsSystemImpl#init is called and multiple metrics are
> registered with the same name. A proposed solution is to declare the metrics
> in AbstractDelegationTokenSecretManager as singleton, which would prevent
> multiple instances DelegationTokenSecretManagerMetrics from being registered.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18167) Add metrics to track delegation token secret manager operations
[ https://issues.apache.org/jira/browse/HADOOP-18167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-18167: -- Assignee: Hector Sandoval Chaverri > Add metrics to track delegation token secret manager operations > --- > > Key: HADOOP-18167 > URL: https://issues.apache.org/jira/browse/HADOOP-18167 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Hector Sandoval Chaverri >Assignee: Hector Sandoval Chaverri >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.4 > > Attachments: HADOOP-18167-branch-2.10-2.patch, > HADOOP-18167-branch-2.10-3.patch, HADOOP-18167-branch-2.10-4.patch, > HADOOP-18167-branch-2.10.patch, HADOOP-18167-branch-3.3.patch > > Time Spent: 6h 10m > Remaining Estimate: 0h > > New metrics to track operations that store, update and remove delegation > tokens in implementations of AbstractDelegationTokenSecretManager. This will > help evaluate the impact of using different secret managers and add > optimizations. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18167) Add metrics to track delegation token secret manager operations
[ https://issues.apache.org/jira/browse/HADOOP-18167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18167: --- Fix Version/s: 3.4.0 3.3.4 > Add metrics to track delegation token secret manager operations > --- > > Key: HADOOP-18167 > URL: https://issues.apache.org/jira/browse/HADOOP-18167 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Hector Sandoval Chaverri >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.4 > > Attachments: HADOOP-18167-branch-2.10-2.patch, > HADOOP-18167-branch-2.10-3.patch, HADOOP-18167-branch-2.10-4.patch, > HADOOP-18167-branch-2.10.patch, HADOOP-18167-branch-3.3.patch > > Time Spent: 6h 10m > Remaining Estimate: 0h > > New metrics to track operations that store, update and remove delegation > tokens in implementations of AbstractDelegationTokenSecretManager. This will > help evaluate the impact of using different secret managers and add > optimizations. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17479) Fix the examples of hadoop config prefix
[
https://issues.apache.org/jira/browse/HADOOP-17479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka resolved HADOOP-17479.
Fix Version/s: 3.4.0
3.2.4
3.3.4
Resolution: Fixed
Committed to trunk, branch-3.3, and branch-3.2. Thank you [~groot] for your
contribution!
> Fix the examples of hadoop config prefix
>
>
> Key: HADOOP-17479
> URL: https://issues.apache.org/jira/browse/HADOOP-17479
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Ashutosh Gupta
>Priority: Minor
> Labels: newbie, pull-request-available
> Fix For: 3.4.0, 3.2.4, 3.3.4
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In
> https://hadoop.apache.org/docs/r3.3.0/hadoop-project-dist/hadoop-common/DownstreamDev.html#XML_Configuration_Files
> {quote}e.g. hadoop, io, ipc, fs, net, file, ftp, kfs, ha, file, dfs, mapred,
> mapreduce, and yarn.
> {quote}
> * There are two "file"
> * kfs has been removed since Hadoop 2.x
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17479) Fix the examples of hadoop config prefix
[
https://issues.apache.org/jira/browse/HADOOP-17479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-17479:
---
Issue Type: Bug (was: Improvement)
> Fix the examples of hadoop config prefix
>
>
> Key: HADOOP-17479
> URL: https://issues.apache.org/jira/browse/HADOOP-17479
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Ashutosh Gupta
>Priority: Minor
> Labels: newbie, pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> In
> https://hadoop.apache.org/docs/r3.3.0/hadoop-project-dist/hadoop-common/DownstreamDev.html#XML_Configuration_Files
> {quote}e.g. hadoop, io, ipc, fs, net, file, ftp, kfs, ha, file, dfs, mapred,
> mapreduce, and yarn.
> {quote}
> * There are two "file"
> * kfs has been removed since Hadoop 2.x
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16515) Update the link to compatibility guide
[ https://issues.apache.org/jira/browse/HADOOP-16515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-16515. Fix Version/s: 3.4.0 3.3.4 Resolution: Fixed Committed to trunk and branch-3.3. > Update the link to compatibility guide > -- > > Key: HADOOP-16515 > URL: https://issues.apache.org/jira/browse/HADOOP-16515 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Reporter: Akira Ajisaka >Assignee: Ashutosh Gupta >Priority: Minor > Labels: newbie, pull-request-available > Fix For: 3.4.0, 3.3.4 > > Attachments: HADOOP-16515-01.patch > > Time Spent: 2h 10m > Remaining Estimate: 0h > > There are many old URLs (http://wiki.apache.org/hadoop/Compatibility) in the > source code and they should be replaced with > http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16515) Update the link to compatibility guide
[ https://issues.apache.org/jira/browse/HADOOP-16515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-16515: --- Issue Type: Bug (was: Improvement) > Update the link to compatibility guide > -- > > Key: HADOOP-16515 > URL: https://issues.apache.org/jira/browse/HADOOP-16515 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Akira Ajisaka >Assignee: Ashutosh Gupta >Priority: Minor > Labels: newbie, pull-request-available > Fix For: 3.4.0, 3.3.4 > > Attachments: HADOOP-16515-01.patch > > Time Spent: 2h 10m > Remaining Estimate: 0h > > There are many old URLs (http://wiki.apache.org/hadoop/Compatibility) in the > source code and they should be replaced with > http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13332) Remove jackson 1.9.13 and switch all jackson code to 2.x code line
[ https://issues.apache.org/jira/browse/HADOOP-13332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-13332. Assignee: (was: Akira Ajisaka) Resolution: Done All the sub-tasks completed. Closing this umbrella JIRA. > Remove jackson 1.9.13 and switch all jackson code to 2.x code line > -- > > Key: HADOOP-13332 > URL: https://issues.apache.org/jira/browse/HADOOP-13332 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 2.8.0 >Reporter: PJ Fanning >Priority: Major > Attachments: HADOOP-13332.00.patch, HADOOP-13332.01.patch, > HADOOP-13332.02.patch, HADOOP-13332.03.patch > > > This jackson 1.9 code line is no longer maintained. Upgrade > Most changes from jackson 1.9 to 2.x just involve changing the package name. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18195) make jackson v1 a runtime scope dependency
[ https://issues.apache.org/jira/browse/HADOOP-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-18195. Resolution: Duplicate The jackson v1 dependency has been removed by HADOOP-15983. Closing. > make jackson v1 a runtime scope dependency > -- > > Key: HADOOP-18195 > URL: https://issues.apache.org/jira/browse/HADOOP-18195 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: PJ Fanning >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > In trunk, jackson v1 is only needed as a transitive dependency of jersey-json > (1.19). > The jackson v1 jars still appear in hadoop pom so it would be useful to make > them 'runtime' scope so that no hadoop code can compile with links to this > old code. > > [https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-project/3.4.0-SNAPSHOT/hadoop-project-3.4.0-20220407.091529-1274.pom] > > the linked PR is not needed if [https://github.com/apache/hadoop/pull/3988] > is merged -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-15983) Use jersey-json that is built to use jackson2
[ https://issues.apache.org/jira/browse/HADOOP-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-15983. Fix Version/s: 3.4.0 Hadoop Flags: Incompatible change Release Note: Use modified jersey-json 1.20 in https://github.com/pjfanning/jersey-1.x/tree/v1.20 that uses Jackson 2.x. By this change, Jackson 1.x dependency has been removed from Hadoop. Resolution: Fixed Merged the PR into trunk. Thank you [~pj.fanning] for your contribution! I really appreciate this work. > Use jersey-json that is built to use jackson2 > - > > Key: HADOOP-15983 > URL: https://issues.apache.org/jira/browse/HADOOP-15983 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15983) Use jersey-json that is built to use jackson2
[ https://issues.apache.org/jira/browse/HADOOP-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-15983: --- Summary: Use jersey-json that is built to use jackson2 (was: Remove the usage of jersey-json to remove jackson 1.x dependency.) > Use jersey-json that is built to use jackson2 > - > > Key: HADOOP-15983 > URL: https://issues.apache.org/jira/browse/HADOOP-15983 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 3h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15983) Use jersey-json that is built to use jackson2
[ https://issues.apache.org/jira/browse/HADOOP-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-15983: -- Assignee: PJ Fanning > Use jersey-json that is built to use jackson2 > - > > Key: HADOOP-15983 > URL: https://issues.apache.org/jira/browse/HADOOP-15983 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Time Spent: 3h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18198) Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
[ https://issues.apache.org/jira/browse/HADOOP-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526550#comment-17526550 ] Akira Ajisaka commented on HADOOP-18198: Built artifacts and uploaded to [https://dist.apache.org/repos/dist/dev/hadoop/3.3.3-RC0/] The maven artifacts are staged at [https://repository.apache.org/content/repositories/orgapachehadoop-1345] The RC tag is at: [https://github.com/apache/hadoop/releases/tag/release-3.3.3-RC0] Hi [~ste...@apache.org], would you look at the artifacts? If you think it's okay, you can start the release vote. bq. we will have to make the docker build configurable to at least optionally turn off http keepalives in maven builds. +1 > Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes > -- > > Key: HADOOP-18198 > URL: https://issues.apache.org/jira/browse/HADOOP-18198 > Project: Hadoop Common > Issue Type: Task > Components: build >Affects Versions: 3.3.2 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > Hadoop 3.3.3 is a minor followup release to Hadoop 3.3.2 with all the > incremental changes which went in to the 3.2.4 release > * minor CVE fixes in Hadoop source > * CVE fixes in dependencies we know of (protobuf unmarshalling leading to > DoS, jackson stack overflow,...) > * replacement of log4j 1.2.17 to reload4j > * node.js update > This is not a release off branch-3.3, it is a fork of 3.3.2 with the changes. > The next release of branch-3.3 will be numbered hadoop-3.3.4; updating maven > versions and JIRA fix versions is part of this release process. > The changes here are already in branch 3.2.4; this completes the set > CVEs fixed > * CVE-2022-26612: Apache Hadoop: Arbitrary file write in > FileUtil#unpackEntries on Windows (HADOOP-18155) -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18198) Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
[
https://issues.apache.org/jira/browse/HADOOP-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526301#comment-17526301
]
Akira Ajisaka commented on HADOOP-18198:
{quote}i've been setting up to do it from an ec2 vm, which doesn't (yet) have
my signing key.
{quote}
I think that's okay. I wanted to say that It's not good to upload your signing
key to an EC2 VM.
I'll try to build the artifacts tonight. Thank you.
> Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
> --
>
> Key: HADOOP-18198
> URL: https://issues.apache.org/jira/browse/HADOOP-18198
> Project: Hadoop Common
> Issue Type: Task
> Components: build
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Hadoop 3.3.3 is a minor followup release to Hadoop 3.3.2 with all the
> incremental changes which went in to the 3.2.4 release
> * minor CVE fixes in Hadoop source
> * CVE fixes in dependencies we know of (protobuf unmarshalling leading to
> DoS, jackson stack overflow,...)
> * replacement of log4j 1.2.17 to reload4j
> * node.js update
> This is not a release off branch-3.3, it is a fork of 3.3.2 with the changes.
> The next release of branch-3.3 will be numbered hadoop-3.3.4; updating maven
> versions and JIRA fix versions is part of this release process.
> The changes here are already in branch 3.2.4; this completes the set
> CVEs fixed
> * CVE-2022-26612: Apache Hadoop: Arbitrary file write in
> FileUtil#unpackEntries on Windows (HADOOP-18155)
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15783) [JDK10] TestSFTPFileSystem.testGetModifyTime fails
[
https://issues.apache.org/jira/browse/HADOOP-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-15783:
---
Fix Version/s: 2.10.2
Backported to branch-2.10 via https://github.com/apache/hadoop/pull/4218
> [JDK10] TestSFTPFileSystem.testGetModifyTime fails
> --
>
> Key: HADOOP-15783
> URL: https://issues.apache.org/jira/browse/HADOOP-15783
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.2.0, 2.10.2
>
> Attachments: HADOOP-15783.01.patch, HADOOP-15783.02.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> {noformat}
> [ERROR] Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 5.039 s <<< FAILURE! - in org.apache.hadoop.fs.sftp.TestSFTPFileSystem
> [ERROR] testGetModifyTime(org.apache.hadoop.fs.sftp.TestSFTPFileSystem) Time
> elapsed: 0.095 s <<< FAILURE!
> java.lang.AssertionError: expected:<1537836464942> but was:<1537836464000>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.fs.sftp.TestSFTPFileSystem.testGetModifyTime(TestSFTPFileSystem.java:329)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18198) Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
[ https://issues.apache.org/jira/browse/HADOOP-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526202#comment-17526202 ] Akira Ajisaka commented on HADOOP-18198: Hi [~ste...@apache.org], can I help you building the release artifacts? I don't think we cannot build the release artifacts on EC2 because the hardware is not under our control: https://www.apache.org/legal/release-policy.html#owned-controlled-hardware > Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes > -- > > Key: HADOOP-18198 > URL: https://issues.apache.org/jira/browse/HADOOP-18198 > Project: Hadoop Common > Issue Type: Task > Components: build >Affects Versions: 3.3.2 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > Hadoop 3.3.3 is a minor followup release to Hadoop 3.3.2 with all the > incremental changes which went in to the 3.2.4 release > * minor CVE fixes in Hadoop source > * CVE fixes in dependencies we know of (protobuf unmarshalling leading to > DoS, jackson stack overflow,...) > * replacement of log4j 1.2.17 to reload4j > * node.js update > This is not a release off branch-3.3, it is a fork of 3.3.2 with the changes. > The next release of branch-3.3 will be numbered hadoop-3.3.4; updating maven > versions and JIRA fix versions is part of this release process. > The changes here are already in branch 3.2.4; this completes the set > CVEs fixed > * CVE-2022-26612: Apache Hadoop: Arbitrary file write in > FileUtil#unpackEntries on Windows (HADOOP-18155) -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-18198) Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
[ https://issues.apache.org/jira/browse/HADOOP-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526202#comment-17526202 ] Akira Ajisaka edited comment on HADOOP-18198 at 4/22/22 5:43 AM: - Hi [~ste...@apache.org], can I help you building the release artifacts? I think we cannot build the release artifacts on EC2 because the hardware is not under our control: [https://www.apache.org/legal/release-policy.html#owned-controlled-hardware] was (Author: ajisakaa): Hi [~ste...@apache.org], can I help you building the release artifacts? I don't think we cannot build the release artifacts on EC2 because the hardware is not under our control: https://www.apache.org/legal/release-policy.html#owned-controlled-hardware > Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes > -- > > Key: HADOOP-18198 > URL: https://issues.apache.org/jira/browse/HADOOP-18198 > Project: Hadoop Common > Issue Type: Task > Components: build >Affects Versions: 3.3.2 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > Hadoop 3.3.3 is a minor followup release to Hadoop 3.3.2 with all the > incremental changes which went in to the 3.2.4 release > * minor CVE fixes in Hadoop source > * CVE fixes in dependencies we know of (protobuf unmarshalling leading to > DoS, jackson stack overflow,...) > * replacement of log4j 1.2.17 to reload4j > * node.js update > This is not a release off branch-3.3, it is a fork of 3.3.2 with the changes. > The next release of branch-3.3 will be numbered hadoop-3.3.4; updating maven > versions and JIRA fix versions is part of this release process. > The changes here are already in branch 3.2.4; this completes the set > CVEs fixed > * CVE-2022-26612: Apache Hadoop: Arbitrary file write in > FileUtil#unpackEntries on Windows (HADOOP-18155) -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17551) Upgrade maven-site-plugin to 3.11.0
[ https://issues.apache.org/jira/browse/HADOOP-17551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-17551. Fix Version/s: 3.4.0 3.3.4 Resolution: Fixed Committed to trunk and branch-3.3. Thank you [~groot] for your contribution. > Upgrade maven-site-plugin to 3.11.0 > --- > > Key: HADOOP-17551 > URL: https://issues.apache.org/jira/browse/HADOOP-17551 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Assignee: Ashutosh Gupta >Priority: Major > Labels: newbie, pull-request-available > Fix For: 3.4.0, 3.3.4 > > Time Spent: 50m > Remaining Estimate: 0h > > After upgrading maven-site-plugin, error messages will be more detailed and > it will help debugging. > > Ref: > https://issues.apache.org/jira/browse/YARN-10656?focusedCommentId=17291846=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17291846 -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18202) create-release fails fatal: unsafe repository ('/build/source' is owned by someone else)
[
https://issues.apache.org/jira/browse/HADOOP-18202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-18202:
---
Fix Version/s: 3.4.0
> create-release fails fatal: unsafe repository ('/build/source' is owned by
> someone else)
>
>
> Key: HADOOP-18202
> URL: https://issues.apache.org/jira/browse/HADOOP-18202
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.3.4, 3.3.3
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.3
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> The first git command in the create-release docker container is failing as
> git doesn't consider /build/source to belong to the current user.
> since CVE-2022-24765 and the april 2022 git releases to refuse to work in
> such directories
> https://github.blog/2022-04-12-git-security-vulnerability-announced/
> {code}
> $ /usr/bin/git clean -xdf -e /patchprocess
> fatal: unsafe repository ('/build/source' is owned by someone else)
> To add an exception for this directory, call:
> git config --global --add safe.directory /build/source
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18160) `org.wildfly.openssl` should not be shaded by Hadoop build
[
https://issues.apache.org/jira/browse/HADOOP-18160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-18160:
---
Fix Version/s: 3.4.0
> `org.wildfly.openssl` should not be shaded by Hadoop build
> --
>
> Key: HADOOP-18160
> URL: https://issues.apache.org/jira/browse/HADOOP-18160
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.3.1
> Environment: hadoop 3.3.1
> spark 3.2.1
> JDK8
>Reporter: André F.
>Assignee: André F.
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.3
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> `org.wildfly.openssl` is a runtime library and its references are being
> shaded on Hadoop, breaking the integration with other frameworks like Spark,
> whenever the "fs.s3a.ssl.channel.mode" is set to "openssl". The error
> produced in this situation is:
> {code:java}
> Suppressed: java.lang.NoClassDefFoundError:
> org/apache/hadoop/shaded/org/wildfly/openssl/OpenSSLProvider{code}
> Whenever it tries to be instantiated from the `DelegatingSSLSocketFactory`.
> Spark tries to add it to its classpath without the shade, thus creating this
> issue.
> Dependencies which are not on "compile" scope should probably not be shaded
> to avoid this kind of integration issues.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18125) Utility to identify git commit / Jira fixVersion discrepancies for RC preparation
[ https://issues.apache.org/jira/browse/HADOOP-18125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17524116#comment-17524116 ] Akira Ajisaka commented on HADOOP-18125: Hi [~ste...@apache.org], please don't remove 3.4.0 from the fix versions. Setting the version to 3.4.0 means the commit is merged in trunk, so the 3.4.0 version is meaningful. > Utility to identify git commit / Jira fixVersion discrepancies for RC > preparation > - > > Key: HADOOP-18125 > URL: https://issues.apache.org/jira/browse/HADOOP-18125 > Project: Hadoop Common > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 2.10.2, 3.2.3, 3.3.3 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > As part of RC preparation, we need to identify all git commits that landed > on release branch, however their corresponding Jira is either not resolved > yet or does not contain expected fixVersions. Only when we have git commits > and corresponding Jiras with expected fixVersion resolved, we get all such > Jiras included in auto-generated CHANGES.md as per Yetus changelog generator. > Proposal of this Jira is to provide such script that can be useful for all > upcoming RC preparations and list down all Jiras where we need manual > intervention. This utility script should use Jira API to retrieve individual > fields and use git log to loop through commit history. > The script should identify these issues: > # commit is reverted as per commit message > # commit does not contain Jira number format (e.g. HADOOP- / HDFS- > etc) in message > # Jira does not have expected fixVersion > # Jira has expected fixVersion, but it is not yet resolved > # Jira has release corresponding fixVersion and is resolved, but no > corresponding commit yet found > It can take inputs as: > # First commit hash to start excluding commits from history > # Fix Version > # JIRA Project Name > # Path of project's working dir > # Jira server url -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18125) Utility to identify git commit / Jira fixVersion discrepancies for RC preparation
[ https://issues.apache.org/jira/browse/HADOOP-18125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18125: --- Fix Version/s: 3.4.0 > Utility to identify git commit / Jira fixVersion discrepancies for RC > preparation > - > > Key: HADOOP-18125 > URL: https://issues.apache.org/jira/browse/HADOOP-18125 > Project: Hadoop Common > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 2.10.2, 3.2.3, 3.3.3 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > As part of RC preparation, we need to identify all git commits that landed > on release branch, however their corresponding Jira is either not resolved > yet or does not contain expected fixVersions. Only when we have git commits > and corresponding Jiras with expected fixVersion resolved, we get all such > Jiras included in auto-generated CHANGES.md as per Yetus changelog generator. > Proposal of this Jira is to provide such script that can be useful for all > upcoming RC preparations and list down all Jiras where we need manual > intervention. This utility script should use Jira API to retrieve individual > fields and use git log to loop through commit history. > The script should identify these issues: > # commit is reverted as per commit message > # commit does not contain Jira number format (e.g. HADOOP- / HDFS- > etc) in message > # Jira does not have expected fixVersion > # Jira has expected fixVersion, but it is not yet resolved > # Jira has release corresponding fixVersion and is resolved, but no > corresponding commit yet found > It can take inputs as: > # First commit hash to start excluding commits from history > # Fix Version > # JIRA Project Name > # Path of project's working dir > # Jira server url -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17551) Upgrade maven-site-plugin to 3.11.0
[ https://issues.apache.org/jira/browse/HADOOP-17551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-17551: --- Summary: Upgrade maven-site-plugin to 3.11.0 (was: Upgrade maven-site-plugin to 3.9.1) > Upgrade maven-site-plugin to 3.11.0 > --- > > Key: HADOOP-17551 > URL: https://issues.apache.org/jira/browse/HADOOP-17551 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Assignee: Ashutosh Gupta >Priority: Major > Labels: newbie, pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > After upgrading maven-site-plugin, error messages will be more detailed and > it will help debugging. > > Ref: > https://issues.apache.org/jira/browse/YARN-10656?focusedCommentId=17291846=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17291846 -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18200) Update ZooKeeper to 3.6.4 when released
[ https://issues.apache.org/jira/browse/HADOOP-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18200: --- Description: Regarding ZOOKEEPER-4455, ZooKeeper 3.6.4 uses reload4j instead of log4j 1.x. Let's upgrade when released. (was: The official document (https://zookeeper.apache.org/releases.html) says ZooKeeper 3.5.x will become EoL from 2022-06-01. Let's upgrade to 3.6.x or upper.) > Update ZooKeeper to 3.6.4 when released > --- > > Key: HADOOP-18200 > URL: https://issues.apache.org/jira/browse/HADOOP-18200 > Project: Hadoop Common > Issue Type: Improvement > Components: build, ha >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > > Regarding ZOOKEEPER-4455, ZooKeeper 3.6.4 uses reload4j instead of log4j 1.x. > Let's upgrade when released. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18200) Update ZooKeeper to 3.6.4 when released
[ https://issues.apache.org/jira/browse/HADOOP-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18200: --- Target Version/s: 3.4.0 (was: 3.4.0, 3.3.4) > Update ZooKeeper to 3.6.4 when released > --- > > Key: HADOOP-18200 > URL: https://issues.apache.org/jira/browse/HADOOP-18200 > Project: Hadoop Common > Issue Type: Improvement > Components: build, ha >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > > Regarding ZOOKEEPER-4455, ZooKeeper 3.6.4 uses reload4j instead of log4j 1.x. > Let's upgrade when released. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18200) Update ZooKeeper to 3.6.x or upper
[ https://issues.apache.org/jira/browse/HADOOP-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521563#comment-17521563 ] Akira Ajisaka commented on HADOOP-18200: Sorry, the zookeeper version is already 3.6.3 in HADOOP-17612. Let me change the description to "update to 3.6.4 when released". > Update ZooKeeper to 3.6.x or upper > -- > > Key: HADOOP-18200 > URL: https://issues.apache.org/jira/browse/HADOOP-18200 > Project: Hadoop Common > Issue Type: Improvement > Components: build, ha >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > > The official document (https://zookeeper.apache.org/releases.html) says > ZooKeeper 3.5.x will become EoL from 2022-06-01. Let's upgrade to 3.6.x or > upper. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18200) Update ZooKeeper to 3.6.4 when released
[ https://issues.apache.org/jira/browse/HADOOP-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18200: --- Summary: Update ZooKeeper to 3.6.4 when released (was: Update ZooKeeper to 3.6.x or upper) > Update ZooKeeper to 3.6.4 when released > --- > > Key: HADOOP-18200 > URL: https://issues.apache.org/jira/browse/HADOOP-18200 > Project: Hadoop Common > Issue Type: Improvement > Components: build, ha >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > > The official document (https://zookeeper.apache.org/releases.html) says > ZooKeeper 3.5.x will become EoL from 2022-06-01. Let's upgrade to 3.6.x or > upper. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18200) Update ZooKeeper to 3.6.x or upper
[ https://issues.apache.org/jira/browse/HADOOP-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521562#comment-17521562 ] Akira Ajisaka commented on HADOOP-18200: Note that ZooKeeper 3.6.4 will use reload4j, but the version is not released as of now. I'll create a PR to upgrade to 3.6.3. That makes easier for us to upgrade to 3.6.4 when it is released. > Update ZooKeeper to 3.6.x or upper > -- > > Key: HADOOP-18200 > URL: https://issues.apache.org/jira/browse/HADOOP-18200 > Project: Hadoop Common > Issue Type: Improvement > Components: build, ha >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > > The official document (https://zookeeper.apache.org/releases.html) says > ZooKeeper 3.5.x will become EoL from 2022-06-01. Let's upgrade to 3.6.x or > upper. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18200) Update ZooKeeper to 3.6.x or upper
[ https://issues.apache.org/jira/browse/HADOOP-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-18200: -- Assignee: Akira Ajisaka > Update ZooKeeper to 3.6.x or upper > -- > > Key: HADOOP-18200 > URL: https://issues.apache.org/jira/browse/HADOOP-18200 > Project: Hadoop Common > Issue Type: Improvement > Components: build, ha >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > > The official document (https://zookeeper.apache.org/releases.html) says > ZooKeeper 3.5.x will become EoL from 2022-06-01. Let's upgrade to 3.6.x or > upper. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18200) Update ZooKeeper to 3.6.x or upper
Akira Ajisaka created HADOOP-18200: -- Summary: Update ZooKeeper to 3.6.x or upper Key: HADOOP-18200 URL: https://issues.apache.org/jira/browse/HADOOP-18200 Project: Hadoop Common Issue Type: Improvement Components: build, ha Reporter: Akira Ajisaka The official document (https://zookeeper.apache.org/releases.html) says ZooKeeper 3.5.x will become EoL from 2022-06-01. Let's upgrade to 3.6.x or upper. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18178) Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2
[ https://issues.apache.org/jira/browse/HADOOP-18178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-18178. Fix Version/s: 3.3.3 Resolution: Fixed Backported to branch-3.3. > Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2 > -- > > Key: HADOOP-18178 > URL: https://issues.apache.org/jira/browse/HADOOP-18178 > Project: Hadoop Common > Issue Type: Bug >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.3 > > Time Spent: 3h > Remaining Estimate: 0h > > https://github.com/FasterXML/jackson-databind/issues/2816 -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18188) Support touch command for directory
[ https://issues.apache.org/jira/browse/HADOOP-18188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18188: --- Fix Version/s: 3.4.0 Resolution: Fixed Status: Resolved (was: Patch Available) Merged the PR into trunk. Thank you [~vjasani] for your contribution. > Support touch command for directory > --- > > Key: HADOOP-18188 > URL: https://issues.apache.org/jira/browse/HADOOP-18188 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Currently hadoop fs -touch command cannot update the mtime and the atime of > directory. The feature would be useful when we check whether the filesystem > is ready to write or not without creating any file. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18178) Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2
[ https://issues.apache.org/jira/browse/HADOOP-18178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18178: --- Target Version/s: 3.4.0, 3.3.3 > Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2 > -- > > Key: HADOOP-18178 > URL: https://issues.apache.org/jira/browse/HADOOP-18178 > Project: Hadoop Common > Issue Type: Bug >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > https://github.com/FasterXML/jackson-databind/issues/2816 -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18178) Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2
[ https://issues.apache.org/jira/browse/HADOOP-18178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518647#comment-17518647 ] Akira Ajisaka commented on HADOOP-18178: Opened [https://github.com/apache/hadoop/pull/4147] for branch-3.3 > Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2 > -- > > Key: HADOOP-18178 > URL: https://issues.apache.org/jira/browse/HADOOP-18178 > Project: Hadoop Common > Issue Type: Bug >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > https://github.com/FasterXML/jackson-databind/issues/2816 -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18178) Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2
[ https://issues.apache.org/jira/browse/HADOOP-18178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18178: --- Fix Version/s: 3.4.0 Merged the PR into trunk. > Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2 > -- > > Key: HADOOP-18178 > URL: https://issues.apache.org/jira/browse/HADOOP-18178 > Project: Hadoop Common > Issue Type: Bug >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h > Remaining Estimate: 0h > > https://github.com/FasterXML/jackson-databind/issues/2816 -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18178) Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2
[ https://issues.apache.org/jira/browse/HADOOP-18178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18178: --- Summary: Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2 (was: upgrade jackson-databind to 2.13.2.2 due to cve) > Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2 > -- > > Key: HADOOP-18178 > URL: https://issues.apache.org/jira/browse/HADOOP-18178 > Project: Hadoop Common > Issue Type: Bug >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > https://github.com/FasterXML/jackson-databind/issues/2816 -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18178) upgrade jackson-databind to 2.13.2.2 due to cve
[ https://issues.apache.org/jira/browse/HADOOP-18178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518635#comment-17518635 ] Akira Ajisaka commented on HADOOP-18178: The upgrade fixes https://nvd.nist.gov/vuln/detail/CVE-2020-36518 > upgrade jackson-databind to 2.13.2.2 due to cve > --- > > Key: HADOOP-18178 > URL: https://issues.apache.org/jira/browse/HADOOP-18178 > Project: Hadoop Common > Issue Type: Bug >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > https://github.com/FasterXML/jackson-databind/issues/2816 -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18178) upgrade jackson-databind to 2.13.2.2 due to cve
[ https://issues.apache.org/jira/browse/HADOOP-18178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-18178: -- Assignee: PJ Fanning > upgrade jackson-databind to 2.13.2.2 due to cve > --- > > Key: HADOOP-18178 > URL: https://issues.apache.org/jira/browse/HADOOP-18178 > Project: Hadoop Common > Issue Type: Bug >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > https://github.com/FasterXML/jackson-databind/issues/2816 -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18188) Support touch command for directory
[ https://issues.apache.org/jira/browse/HADOOP-18188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-18188: --- Release Note: Before this improvement, "hadoop fs -touch" command threw PathIsDirectoryException for directory. Now the command supports directory and will not throw that exception. > Support touch command for directory > --- > > Key: HADOOP-18188 > URL: https://issues.apache.org/jira/browse/HADOOP-18188 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Currently hadoop fs -touch command cannot update the mtime and the atime of > directory. The feature would be useful when we check whether the filesystem > is ready to write or not without creating any file. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18188) Support touch command for directory
[
https://issues.apache.org/jira/browse/HADOOP-18188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17517828#comment-17517828
]
Akira Ajisaka commented on HADOOP-18188:
{quote}I mean if some application expects PathIsDirectoryException to be thrown
for touch command used with directory, from that viewpoint this behaviour
change might be treated as incompatible?
{quote}
Thank you [~vjasani] for your comment. It makes sense to me, so I will add a
release note to warn the users. Anyway I'm not a fan of adding a new option
because the option is not consistent with the touch command in POSIX.
> Support touch command for directory
> ---
>
> Key: HADOOP-18188
> URL: https://issues.apache.org/jira/browse/HADOOP-18188
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Akira Ajisaka
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently hadoop fs -touch command cannot update the mtime and the atime of
> directory. The feature would be useful when we check whether the filesystem
> is ready to write or not without creating any file.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
