[jira] [Commented] (HDFS-15491) Journal transactions batched in sync metrics are missing from Journal Node JMX
[
https://issues.apache.org/jira/browse/HDFS-15491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183713#comment-17183713
]
jianghua zhu commented on HDFS-15491:
-
[~m@hit1994] , can I solve this problem?
> Journal transactions batched in sync metrics are missing from Journal Node
> JMX
>
>
> Key: HDFS-15491
> URL: https://issues.apache.org/jira/browse/HDFS-15491
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: 3.1.1, hdfs, journal-node, qjm
>Affects Versions: 3.1.1
>Reporter: Mohit Saindane
>Priority: Critical
> Labels: newbie, patch, pull-request-available
> Fix For: 3.1.1
>
> Attachments: JournalNode metrics issue.png
>
>
> I have enable the HA in HDP3.1.5 multinode cluster, I was trying to access
> the REST response given by Journal Node on http 8480 port, but as per [Apache
> Hadoop 3.1.1
> Documentation|[https://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-common/Metrics.html#JournalNode]]
> I found that some of the Journal Node specific metrics are missing. (metrics
> like NumTransactionsBatchedInSync_(60,300,3600)_sNumOps and
> NumTransactionsBatchedInSync_Num_s_(50/75/90/95/99)_thPercentileLatencyMicros),
> So checked the _JournalMetrics.java_ code given in
> _hadoop-hdfs-3.1.1.3.1.5.0-152-sources.jar_ as well as __ [Hadoop Open Source
> Code|[https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournalMetrics.java]]
> found that the following code is missing :
>
> @Metric("Journal transactions batched in sync")
> final MutableQuantiles[] numTransactionsBatchedInSync;
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8631) WebHDFS : Support setQuota
[ https://issues.apache.org/jira/browse/HDFS-8631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183680#comment-17183680 ] Uma Maheswara Rao G commented on HDFS-8631: --- Updated back-port patch to 3.2. Same patch to trigger Jenkins. Cherry-pick has conflicts, but HDFS-8631-011.patch can be applied successfully. > WebHDFS : Support setQuota > -- > > Key: HDFS-8631 > URL: https://issues.apache.org/jira/browse/HDFS-8631 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 2.7.2 >Reporter: nijel >Assignee: Chao Sun >Priority: Major > Fix For: 3.3.0 > > Attachments: HDFS-8631-001.patch, HDFS-8631-002.patch, > HDFS-8631-003.patch, HDFS-8631-004.patch, HDFS-8631-005.patch, > HDFS-8631-006.patch, HDFS-8631-007.patch, HDFS-8631-008.patch, > HDFS-8631-009.patch, HDFS-8631-010.patch, HDFS-8631-011.patch, > HDFS-8631-branch-3.2.001.patch > > > User is able do quota management from filesystem object. Same operation can > be allowed trough REST API. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-8631) WebHDFS : Support setQuota
[ https://issues.apache.org/jira/browse/HDFS-8631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-8631: -- Attachment: HDFS-8631-branch-3.2.001.patch > WebHDFS : Support setQuota > -- > > Key: HDFS-8631 > URL: https://issues.apache.org/jira/browse/HDFS-8631 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 2.7.2 >Reporter: nijel >Assignee: Chao Sun >Priority: Major > Fix For: 3.3.0 > > Attachments: HDFS-8631-001.patch, HDFS-8631-002.patch, > HDFS-8631-003.patch, HDFS-8631-004.patch, HDFS-8631-005.patch, > HDFS-8631-006.patch, HDFS-8631-007.patch, HDFS-8631-008.patch, > HDFS-8631-009.patch, HDFS-8631-010.patch, HDFS-8631-011.patch, > HDFS-8631-branch-3.2.001.patch > > > User is able do quota management from filesystem object. Same operation can > be allowed trough REST API. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15536) RBF: Clear Quota in Router was not consistent
[
https://issues.apache.org/jira/browse/HDFS-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183580#comment-17183580
]
Íñigo Goiri commented on HDFS-15536:
Thanks [~hemanthboyina] for the update.
A few style comments:
* Instead of adding the full new admin for everybody, we can create a method
called {{getRouterAdmin()}} only used for this method.
* No need to define addEntry on top, just when we create it.
* destMap could be created with Collections.singletonMap().
* Add a break line before the clear part just to differentiate a little more
and instead of using cs again, create a new one.
> RBF: Clear Quota in Router was not consistent
> ---
>
> Key: HDFS-15536
> URL: https://issues.apache.org/jira/browse/HDFS-15536
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Critical
> Attachments: HDFS-15536.001.patch, HDFS-15536.002.patch,
> HDFS-15536.testrepro.patch
>
>
> *)create a mount point
> *) set quota for mount point through dfsrouteradmin
> *) clear quota for the same mount point through dfsrouteradmin
> check the content summary of mount point , quota was not cleared , though the
> mount table store has the quota cleared
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15536) RBF: Clear Quota in Router was not consistent
[
https://issues.apache.org/jira/browse/HDFS-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183575#comment-17183575
]
Hadoop QA commented on HDFS-15536:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 25m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 54s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
10s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
7s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 30s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 6s{color}
| {color:red} hadoop-hdfs-rbf in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
[jira] [Commented] (HDFS-15536) RBF: Clear Quota in Router was not consistent
[
https://issues.apache.org/jira/browse/HDFS-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183439#comment-17183439
]
Hemanth Boyina commented on HDFS-15536:
---
thanks [~elgoiri] for the review
fixed checkstyle issues and updated the patch
i think test failures are not related , ran test cases in local
{code:java}
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running org.apache.hadoop.hdfs.server.federation.router.TestRouterRpc
[INFO] Tests run: 44, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 37.53 s
- in org.apache.hadoop.hdfs.server.federation.router.TestRouterRpc
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 44, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] BUILD SUCCESS {code}
{code:java}
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running
org.apache.hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination
[INFO] Tests run: 50, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.108
s - in
org.apache.hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 50, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] BUILD SUCCESS {code}
> RBF: Clear Quota in Router was not consistent
> ---
>
> Key: HDFS-15536
> URL: https://issues.apache.org/jira/browse/HDFS-15536
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Critical
> Attachments: HDFS-15536.001.patch, HDFS-15536.002.patch,
> HDFS-15536.testrepro.patch
>
>
> *)create a mount point
> *) set quota for mount point through dfsrouteradmin
> *) clear quota for the same mount point through dfsrouteradmin
> check the content summary of mount point , quota was not cleared , though the
> mount table store has the quota cleared
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15536) RBF: Clear Quota in Router was not consistent
[ https://issues.apache.org/jira/browse/HDFS-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemanth Boyina updated HDFS-15536: -- Attachment: HDFS-15536.002.patch > RBF: Clear Quota in Router was not consistent > --- > > Key: HDFS-15536 > URL: https://issues.apache.org/jira/browse/HDFS-15536 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Hemanth Boyina >Assignee: Hemanth Boyina >Priority: Critical > Attachments: HDFS-15536.001.patch, HDFS-15536.002.patch, > HDFS-15536.testrepro.patch > > > *)create a mount point > *) set quota for mount point through dfsrouteradmin > *) clear quota for the same mount point through dfsrouteradmin > check the content summary of mount point , quota was not cleared , though the > mount table store has the quota cleared -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15448) Remove duplicate BlockPoolManager starting when run DataNode
[
https://issues.apache.org/jira/browse/HDFS-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoqiao He updated HDFS-15448:
---
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
Commit trunk.
Thanks [~jianghuazhu] for your report and contribution.
Thanks [~linyiqun],[~elgoiri] and [~hemanthboyina] for your comments and
reviews.
> Remove duplicate BlockPoolManager starting when run DataNode
>
>
> Key: HDFS-15448
> URL: https://issues.apache.org/jira/browse/HDFS-15448
> Project: Hadoop HDFS
> Issue Type: Improvement
>Affects Versions: 3.1.1
>Reporter: jianghua zhu
>Assignee: jianghua zhu
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: HDFS-15448.001.patch, HDFS-15448.002.patch,
> HDFS-15448.003.patch, method_invoke_path.jpg
>
>
> When starting a DataNode, call BlockPoolManager#startAll() twice.
> The first call:
> BlockPoolManager#doRefreshNamenodes()
> private void doRefreshNamenodes(
> Map> addrMap,
> Map> lifelineAddrMap)
> throws IOException {
> ...
> startAll();
> ...
> }
> The second call:
> DataNode#runDatanodeDaemon()
> public void runDatanodeDaemon() throws IOException {
> blockPoolManager.startAll();
> ...
> }
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15448) Remove duplicate BlockPoolManager starting when run DataNode
[
https://issues.apache.org/jira/browse/HDFS-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoqiao He updated HDFS-15448:
---
Summary: Remove duplicate BlockPoolManager starting when run DataNode
(was: When starting a DataNode, call BlockPoolManager#startAll() twice.)
> Remove duplicate BlockPoolManager starting when run DataNode
>
>
> Key: HDFS-15448
> URL: https://issues.apache.org/jira/browse/HDFS-15448
> Project: Hadoop HDFS
> Issue Type: Improvement
>Affects Versions: 3.1.1
>Reporter: jianghua zhu
>Assignee: jianghua zhu
>Priority: Major
> Attachments: HDFS-15448.001.patch, HDFS-15448.002.patch,
> HDFS-15448.003.patch, method_invoke_path.jpg
>
>
> When starting a DataNode, call BlockPoolManager#startAll() twice.
> The first call:
> BlockPoolManager#doRefreshNamenodes()
> private void doRefreshNamenodes(
> Map> addrMap,
> Map> lifelineAddrMap)
> throws IOException {
> ...
> startAll();
> ...
> }
> The second call:
> DataNode#runDatanodeDaemon()
> public void runDatanodeDaemon() throws IOException {
> blockPoolManager.startAll();
> ...
> }
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Deleted] (HDFS-15537) benjapuk
[ https://issues.apache.org/jira/browse/HDFS-15537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka deleted HDFS-15537: - > benjapuk > > > Key: HDFS-15537 > URL: https://issues.apache.org/jira/browse/HDFS-15537 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: benjapuk >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
