[jira] [Commented] (HDFS-14997) BPServiceActor processes commands from NameNode asynchronously
[
https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181563#comment-17181563
]
zy.jordan commented on HDFS-14997:
--
Hi ,[~hexiaoqiao]
>From this code, it looks like run process cmd async instead of in heartbeat
>thread. It looks like perfect.
But look insight, I found heartbeat thread run
bpos.updateActorStatesFromHeartbeat() func, and updateActorStatesFromHeartbeat
held write lock()
{code:java}
private void offerService() throws Exception {
...
while (shouldRun()) {
...
bpos.updateActorStatesFromHeartbeat(this, resp.getNameNodeHaState());
...
}
...
}{code}
{code:java}
void updateActorStatesFromHeartbeat( BPServiceActor actor, NNHAStatusHeartbeat
nnHaState) {
writeLock();
try {
...
} finally {
writeUnlock();
}
...
}
{code}
And in async process cmd thread, proces cmd func also held the same lock
{code:java}
boolean processCommandFromActor(DatanodeCommand cmd, BPServiceActor actor)
throws IOException {
...
writeLock();
try {
...
} finally {
writeUnlock();
}
}
{code}
On my view, the process cmd looks like not async
> BPServiceActor processes commands from NameNode asynchronously
> --
>
> Key: HDFS-14997
> URL: https://issues.apache.org/jira/browse/HDFS-14997
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Reporter: Xiaoqiao He
>Assignee: Xiaoqiao He
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HDFS-14997.001.patch, HDFS-14997.002.patch,
> HDFS-14997.003.patch, HDFS-14997.004.patch, HDFS-14997.005.patch,
> HDFS-14997.addendum.patch, image-2019-12-26-16-15-44-814.png
>
>
> There are two core functions, report(#sendHeartbeat, #blockReport,
> #cacheReport) and #processCommand in #BPServiceActor main process flow. If
> processCommand cost long time it will block send report flow. Meanwhile
> processCommand could cost long time(over 1000s the worst case I meet) when IO
> load of DataNode is very high. Since some IO operations are under
> #datasetLock, So it has to wait to acquire #datasetLock long time when
> process some of commands(such as #DNA_INVALIDATE). In such case, #heartbeat
> will not send to NameNode in-time, and trigger other disasters.
> I propose to improve #processCommand asynchronously and not block
> #BPServiceActor to send heartbeat back to NameNode when meet high IO load.
> Notes:
> 1. Lifeline could be one effective solution, however some old branches are
> not support this feature.
> 2. IO operations under #datasetLock is another issue, I think we should solve
> it at another JIRA.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15484) Add new method batchRename for DistributedFileSystem and WebHdfsFileSystem
[
https://issues.apache.org/jira/browse/HDFS-15484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181556#comment-17181556
]
Yang Yun commented on HDFS-15484:
-
Thanks [~ste...@apache.org] for the instruction.
I create a PR,
[https://github.com/apache/hadoop/pull/2235/]
> Add new method batchRename for DistributedFileSystem and WebHdfsFileSystem
> --
>
> Key: HDFS-15484
> URL: https://issues.apache.org/jira/browse/HDFS-15484
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: dfsclient, namenode, performance
>Affects Versions: 3.3.0
>Reporter: Yang Yun
>Assignee: Yang Yun
>Priority: Minor
> Attachments: HDFS-15484.001.patch, HDFS-15484.002.patch,
> HDFS-15484.003.patch
>
>
> Sometime we need rename many files after a task, add a new option in enum
> Rename to support batch rename, which only need one RPC and one lock. For
> example,
> rename(new Path("/dir1/f1::/dir2/f2"), new Path("/dir3/f1::dir4/f4"),
> Rename.BATCH)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15290) NPE in HttpServer during NameNode startup
[ https://issues.apache.org/jira/browse/HDFS-15290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181494#comment-17181494 ] Chen Liang commented on HDFS-15290: --- I have committed v03 patch to trunk, branch-3.x. Thanks for the contribution [~simbadzina]! There is a conflict when backporting to branch-2.10 though, due to log4j usage. Mind providing a version for branch-2.10? > NPE in HttpServer during NameNode startup > - > > Key: HDFS-15290 > URL: https://issues.apache.org/jira/browse/HDFS-15290 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 2.10.0, 2.7.8, 3.3.0 >Reporter: Konstantin Shvachko >Assignee: Simbarashe Dzinamarira >Priority: Major > Attachments: HDFS-15290.001.patch, HDFS-15290.002.patch, > HDFS-15290.003.patch > > > When NameNode starts it first starts HttpServer, then starts loading fsImage > and edits. While loading the namesystem field in NameNode is null. I saw that > a StandbyNode sends a checkpoint request, which fails with NPE because > NNStorage is not instantiated yet. > We should check the NameNode startup status before accepting checkpoint > requests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15438) Setting dfs.disk.balancer.max.disk.errors = 0 will fail the block copy
[
https://issues.apache.org/jira/browse/HDFS-15438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181417#comment-17181417
]
Hadoop QA commented on HDFS-15438:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 49s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m
59s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
55s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 35s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}124m 50s{color}
| {color:red} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
|
[jira] [Commented] (HDFS-15535) RBF: Fix Namespace path to snapshot path resolution for snapshot API
[
https://issues.apache.org/jira/browse/HDFS-15535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181374#comment-17181374
]
Ayush Saxena commented on HDFS-15535:
-
Committed to trunk.
Thanx [~elgoiri] for the review!!!
> RBF: Fix Namespace path to snapshot path resolution for snapshot API
>
>
> Key: HDFS-15535
> URL: https://issues.apache.org/jira/browse/HDFS-15535
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Attachments: HDFS-15535-01.patch, HDFS-15535-02.patch,
> HDFS-15535-03.patch, HDFS-15535-04.patch, HDFS-15535-05.patch,
> HDFS-15535-06.patch, HDFS-15535-07.patch, HDFS-15535-08.patch
>
>
> Presently, after invoking {{createSnapshot}} and {{getSnapshotListing}}, the
> namespace path is replaced with mount path.
> This presumes as of now that, the invokedLocation shall always be the first
> one in the sequence, but there are multiple reasons, where the directory
> might not be in the first location.
> So, rather than replacing using firstLocation, we should replace path using
> actual invoked Location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15535) RBF: Fix Namespace path to snapshot path resolution for snapshot API
[
https://issues.apache.org/jira/browse/HDFS-15535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HDFS-15535:
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> RBF: Fix Namespace path to snapshot path resolution for snapshot API
>
>
> Key: HDFS-15535
> URL: https://issues.apache.org/jira/browse/HDFS-15535
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: HDFS-15535-01.patch, HDFS-15535-02.patch,
> HDFS-15535-03.patch, HDFS-15535-04.patch, HDFS-15535-05.patch,
> HDFS-15535-06.patch, HDFS-15535-07.patch, HDFS-15535-08.patch
>
>
> Presently, after invoking {{createSnapshot}} and {{getSnapshotListing}}, the
> namespace path is replaced with mount path.
> This presumes as of now that, the invokedLocation shall always be the first
> one in the sequence, but there are multiple reasons, where the directory
> might not be in the first location.
> So, rather than replacing using firstLocation, we should replace path using
> actual invoked Location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15536) RBF: Clear Quota in Router was not consistent
[
https://issues.apache.org/jira/browse/HDFS-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181361#comment-17181361
]
Hemanth Boyina commented on HDFS-15536:
---
i think MountTable#equals should also validate the nsQuota and ssQuota as
MountTableResolver#refreshEntries will check for updates of mount table through
equals method
{code:java}
@Override
public boolean equals(Object obj) {
if (obj instanceof MountTable) {
MountTable other = (MountTable)obj;
return new EqualsBuilder()
.append(this.getSourcePath(), other.getSourcePath())
.append(this.getDestinations(), other.getDestinations())
.append(this.isReadOnly(), other.isReadOnly())
.append(this.getDestOrder(), other.getDestOrder())
.append(this.isFaultTolerant(), other.isFaultTolerant())
.isEquals();
}
return false;
} {code}
> RBF: Clear Quota in Router was not consistent
> ---
>
> Key: HDFS-15536
> URL: https://issues.apache.org/jira/browse/HDFS-15536
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Critical
> Attachments: HDFS-15536.testrepro.patch
>
>
> *)create a mount point
> *) set quota for mount point through dfsrouteradmin
> *) clear quota for the same mount point through dfsrouteradmin
> check the content summary of mount point , quota was not cleared , though the
> mount table store has the quota cleared
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15536) RBF: Clear Quota in Router was not consistent
[ https://issues.apache.org/jira/browse/HDFS-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemanth Boyina updated HDFS-15536: -- Attachment: HDFS-15536.testrepro.patch > RBF: Clear Quota in Router was not consistent > --- > > Key: HDFS-15536 > URL: https://issues.apache.org/jira/browse/HDFS-15536 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Hemanth Boyina >Assignee: Hemanth Boyina >Priority: Critical > Attachments: HDFS-15536.testrepro.patch > > > *)create a mount point > *) set quota for mount point through dfsrouteradmin > *) clear quota for the same mount point through dfsrouteradmin > check the content summary of mount point , quota was not cleared , though the > mount table store has the quota cleared -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15536) RBF: Clear Quota in Router was not consistent
Hemanth Boyina created HDFS-15536: - Summary: RBF: Clear Quota in Router was not consistent Key: HDFS-15536 URL: https://issues.apache.org/jira/browse/HDFS-15536 Project: Hadoop HDFS Issue Type: Bug Reporter: Hemanth Boyina Assignee: Hemanth Boyina *)create a mount point *) set quota for mount point through dfsrouteradmin *) clear quota for the same mount point through dfsrouteradmin check the content summary of mount point , quota was not cleared , though the mount table store has the quota cleared -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15510) RBF: Quota and Content Summary was not correct in Multiple Destinations
[
https://issues.apache.org/jira/browse/HDFS-15510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181323#comment-17181323
]
Hemanth Boyina commented on HDFS-15510:
---
thanks for your analysis and for your comment [~ayushtkn]
{quote}create a directory {{/router_test/dir}} and then set quota on it, If
there are two namespaces, the {{setQuota}} will be a concurrent call and set on
both namespaces, The point is with #3 in count command the quota will shows
correct, but will it throw Quota exceeded exception at the value? I tried a
quick UT and it didn't. since for the namespace directories the quota is
maintained at router
{quote}
it wont throw quota exceeded exception here , because router has quota
validation for only mount points as this directory {{/router_test/dir}} is not
a mount point there wont be any validation in router
{quote}{color:#172b4d}The present code, overlooks the overflow of
{color}{{LONG}}{color:#172b4d} due to addition of quota, set nsQuota to
{color}{{LONG.MAX-2}}{color:#172b4d} in the current code and this goes
wrong.{color}{quote}
yes it goes wrong , as this happens while aggregation in content summary ,
though in mount table store we have only the value which is set without any
modifications
{quote}Can you check once, if for directories quota is protected, not just
{{mount entries}}
please feel free to ignore and move ahead, if others are happy
{quote}
as this jira is mainly pointed about Quota for Mount points , i think we can go
ahead and we can raise a new jira for quota in router for non mount points
> RBF: Quota and Content Summary was not correct in Multiple Destinations
> ---
>
> Key: HDFS-15510
> URL: https://issues.apache.org/jira/browse/HDFS-15510
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Critical
> Attachments: 15510.png, HDFS-15510.001.patch, HDFS-15510.002.patch
>
>
> steps :
> *) create a mount entry with multiple destinations ( for suppose 2)
> *) Set NS quota as 10 for mount entry by dfsrouteradmin command, Content
> Summary on the Mount Entry shows NS quota as 20
> *) Create 10 files through router, on creating 11th file , NS Quota Exceeded
> Exception is coming
> though the Content Summary showing the NS quota as 20 , we are not able to
> create 20 files
>
> the problem here is router stores the mount entry's NS quota as 10 , but
> invokes NS quota on both the name services by set NS quota as 10 , so content
> summary on mount entry aggregates the content summary of both the name
> services by making NS quota as 20
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14504) Rename with Snapshots does not honor quota limit
[
https://issues.apache.org/jira/browse/HDFS-14504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181309#comment-17181309
]
Hadoop QA commented on HDFS-14504:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 23s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m
19s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
17s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
2s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 54s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}124m 46s{color}
| {color:red} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
|
[jira] [Commented] (HDFS-15535) RBF: Fix Namespace path to snapshot path resolution for snapshot API
[
https://issues.apache.org/jira/browse/HDFS-15535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181306#comment-17181306
]
Íñigo Goiri commented on HDFS-15535:
+1 on [^HDFS-15535-08.patch].
> RBF: Fix Namespace path to snapshot path resolution for snapshot API
>
>
> Key: HDFS-15535
> URL: https://issues.apache.org/jira/browse/HDFS-15535
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Attachments: HDFS-15535-01.patch, HDFS-15535-02.patch,
> HDFS-15535-03.patch, HDFS-15535-04.patch, HDFS-15535-05.patch,
> HDFS-15535-06.patch, HDFS-15535-07.patch, HDFS-15535-08.patch
>
>
> Presently, after invoking {{createSnapshot}} and {{getSnapshotListing}}, the
> namespace path is replaced with mount path.
> This presumes as of now that, the invokedLocation shall always be the first
> one in the sequence, but there are multiple reasons, where the directory
> might not be in the first location.
> So, rather than replacing using firstLocation, we should replace path using
> actual invoked Location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15484) Add new method batchRename for DistributedFileSystem and WebHdfsFileSystem
[
https://issues.apache.org/jira/browse/HDFS-15484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181271#comment-17181271
]
Steve Loughran commented on HDFS-15484:
---
github PR should be a matter of
-push the branch to your own github repo
-from the github UI, navigate to that branch and create a PR against the
apache/hadoop repo
> Add new method batchRename for DistributedFileSystem and WebHdfsFileSystem
> --
>
> Key: HDFS-15484
> URL: https://issues.apache.org/jira/browse/HDFS-15484
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: dfsclient, namenode, performance
>Affects Versions: 3.3.0
>Reporter: Yang Yun
>Assignee: Yang Yun
>Priority: Minor
> Attachments: HDFS-15484.001.patch, HDFS-15484.002.patch,
> HDFS-15484.003.patch
>
>
> Sometime we need rename many files after a task, add a new option in enum
> Rename to support batch rename, which only need one RPC and one lock. For
> example,
> rename(new Path("/dir1/f1::/dir2/f2"), new Path("/dir3/f1::dir4/f4"),
> Rename.BATCH)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14504) Rename with Snapshots does not honor quota limit
[
https://issues.apache.org/jira/browse/HDFS-14504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181161#comment-17181161
]
Hadoop QA commented on HDFS-14504:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 25m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 50s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m
59s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
57s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 3s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}107m 52s{color}
| {color:red} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
|
[jira] [Commented] (HDFS-14852) Remove of LowRedundancyBlocks do NOT remove the block from all queues
[
https://issues.apache.org/jira/browse/HDFS-14852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181058#comment-17181058
]
Hadoop QA commented on HDFS-14852:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
46s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 33m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 39s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
39s{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m
59s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 9s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} the patch passed with JDK Private
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 95m 53s{color}
| {color:red} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
|
