[jira] [Updated] (HDFS-12217) HDFS snapshots doesn't capture all open files when one of the open files is deleted
[
https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated HDFS-12217:
--
Fix Version/s: (was: 2.9.0)
> HDFS snapshots doesn't capture all open files when one of the open files is
> deleted
> ---
>
> Key: HDFS-12217
> URL: https://issues.apache.org/jira/browse/HDFS-12217
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-alpha1
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
> Fix For: 3.0.0-beta1
>
> Attachments: HDFS-12217.01.patch, HDFS-12217.02.patch,
> HDFS-12217.03.patch, HDFS-12217.04.patch, HDFS-12217.05.patch
>
>
> With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
> open files. Just like all other files, these open files in the snapshots will
> remain immutable. But, sometimes it is found that snapshots fail to capture
> all the open files in the system.
> Under the following conditions, LeaseManager will fail to find INode
> corresponding to an active lease
> * a file is opened for writing (LeaseManager allots a lease), and
> * the same file is deleted while it is still open for writing and having
> active lease, and
> * the same file is not referenced in any other Snapshots/Trash
> {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
> leases there by causing the caller to trip over and not return all the open
> files needed by the snapshot manager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12217) HDFS snapshots doesn't capture all open files when one of the open files is deleted
[
https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-12217:
--
Target Version/s: 2.9.0, 3.0.0-beta1 (was: 3.0.0-beta1)
Fix Version/s: 3.0.0-beta1
2.9.0
Committed to trunk and branch-2.
Thanks for the review [~jojochuang] and [~yzhangal].
> HDFS snapshots doesn't capture all open files when one of the open files is
> deleted
> ---
>
> Key: HDFS-12217
> URL: https://issues.apache.org/jira/browse/HDFS-12217
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-alpha1
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: HDFS-12217.01.patch, HDFS-12217.02.patch,
> HDFS-12217.03.patch, HDFS-12217.04.patch, HDFS-12217.05.patch
>
>
> With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
> open files. Just like all other files, these open files in the snapshots will
> remain immutable. But, sometimes it is found that snapshots fail to capture
> all the open files in the system.
> Under the following conditions, LeaseManager will fail to find INode
> corresponding to an active lease
> * a file is opened for writing (LeaseManager allots a lease), and
> * the same file is deleted while it is still open for writing and having
> active lease, and
> * the same file is not referenced in any other Snapshots/Trash
> {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
> leases there by causing the caller to trip over and not return all the open
> files needed by the snapshot manager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12217) HDFS snapshots doesn't capture all open files when one of the open files is deleted
[
https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-12217:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> HDFS snapshots doesn't capture all open files when one of the open files is
> deleted
> ---
>
> Key: HDFS-12217
> URL: https://issues.apache.org/jira/browse/HDFS-12217
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-alpha1
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: HDFS-12217.01.patch, HDFS-12217.02.patch,
> HDFS-12217.03.patch, HDFS-12217.04.patch, HDFS-12217.05.patch
>
>
> With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
> open files. Just like all other files, these open files in the snapshots will
> remain immutable. But, sometimes it is found that snapshots fail to capture
> all the open files in the system.
> Under the following conditions, LeaseManager will fail to find INode
> corresponding to an active lease
> * a file is opened for writing (LeaseManager allots a lease), and
> * the same file is deleted while it is still open for writing and having
> active lease, and
> * the same file is not referenced in any other Snapshots/Trash
> {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
> leases there by causing the caller to trip over and not return all the open
> files needed by the snapshot manager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12217) HDFS snapshots doesn't capture all open files when one of the open files is deleted
[
https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-12217:
--
Attachment: HDFS-12217.05.patch
Thanks for the review [~jojochuang]. Attached v05 patch with following comment
addressed. Please take a look.
bq. DirectorySnapshottableFeature#addSnapshot, because an exception could be
thrown from places other than LeaseManager#getINodeWithLeases
Sure. Modified the SnapshotException to accept both message and throwable. This
way, the exception is printed is the full chain instead of the top one. This
exception is also get sent to the Client. When the DEUBG logging is enabled,
the client command can also see the full error exception stack tracer.
> HDFS snapshots doesn't capture all open files when one of the open files is
> deleted
> ---
>
> Key: HDFS-12217
> URL: https://issues.apache.org/jira/browse/HDFS-12217
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-alpha1
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
> Attachments: HDFS-12217.01.patch, HDFS-12217.02.patch,
> HDFS-12217.03.patch, HDFS-12217.04.patch, HDFS-12217.05.patch
>
>
> With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
> open files. Just like all other files, these open files in the snapshots will
> remain immutable. But, sometimes it is found that snapshots fail to capture
> all the open files in the system.
> Under the following conditions, LeaseManager will fail to find INode
> corresponding to an active lease
> * a file is opened for writing (LeaseManager allots a lease), and
> * the same file is deleted while it is still open for writing and having
> active lease, and
> * the same file is not referenced in any other Snapshots/Trash
> {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
> leases there by causing the caller to trip over and not return all the open
> files needed by the snapshot manager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12217) HDFS snapshots doesn't capture all open files when one of the open files is deleted
[
https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-12217:
--
Attachment: HDFS-12217.04.patch
Thanks for the quick review [~jojochuang]. Attached v04 patch to address the
following. Can you please take a look at the latest patch.
bq. LeaseManager#getINodeWithLeases() seems to be a test-only method. If so, it
doesn't need public modifier (package private is sufficient) and we can also
add a {{@VisibleForTesting }} annotation.
Done.
bq. On a separate note, and this is totally unrelated to this patch. It looks
like LeaseManager#addLease assumes the inodeId is an id for an INodeFile, which
makes perfect sense.
Will take this up in a new patch as there are many callers that need to get
refactored for this,.
bq. Note that in your patch, DirectorySnapshottableFeature#addSnapshot would
capture exception, but doesn't log the exception. I also think that in addition
to the snapshot name, you should print the snapshot root path for the error
message.
Done.
-- Exception is already logged at LeaseManager#getINodeWithLeases.
-- Now added exception details to the SnapshotException message also.
> HDFS snapshots doesn't capture all open files when one of the open files is
> deleted
> ---
>
> Key: HDFS-12217
> URL: https://issues.apache.org/jira/browse/HDFS-12217
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-alpha1
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
> Attachments: HDFS-12217.01.patch, HDFS-12217.02.patch,
> HDFS-12217.03.patch, HDFS-12217.04.patch
>
>
> With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
> open files. Just like all other files, these open files in the snapshots will
> remain immutable. But, sometimes it is found that snapshots fail to capture
> all the open files in the system.
> Under the following conditions, LeaseManager will fail to find INode
> corresponding to an active lease
> * a file is opened for writing (LeaseManager allots a lease), and
> * the same file is deleted while it is still open for writing and having
> active lease, and
> * the same file is not referenced in any other Snapshots/Trash
> {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
> leases there by causing the caller to trip over and not return all the open
> files needed by the snapshot manager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12217) HDFS snapshots doesn't capture all open files when one of the open files is deleted
[
https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-12217:
--
Attachment: HDFS-12217.03.patch
Thanks for the quick review [~yzhangal]. Attached v03 patch with additional
logging.
bq. Suggest to put the e as the cause in the exception throwing.
The SnasphotException message goes to the user. It already has the high level
details on what went wrong. Adding IOException details to the client shell
command output might not look good. On the server side, the RPC server handler
already logs the exception. Additionally, now I have added the throwable
details in the warning log at LeaseManager. These details are good for
debugging if any problem arises.
{noformat}
# hdfs dfs -createSnapshot /sub1 sn0
Failed to add snapshot: Unable to capture all open files for the snapshot 'sn0'.
{noformat}
> HDFS snapshots doesn't capture all open files when one of the open files is
> deleted
> ---
>
> Key: HDFS-12217
> URL: https://issues.apache.org/jira/browse/HDFS-12217
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-alpha1
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
> Attachments: HDFS-12217.01.patch, HDFS-12217.02.patch,
> HDFS-12217.03.patch
>
>
> With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
> open files. Just like all other files, these open files in the snapshots will
> remain immutable. But, sometimes it is found that snapshots fail to capture
> all the open files in the system.
> Under the following conditions, LeaseManager will fail to find INode
> corresponding to an active lease
> * a file is opened for writing (LeaseManager allots a lease), and
> * the same file is deleted while it is still open for writing and having
> active lease, and
> * the same file is not referenced in any other Snapshots/Trash
> {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
> leases there by causing the caller to trip over and not return all the open
> files needed by the snapshot manager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12217) HDFS snapshots doesn't capture all open files when one of the open files is deleted
[
https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-12217:
--
Attachment: HDFS-12217.02.patch
Thanks for the detailed review [~yzhangal]. Attached v02 patch with following
addressed. Please take a look at the latest patch.
bq. 1. we can use package wide scope for the isFileDeleted method instead of
protected
Done.
bq. 2. f can't be null in the code, I mean, either an exception will be thrown,
or f will valid value.
True. But, f.get() can be null if the task didn't get to run properly.
Enclosed this code section in try{}catch{} block and now throwing IOException()
on any errors.
bq. 3. If any exception is thrown in the above code, we can log a waning, and
rethrow the exception, because the exception likely indicate a real problem
that we don't expect to happen. If we continue, we might get inconsistent
snapshot.
Makes sense. Now throwing the exception, {{addSnasphot()}} caller handles the
exception and rethrows it as SnaphotException to the upper layer to abort the
snapshot creation.
> HDFS snapshots doesn't capture all open files when one of the open files is
> deleted
> ---
>
> Key: HDFS-12217
> URL: https://issues.apache.org/jira/browse/HDFS-12217
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-alpha1
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
> Attachments: HDFS-12217.01.patch, HDFS-12217.02.patch
>
>
> With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
> open files. Just like all other files, these open files in the snapshots will
> remain immutable. But, sometimes it is found that snapshots fail to capture
> all the open files in the system.
> Under the following conditions, LeaseManager will fail to find INode
> corresponding to an active lease
> * a file is opened for writing (LeaseManager allots a lease), and
> * the same file is deleted while it is still open for writing and having
> active lease, and
> * the same file is not referenced in any other Snapshots/Trash
> {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
> leases there by causing the caller to trip over and not return all the open
> files needed by the snapshot manager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12217) HDFS snapshots doesn't capture all open files when one of the open files is deleted
[
https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-12217:
--
Status: Patch Available (was: Open)
> HDFS snapshots doesn't capture all open files when one of the open files is
> deleted
> ---
>
> Key: HDFS-12217
> URL: https://issues.apache.org/jira/browse/HDFS-12217
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-alpha1
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
> Attachments: HDFS-12217.01.patch
>
>
> With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
> open files. Just like all other files, these open files in the snapshots will
> remain immutable. But, sometimes it is found that snapshots fail to capture
> all the open files in the system.
> Under the following conditions, LeaseManager will fail to find INode
> corresponding to an active lease
> * a file is opened for writing (LeaseManager allots a lease), and
> * the same file is deleted while it is still open for writing and having
> active lease, and
> * the same file is not referenced in any other Snapshots/Trash
> {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
> leases there by causing the caller to trip over and not return all the open
> files needed by the snapshot manager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12217) HDFS snapshots doesn't capture all open files when one of the open files is deleted
[
https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-12217:
--
Attachment: HDFS-12217.01.patch
Attached v01 patch to address the following
1. {{LeaseManager#getINodesWithLease()}} verify if the file is not deleted
before giving out the INodes.
2. Added a new test in {{TestOpenFilesWithSnapshot}} to show how a pending
create / open file deleted could cause problem when taking new snapshots. With
the fix, the test passes through.
[~yzhangal], can you please take a look at the patch ?
> HDFS snapshots doesn't capture all open files when one of the open files is
> deleted
> ---
>
> Key: HDFS-12217
> URL: https://issues.apache.org/jira/browse/HDFS-12217
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-alpha1
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
> Attachments: HDFS-12217.01.patch
>
>
> With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
> open files. Just like all other files, these open files in the snapshots will
> remain immutable. But, sometimes it is found that snapshots fail to capture
> all the open files in the system.
> Under the following conditions, LeaseManager will fail to find INode
> corresponding to an active lease
> * a file is opened for writing (LeaseManager allots a lease), and
> * the same file is deleted while it is still open for writing and having
> active lease, and
> * the same file is not referenced in any other Snapshots/Trash
> {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
> leases there by causing the caller to trip over and not return all the open
> files needed by the snapshot manager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12217) HDFS snapshots doesn't capture all open files when one of the open files is deleted
[
https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-12217:
--
Description:
With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
open files. Just like all other files, these open files in the snapshots will
remain immutable. But, sometimes it is found that snapshots fail to capture all
the open files in the system.
Under the following conditions, LeaseManager will fail to find INode
corresponding to an active lease
* a file is opened for writing (LeaseManager allots a lease), and
* the same file is deleted while it is still open for writing and having active
lease, and
* the same file is not referenced in any other Snapshots/Trash
{{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
leases there by causing the caller to trip over and not return all the open
files needed by the snapshot manager.
was:
With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
open files. Just like all other files, these open files in the snapshots will
remain immutable. But, sometimes it is found that snapshots fail to capture all
the open files in the system.
Under the following conditions, LeaseManager will fail to find INode
corresponding to an active lease
* file is opened for writing (LeaseManager allots a lease), and
* files is deleted while it is still open for writing and active lease, and
* file is not referenced in any other Snapshots/Trash
{{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
leases there by causing the caller to trip over and not return all the open
files needed by the snapshot manager.
> HDFS snapshots doesn't capture all open files when one of the open files is
> deleted
> ---
>
> Key: HDFS-12217
> URL: https://issues.apache.org/jira/browse/HDFS-12217
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-alpha1
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
>
> With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
> open files. Just like all other files, these open files in the snapshots will
> remain immutable. But, sometimes it is found that snapshots fail to capture
> all the open files in the system.
> Under the following conditions, LeaseManager will fail to find INode
> corresponding to an active lease
> * a file is opened for writing (LeaseManager allots a lease), and
> * the same file is deleted while it is still open for writing and having
> active lease, and
> * the same file is not referenced in any other Snapshots/Trash
> {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
> leases there by causing the caller to trip over and not return all the open
> files needed by the snapshot manager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12217) HDFS snapshots doesn't capture all open files when one of the open files is deleted
[
https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-12217:
--
Description:
With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
open files. Just like all other files, these open files in the snapshots will
remain immutable. But, sometimes it is found that snapshots fail to capture all
the open files in the system.
Under the following conditions, LeaseManager will fail to find INode
corresponding to an active lease
* file is opened for writing (LeaseManager allots a lease), and
* files is deleted while it is still open for writing and active lease, and
* file is not referenced in any other Snapshots/Trash
{{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
leases there by causing the caller to trip over and not return all the open
files needed by the snapshot manager.
was:
With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
open files. Just like all other files, these open files in the snapshots will
remain immutable. But, sometimes it is found that snapshots fail to capture all
the open files in the system.
Under the following conditions, LeaseManager will fail to find INode
corresponding to an active lease
-- file is opened for writing (LeaseManager allots a lease)
-- files is deleted while it is still open for writing and active lease
-- file is not referenced in any other Snapshots/Trash
{{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
leases there by causing the caller to trip over and not return all the open
files needed by the snapshot manager.
> HDFS snapshots doesn't capture all open files when one of the open files is
> deleted
> ---
>
> Key: HDFS-12217
> URL: https://issues.apache.org/jira/browse/HDFS-12217
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-alpha1
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
>
> With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
> open files. Just like all other files, these open files in the snapshots will
> remain immutable. But, sometimes it is found that snapshots fail to capture
> all the open files in the system.
> Under the following conditions, LeaseManager will fail to find INode
> corresponding to an active lease
> * file is opened for writing (LeaseManager allots a lease), and
> * files is deleted while it is still open for writing and active lease, and
> * file is not referenced in any other Snapshots/Trash
> {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
> leases there by causing the caller to trip over and not return all the open
> files needed by the snapshot manager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12217) HDFS snapshots doesn't capture all open files when one of the open files is deleted
[
https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-12217:
--
Summary: HDFS snapshots doesn't capture all open files when one of the open
files is deleted (was: HDFS snapshots doesn't capture all open files when one
of open file is deleted)
> HDFS snapshots doesn't capture all open files when one of the open files is
> deleted
> ---
>
> Key: HDFS-12217
> URL: https://issues.apache.org/jira/browse/HDFS-12217
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-alpha1
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
>
> With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
> open files. Just like all other files, these open files in the snapshots will
> remain immutable. But, sometimes it is found that snapshots fail to capture
> all the open files in the system.
> Under the following conditions, LeaseManager will fail to find INode
> corresponding to an active lease
> -- file is opened for writing (LeaseManager allots a lease)
> -- files is deleted while it is still open for writing and active lease
> -- file is not referenced in any other Snapshots/Trash
> {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
> leases there by causing the caller to trip over and not return all the open
> files needed by the snapshot manager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
