[jira] [Updated] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HDFS-10608: -- Status: Open (was: Patch Available) > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch, HDFS-10608.v3.patch, HDFS-10608.v4.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales resolved HDFS-10608. --- Resolution: Won't Fix looks like nobody is interested in this patch, so ill just close this off. > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch, HDFS-10608.v3.patch, HDFS-10608.v4.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11218) Add option to skip open files during HDFS Snapshots
[
https://issues.apache.org/jira/browse/HDFS-11218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796535#comment-15796535
]
churro morales commented on HDFS-11218:
---
This seems quite useful. Are you guys working on this patch currently?
> Add option to skip open files during HDFS Snapshots
> ---
>
> Key: HDFS-11218
> URL: https://issues.apache.org/jira/browse/HDFS-11218
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: snapshots
>Affects Versions: 3.0.0-alpha1
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
>
> *Problem:*
> When there are files being written and when HDFS Snapshots are taken in
> parallel, Snapshots do capture all these files, but these being written
> files in Snapshots do not have the point-in-time file length captured.
> At the time of File close or any other meta data modification operation on
> that file which was previously open, HDFS reconciles the file length and
> records the modification in the last taken Snapshot. All the previously taken
> Snapshots continue to have the same open File with no modification recorded.
> So, all those previous snapshots end up using the final modification record
> in the next available snapshot.
> *Proposal:*
> HDFS Snapshot Design goal was to have O(M) space usage for Snapshots, where M
> is the number file modifications. So, it would very expensive to record
> modifications for all the open files in all the snapshots. For applications
> that do not want to capture incomplete / partial being written binary files
> in the snapshots, it would be preferable to have an extra option to skip open
> files. This way, they don't have to worry about restoring inconsistent files
> from the snapshots.
> {noformat}
> hdfs dfs -createSnapshot -skipOpenFiles
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504011#comment-15504011 ] churro morales commented on HDFS-10608: --- [~cmccabe] are you good with the patch in its current form? I would really like to try to get this upstream if possible. Thank you for taking a look. > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch, HDFS-10608.v3.patch, HDFS-10608.v4.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469122#comment-15469122 ] churro morales commented on HDFS-10608: --- [~surendrasingh] just a question about why you wanted blk_ instead of the id itself for this event? I was just wondering the reasoning behind your desire to have a string instead of a long? The latest patch incorporates your comments and makes it a string. Thanks again for reviewing. > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch, HDFS-10608.v3.patch, HDFS-10608.v4.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HDFS-10608: -- Status: Patch Available (was: Open) > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch, HDFS-10608.v3.patch, HDFS-10608.v4.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HDFS-10608: -- Status: Open (was: Patch Available) > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch, HDFS-10608.v3.patch, HDFS-10608.v4.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HDFS-10608: -- Attachment: HDFS-10608.v4.patch [~surendrasingh] Removed the lastBlockLength and converted the blockIds to strings as you requested. Now use simply prepend the blockID with Block.BLOCK_FILE_PREFIX. Hope this is what you wanted and thanks for the review. Let me know if this is in a state to get upstream now. > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch, HDFS-10608.v3.patch, HDFS-10608.v4.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455915#comment-15455915 ] churro morales commented on HDFS-10608: --- [~surendrasingh] Thank you for the review. You are right, size is not necessary for last block (don't know what I was thinking). Will remove it. For (2). I'm not sure I completely follow you, did you want me to switch blockID to a string for both penultimate and last blocks? Many of the API's use a long instead of a string to create Blocks from Id's. I just want to make sure I understand what you want before I submit my next patch. Thanks again. > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch, HDFS-10608.v3.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433618#comment-15433618 ] churro morales commented on HDFS-10608: --- looked over tests that failed this time around, both tests passed locally and seem to be unrelated to this patch. [~surendrasingh] how does it look to you? Thanks again for spending the time and reviewing it. > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch, HDFS-10608.v3.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HDFS-10608: -- Status: Open (was: Patch Available) > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch, HDFS-10608.v3.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HDFS-10608: -- Status: Patch Available (was: Open) > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch, HDFS-10608.v3.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HDFS-10608: -- Attachment: HDFS-10608.v3.patch > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch, HDFS-10608.v3.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433319#comment-15433319 ] churro morales commented on HDFS-10608: --- TestDataNodeLifeline and TestDataNodeHotSwapVolumes pass for me locally. I did break TestDFSUpgradeFromImage, which I fixed. I'll upload the patch which fixes this issue. > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HDFS-10608: -- Status: Patch Available (was: Open) > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HDFS-10608: -- Attachment: HDFS-10608.v2.patch Removed the blockPoolId field and no more exposing Blocks from the AddBlockEvent API. > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HDFS-10608: -- Status: Open (was: Patch Available) > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch, > HDFS-10608.v2.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15428602#comment-15428602 ] churro morales commented on HDFS-10608: --- [~surendrasingh] I will go ahead and add those fields to the AddBlockEvent you suggested [~ajisakaa] I will also remove blockPoolID as a field as it is not needed. Let me know if you think there are any other changes needed. I'll try and get an updated patch up today. > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415608#comment-15415608 ] churro morales commented on HDFS-10608: --- Hi [~surendrasingh] , thanks for the review. Good catch on the Block @IA.Private and evolving. Is that really the case? I haven't seen much change in the Block API in many years. Regardless how about this: I create a new object which has the fields: blockId, genStamp, and numBytes. And that gets exposed instead. Or was wondering if you had any ideas on a more elegant solution. > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414057#comment-15414057 ] churro morales commented on HDFS-10608: --- Hi [~surendrasingh] Thank you for the review. Our original goal for this patch was to use inotify to replicate blocks to a remote data center. Thus when we had a new penultimateBlock we knew it would be complete and could replicate it. The latest block is not ready to be replicated yet, so that is why I added that info. I'll fix items 2,3 and 4 no problem. Are you okay with keeping penultimateBlock in the event? > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HDFS-10608: -- Attachment: HDFS-10608.v1.patch [~surendrasingh] here is the new rebased patch. > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch, HDFS-10608.v1.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412248#comment-15412248 ] churro morales commented on HDFS-10608: --- [~surendrasingh] sure thing, sorry for getting back late to you - was out of town. I will put up a new patch. > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15390235#comment-15390235 ] churro morales commented on HDFS-10608: --- Would love a review if someone has some spare cycles. I think this could be a really useful event to add to the inotify event pipeline. > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371546#comment-15371546 ] churro morales edited comment on HDFS-10608 at 7/11/16 8:29 PM: Here is a patch against trunk, if this looks good to everyone I can provide backports. [~cmccabe] would you mind taking a look to see if this is sufficient. Thanks was (Author: churromorales): Here is a patch against trunk, if this looks good to everyone I can provide backports. [~colinmccabe] would you mind taking a look to see if this is sufficient. Thanks > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
[ https://issues.apache.org/jira/browse/HDFS-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HDFS-10608: -- Attachment: HDFS-10608.patch Here is a patch against trunk, if this looks good to everyone I can provide backports. [~colinmccabe] would you mind taking a look to see if this is sufficient. Thanks > Include event for AddBlock in Inotify Event Stream > -- > > Key: HDFS-10608 > URL: https://issues.apache.org/jira/browse/HDFS-10608 > Project: Hadoop HDFS > Issue Type: Task >Reporter: churro morales >Priority: Minor > Attachments: HDFS-10608.patch > > > It would be nice to have an AddBlockEvent in the INotify pipeline. Based on > discussions from mailing list: > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-10608) Include event for AddBlock in Inotify Event Stream
churro morales created HDFS-10608: - Summary: Include event for AddBlock in Inotify Event Stream Key: HDFS-10608 URL: https://issues.apache.org/jira/browse/HDFS-10608 Project: Hadoop HDFS Issue Type: Task Reporter: churro morales Priority: Minor It would be nice to have an AddBlockEvent in the INotify pipeline. Based on discussions from mailing list: http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201607.mbox/%3C1467743792.4040080.657624289.7BE240AD%40webmail.messagingengine.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
