I was thinking we provided the blockId, blockPoolId, path thats enough for the client to get an ExtendedBlock and they can do the rest?
Agree on the snapshot operations for Inotify. I'll create a jira ticket. thanks for the feedback rahul On Tue, Jul 5, 2016 at 11:36 AM, Colin McCabe <cmcc...@apache.org> wrote: > I think it makes sense to have an AddBlockEvent. It seems like we could > provide something like the block ID, block pool ID, and genstamp, as > well as the inode ID and path of the file which the block was added to. > Clearly, we cannot provide the length, since we don't know how many > bytes the client will write. Would you mind filing a JIRA for this? > > We should also set up inotify events for the snapshot operations at some > point as well. > > regards, > Colin > > On Fri, Jul 1, 2016, at 10:28, rahul gidwani wrote: > > Hello kind folks, > > > > I was wondering if anyone would be interested in adding a AddBlock Event > > to > > the inotify pipeline? We were thinking about using Inotify for hdfs > > replication to another datacenter. > > > > Right now the problem is with the appends. We get a notification when an > > append starts with an AppendEvent and then we know the file is complete > > when we get a CloseEvent. This would increase our latency considerably. > > > > So if we could add an AddBlockEvent whenever we get an OP_ADD_BLOCK that > > would give us the ability to ship blocks. > > > > This would basically give us an ExtendedBlock and then we could ask the > > namenode to give us the corresponding LocatedBlock which we could just > > get > > a stream of bytes ready to ship to our destination cluster. > > > > Is this something the community would be interested in? > > > > Thank you > > rahul > > --------------------------------------------------------------------- > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > >
