good suggestion.
On Wed, Apr 17, 2013 at 4:10 PM, Harsh J <ha...@cloudera.com> wrote: > Pardon my late inquisition here but since HBase already shipped out > with a name .snapshots/, why do we force them to change it, and not > rename HDFS' snapshots to use .hdfs-snapshots, given that HDFS > Snapshots has not been released for any users yet. The way I see it, > that'd be much more easier to do than making a workaround for a done > deal on HBase, which already has its snapshot's users. > > @Tsz-Wo - If the snapshots in HDFS aren't a 'generic' feature > applicable to other FileSystem interface implementations as well, then > .hdfs-snapshots should be fine for it - no? > > On Wed, Apr 17, 2013 at 2:32 AM, Ted Yu <yuzhih...@gmail.com> wrote: > > Hi, > > Please take a look at patch v5 attached to HBASE-8352. > > > > It would be nice to resolve this blocker today so that 0.94.7 RC can be > cut. > > > > Thanks > > > > On Tue, Apr 16, 2013 at 10:12 AM, lars hofhansl <la...@apache.org> > wrote: > > > >> Please see my last comment on the jira. We can make this work without > >> breaking users who are using HDFS snapshots. > >> > >> ------------------------------ > >> *From:* Ted Yu <yuzhih...@gmail.com> > >> *To:* d...@hbase.apache.org > >> *Cc:* hdfs-dev@hadoop.apache.org; lars hofhansl <la...@apache.org> > >> *Sent:* Tuesday, April 16, 2013 10:00 AM > >> *Subject:* Re: collision in the naming of '.snapshot' directory between > >> hdfs snapshot and hbase snapshot > >> > >> Let's get proper release notes for HBASE-8352 . > >> > >> Either Lars or I can send out notification to user mailing list so that > >> there is enough preparation for this change. > >> > >> Cheers > >> > >> On Tue, Apr 16, 2013 at 8:46 AM, Jonathan Hsieh <j...@cloudera.com> > wrote: > >> > >> I was away from keyboard when I asserted that hdfs snapshot was a hadoop > >> 2.1 or 3.0 feature. Apparently it is targeted as a hadoop 2.0.5 > feature. > >> (I'm a little surprised -- expected this to be a hadoop2 compat > breaking > >> feature) -- so I agree that this is a bit more urgent. > >> > >> Anyway, I agree that the fs .snapshot naming convention is long standing > >> and should win. > >> > >> My concern is with breaking compatibility in 0.94 again -- if we don't > go > >> down the conf variable route, I consider having docs to properly > document > >> how to do the upgrade and caveats of doing the upgrade in the > docs/release > >> notes blocker to hbase 0.94.7. (specifically mentioning from 0.94.6 to > >> 0.94.7, and to possibly to 0.95). > >> > >> Jon. > >> > >> On Mon, Apr 15, 2013 at 9:00 PM, Ted Yu <yuzhih...@gmail.com> wrote: > >> > >> > bq. Alternatively, we can detect the underlying Hadoop version, and > use > >> > either .snapshot or .hbase_snapshot in 0.94 depending on h1 & h2. > >> > > >> > I think this would introduce more confusion, especially for > operations. > >> > > >> > Cheers > >> > > >> > On Mon, Apr 15, 2013 at 8:52 PM, Enis Söztutar <enis....@gmail.com> > >> wrote: > >> > > >> > > Because HDFS exposes the snapshots so that the normal file system > >> > > operations are mapped inside snapshot dirs, I think HDFS reserving > the > >> > > .snapshot name makes sense. OTOH, nothing is specific about the dir > >> name > >> > > that is chosen by HBase. > >> > > > >> > > I would prefer to change the dir name in 0.94 as well, since 0.94 is > >> also > >> > > being run on top of hadoop 2. Alternatively, we can detect the > >> underlying > >> > > Hadoop version, and use either .snapshot or .hbase_snapshot in 0.94 > >> > > depending on h1 & h2. > >> > > > >> > > Enis > >> > > > >> > > > >> > > On Mon, Apr 15, 2013 at 8:31 PM, Ted Yu <yuzhih...@gmail.com> > wrote: > >> > > > >> > > > bq. let's make the hbase snapshot for a conf variable. > >> > > > > >> > > > Once we decide on the new name of snapshot directory, we should > still > >> > use > >> > > > hardcoded value. This aligns with current code base: > >> > > > See this snippet from HConstants: > >> > > > > >> > > > public static final List<String> HBASE_NON_TABLE_DIRS = > >> > > > > >> > > > Collections.unmodifiableList(Arrays.asList(new String[] { > >> > > > HREGION_LOGDIR_NAME, > >> > > > > >> > > > HREGION_OLDLOGDIR_NAME, CORRUPT_DIR_NAME, SPLIT_LOGDIR_NAME, > >> > > > > >> > > > HBCK_SIDELINEDIR_NAME, HFILE_ARCHIVE_DIRECTORY, > >> > SNAPSHOT_DIR_NAME, > >> > > > HBASE_TEMP_DIRECTORY })); > >> > > > Cheers > >> > > > > >> > > > On Mon, Apr 15, 2013 at 8:24 PM, Jonathan Hsieh <j...@cloudera.com > > > >> > > wrote: > >> > > > > >> > > > > constraints: > >> > > > > > >> > > > > 1) hbase 0.94.6 is released and .snapshot is hardcoded in there. > >> > > > > 2) hdfs snapshots is a Hadoop 2.1 or 3.0 feature. I doubt that > it > >> > will > >> > > > ever > >> > > > > make it to 1.x. This hdfs feature ideally this shouldn't affect > >> > > current > >> > > > A > >> > > > > pache Hbase 0.94.x's. > >> > > > > 3) hbase 95/96 may default to Hadoop1 or Hadoop 2. these > versions > >> > > should > >> > > > > pick a different table snapshot name to respect fs conventions. > >> > > > > > >> > > > > proposed actions: > >> > > > > > >> > > > > 1) let's make the hbase snapshot for a conf variable. (hbase. > >> > > > > snapshots.dir) let's change the default for hbase 95+. (maybe > >> > > > > .hbase-snapshots). we'll also port this patch to 0.94.x > >> > > > > 2) let's publish instructions on how to update the hbase > snapshot > >> > dir: > >> > > > > shutdown hbase, config update, rename dir, restart hbase. > >> > > > > 3) I lean towards leaving the current default hbase snapshot > dir in > >> > 94 > >> > > > > since it shouldn't be affected. upgrading hbase to 95/96 will > >> > require > >> > > > > shutdown and update scripts so it seems like the ideal time to > >> > > autoforce > >> > > > > this default change. > >> > > > > > >> > > > > Thoughts? > >> > > > > > >> > > > > > >> > > > > On Monday, April 15, 2013, lars hofhansl wrote: > >> > > > > > >> > > > > > OK. Let's try to fix that quickly, so that I can release HBase > >> > > 0.94.7. > >> > > > > > > >> > > > > > -- Lars > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > ________________________________ > >> > > > > > From: Ted Yu <yuzhih...@gmail.com> > >> > > > > > To: d...@hbase.apache.org; hdfs-dev@hadoop.apache.org > >> > > > > > Sent: Monday, April 15, 2013 7:13 PM > >> > > > > > Subject: collision in the naming of '.snapshot' directory > between > >> > > hdfs > >> > > > > > snapshot and hbase snapshot > >> > > > > > > >> > > > > > > >> > > > > > Hi, > >> > > > > > This afternoon Huned ad I discovered an issue while playing > with > >> > > HBase > >> > > > > > Snapshots on top of Hadoop's Snapshot branch ( > >> > > > > > > http://svn.apache.org/viewvc/hadoop/common/branches/HDFS-2802/). > >> > > > > > > >> > > > > > HDFS (built from HDFS-2802 branch) doesn't allow paths with > >> > .snapshot > >> > > > as > >> > > > > a > >> > > > > > component while HBase tries to create paths with .snapshot as > a > >> > > > > component. > >> > > > > > This leads to issues in HBase, and one of HDFS or HBase needs > to > >> > give > >> > > > up > >> > > > > > the .snapshot reserved keyword. HBase released Snapshots > feature > >> in > >> > > > > 0.94.6 > >> > > > > > (quite recently) and it may not be too late to change HBase to > >> use > >> > a > >> > > > > > different path component in an upcoming new release. > >> > > > > > > >> > > > > > In HBase these path names are not user visible. If there is a > >> > > > deployment > >> > > > > of > >> > > > > > 0.94.6, one could provide a migration tool that renames > .snapshot > >> > to > >> > > > > > .hbase-snapshot or something to be able to move to the > Snapshot > >> > > release > >> > > > > of > >> > > > > > Hadoop. On the other hand, .snapshot in HDFS is a user visible > >> name > >> > > and > >> > > > > is > >> > > > > > a convention that is used by many file systems. It's a matter > of > >> > > > > > familiarity with such path names that would help users in > using > >> > HDFS > >> > > > > > snapshots. > >> > > > > > > >> > > > > > I am including the hdfs-dev in this email. Would appreciate > if we > >> > > could > >> > > > > > work together and come up with a solution. > >> > > > > > > >> > > > > > You can find sample output from hdfs command here: > >> > > > > > http://pastebin.com/bBqR4Fvr > >> > > > > > > >> > > > > > Cheers > >> > > > > > >> > > > > > >> > > > > > >> > > > > -- > >> > > > > // Jonathan Hsieh (shay) > >> > > > > // Software Engineer, Cloudera > >> > > > > // j...@cloudera.com > >> > > > > > >> > > > > >> > > > >> > > >> > >> > >> > >> -- > >> // Jonathan Hsieh (shay) > >> // Software Engineer, Cloudera > >> // j...@cloudera.com > >> > >> > >> > >> > >> > > > > -- > Harsh J >