I have logged HBASE-8352 Cheers
On Mon, Apr 15, 2013 at 9:17 PM, Ted Yu <yuzhih...@gmail.com> wrote: > I have a patch which touched these tests: > > http://pastebin.com/P4p8LEAZ > > I am running 0.94 test suite now - will publish patch and test result in > the morning. > > Cheers > > > On Mon, Apr 15, 2013 at 9:00 PM, Ted Yu <yuzhih...@gmail.com> wrote: > >> bq. Alternatively, we can detect the underlying Hadoop version, and use >> either .snapshot or .hbase_snapshot in 0.94 depending on h1 & h2. >> >> I think this would introduce more confusion, especially for operations. >> >> Cheers >> >> On Mon, Apr 15, 2013 at 8:52 PM, Enis Söztutar <enis....@gmail.com>wrote: >> >>> Because HDFS exposes the snapshots so that the normal file system >>> operations are mapped inside snapshot dirs, I think HDFS reserving the >>> .snapshot name makes sense. OTOH, nothing is specific about the dir name >>> that is chosen by HBase. >>> >>> I would prefer to change the dir name in 0.94 as well, since 0.94 is also >>> being run on top of hadoop 2. Alternatively, we can detect the underlying >>> Hadoop version, and use either .snapshot or .hbase_snapshot in 0.94 >>> depending on h1 & h2. >>> >>> Enis >>> >>> >>> On Mon, Apr 15, 2013 at 8:31 PM, Ted Yu <yuzhih...@gmail.com> wrote: >>> >>> > bq. let's make the hbase snapshot for a conf variable. >>> > >>> > Once we decide on the new name of snapshot directory, we should still >>> use >>> > hardcoded value. This aligns with current code base: >>> > See this snippet from HConstants: >>> > >>> > public static final List<String> HBASE_NON_TABLE_DIRS = >>> > >>> > Collections.unmodifiableList(Arrays.asList(new String[] { >>> > HREGION_LOGDIR_NAME, >>> > >>> > HREGION_OLDLOGDIR_NAME, CORRUPT_DIR_NAME, SPLIT_LOGDIR_NAME, >>> > >>> > HBCK_SIDELINEDIR_NAME, HFILE_ARCHIVE_DIRECTORY, >>> SNAPSHOT_DIR_NAME, >>> > HBASE_TEMP_DIRECTORY })); >>> > Cheers >>> > >>> > On Mon, Apr 15, 2013 at 8:24 PM, Jonathan Hsieh <j...@cloudera.com> >>> wrote: >>> > >>> > > constraints: >>> > > >>> > > 1) hbase 0.94.6 is released and .snapshot is hardcoded in there. >>> > > 2) hdfs snapshots is a Hadoop 2.1 or 3.0 feature. I doubt that it >>> will >>> > ever >>> > > make it to 1.x. This hdfs feature ideally this shouldn't affect >>> current >>> > A >>> > > pache Hbase 0.94.x's. >>> > > 3) hbase 95/96 may default to Hadoop1 or Hadoop 2. these versions >>> should >>> > > pick a different table snapshot name to respect fs conventions. >>> > > >>> > > proposed actions: >>> > > >>> > > 1) let's make the hbase snapshot for a conf variable. (hbase. >>> > > snapshots.dir) let's change the default for hbase 95+. (maybe >>> > > .hbase-snapshots). we'll also port this patch to 0.94.x >>> > > 2) let's publish instructions on how to update the hbase snapshot >>> dir: >>> > > shutdown hbase, config update, rename dir, restart hbase. >>> > > 3) I lean towards leaving the current default hbase snapshot dir in >>> 94 >>> > > since it shouldn't be affected. upgrading hbase to 95/96 will >>> require >>> > > shutdown and update scripts so it seems like the ideal time to >>> autoforce >>> > > this default change. >>> > > >>> > > Thoughts? >>> > > >>> > > >>> > > On Monday, April 15, 2013, lars hofhansl wrote: >>> > > >>> > > > OK. Let's try to fix that quickly, so that I can release HBase >>> 0.94.7. >>> > > > >>> > > > -- Lars >>> > > > >>> > > > >>> > > > >>> > > > ________________________________ >>> > > > From: Ted Yu <yuzhih...@gmail.com> >>> > > > To: d...@hbase.apache.org; hdfs-dev@hadoop.apache.org >>> > > > Sent: Monday, April 15, 2013 7:13 PM >>> > > > Subject: collision in the naming of '.snapshot' directory between >>> hdfs >>> > > > snapshot and hbase snapshot >>> > > > >>> > > > >>> > > > Hi, >>> > > > This afternoon Huned ad I discovered an issue while playing with >>> HBase >>> > > > Snapshots on top of Hadoop's Snapshot branch ( >>> > > > http://svn.apache.org/viewvc/hadoop/common/branches/HDFS-2802/). >>> > > > >>> > > > HDFS (built from HDFS-2802 branch) doesn't allow paths with >>> .snapshot >>> > as >>> > > a >>> > > > component while HBase tries to create paths with .snapshot as a >>> > > component. >>> > > > This leads to issues in HBase, and one of HDFS or HBase needs to >>> give >>> > up >>> > > > the .snapshot reserved keyword. HBase released Snapshots feature in >>> > > 0.94.6 >>> > > > (quite recently) and it may not be too late to change HBase to use >>> a >>> > > > different path component in an upcoming new release. >>> > > > >>> > > > In HBase these path names are not user visible. If there is a >>> > deployment >>> > > of >>> > > > 0.94.6, one could provide a migration tool that renames .snapshot >>> to >>> > > > .hbase-snapshot or something to be able to move to the Snapshot >>> release >>> > > of >>> > > > Hadoop. On the other hand, .snapshot in HDFS is a user visible >>> name and >>> > > is >>> > > > a convention that is used by many file systems. It's a matter of >>> > > > familiarity with such path names that would help users in using >>> HDFS >>> > > > snapshots. >>> > > > >>> > > > I am including the hdfs-dev in this email. Would appreciate if we >>> could >>> > > > work together and come up with a solution. >>> > > > >>> > > > You can find sample output from hdfs command here: >>> > > > http://pastebin.com/bBqR4Fvr >>> > > > >>> > > > Cheers >>> > > >>> > > >>> > > >>> > > -- >>> > > // Jonathan Hsieh (shay) >>> > > // Software Engineer, Cloudera >>> > > // j...@cloudera.com >>> > > >>> > >>> >> >> >
