What if there is a bad sector on the disk of remote server ? Using hdfs, you're shielded from such scenario.
On Dec 1, 2013, at 6:56 AM, oc tsdb <[email protected]> wrote: > We want to use that exported data in case of any disaster happens. > > Thanks > -OC > > > On Sun, Dec 1, 2013 at 2:25 PM, Ted Yu <[email protected]> wrote: > >> Since the remote server doesn't have hdfs, what would the backup be used >> for ? >> >> Cheers >> >> On Nov 30, 2013, at 10:34 PM, oc tsdb <[email protected]> wrote: >> >>> Hi, >>> >>>>> Can you clarify what 'without copying to local file system' means ? >>> >>> We want to take HBase tables backup.So we periodically take snapshots and >>> export data to backup server which is not in cluster. >>> >>> Currently we are achieving this as mentioned below. >>> >>> 1.Create snapshots from HBase command line >>> lets say we have created snapshot for a table called 'hbase_tabel' and >>> its snapshot name is 'hbase_tabel_snapshot' >>> 2. Then export snapshot data to remote backup server which is not in >>> cluster.We are achieving this in two steps. >>> a. Export snapshot to local file system(sample path /tmp/hbase_backup) >>> from hdfs file system as mentioned below >>> hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot >>> hbase_tabel_snapshot -copy-to file:///tmp/hbase_backup - >>> mappers 16 >>> Here we should have enough disc space in local file system to export >>> snapshot from hdfs. >>> b.Transfer(scp) exported snapshot from local file system >>> (/tmp/hbase_backup) >>> to backup server which is not hadoop/HBase running >>> server. >>> >>> That means first we have to export snapshot to local system from hdfs >> and >>> then transfer ( scp ) to remote server.Instead of doing in two steps can >> we >>> directly export snapshot to remote server(which is not in cluster) like >> how >>> it is exporting to another cluster. >>> . >>> Thanks >>> OC >>> >>> >>> >>> >>> >>> On Sat, Nov 30, 2013 at 11:11 AM, Ted Yu <[email protected]> wrote: >>> >>>> HBASE-4618 is not related to snapshots. >>>> Lars did some work in HBASE-5509, a sub-task of HBASE-4618. >>>> >>>> FYI >>>> >>>> >>>> On Fri, Nov 29, 2013 at 9:31 PM, Yu Li <[email protected]> wrote: >>>> >>>>> I guess this might be a use case of >>>>> HBASE-7912<https://issues.apache.org/jira/browse/HBASE-7912>or >>>>> HBASE-4618 <https://issues.apache.org/jira/browse/HBASE-4618>? Maybe >>>> some >>>>> backup/restore tools taking use of snapshot would be nice to have? >>>>> >>>>> >>>>> On 30 November 2013 12:58, Ted Yu <[email protected]> wrote: >>>>> >>>>>> The remote server is in different cluster, right ? >>>>>> >>>>>> Looking at ExportSnapshot#ExportMapper#copyFile(), it directly copies >>>>> data >>>>>> to outputPath which is in the other cluster. >>>>>> >>>>>> Can you clarify what 'without copying to local file system' means ? >>>>>> >>>>>> Cheers >>>>>> >>>>>> >>>>>> On Fri, Nov 29, 2013 at 8:49 PM, oc tsdb <[email protected]> wrote: >>>>>> >>>>>>> Hi, >>>>>>> thanks for your replay. >>>>>>> >>>>>>> We are able to export snapshot to another cluster But this is not >>>> what >>>>> we >>>>>>> are looking for. >>>>>>> We want to export snapshot to remote server without copying to local >>>>> file >>>>>>> system. >>>>>>> >>>>>>> Thanks >>>>>>> OC >>>>>>> >>>>>>> On Fri, Nov 29, 2013 at 8:52 PM, Ted Yu <[email protected]> wrote: >>>>>>> >>>>>>>> Here is a sample command exporting to another cluster: >>>>>>>> >>>>>>>> hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot >>>>>>>> gs_1383068263 -copy-to hdfs://experiment-1d:8020/hbase -mappers 6 >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Nov 29, 2013 at 4:02 AM, oc tsdb <[email protected]> >>>> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> can someone please let us know if we can export snapshots data >>>>>> directly >>>>>>>> to >>>>>>>>> outside of cluster without copying to local. >>>>>>>>> >>>>>>>>> Please refer us if there are any blog with example. >>>>>>>>> >>>>>>>>> Below command is copying data to local file system under /tmp. >>>>>>>>> hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot >>>>>> mytable >>>>>>>>> -copy-to file:///tmp/hbase_backup -mappers 16 >>>>>>>>> >>>>>>>>> Without copying to local system, we want to export snapshot(eg >>>> scp) >>>>>>> data >>>>>>>> to >>>>>>>>> a server which is not part of any cluster but its reachable from >>>>>>> cluster. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> OC >>>>> >>>>> >>>>> >>>>> -- >>>>> Best Regards, >>>>> Li Yu >>
