Ted, 2016-12-30 17:39 GMT+01:00 Ted Yu <[email protected]>:
> For hbase094_hbase_default.xml, currently it is > under hbase094/src/main/resources > Looks like it belongs to the test directory. > Yes because for now I have used a JUnit test in order to run the project. I will write done some command-line tool in order to run the full migration If you or other users are interested in such a tool you can write down some use-case and I will be happy to keep it in consideration. > > bq. use the 'Configuration#set" method to programmatically change the > values > Maybe I will let the user pass its own hbase-site.xml file, but instead of loading directly I will load into memory and then call HBaseConfiguration.set(key,value) for each entry. The way HBaseConfiguration looks ho hbase-site.xml is somehow tricky because it happens in the constructor and so even with subclassing it is not possible to disable the 'version check' > For end user, it would be more convenient to pass hbase-site.xml > > Cheers > > On Fri, Dec 30, 2016 at 8:19 AM, Enrico Olivelli <[email protected]> > wrote: > > > Hi Ted, > > > > 2016-12-30 17:11 GMT+01:00 Ted Yu <[email protected]>: > > > > > For scanHBase094Table(): > > > > > > baseDefaults.addResource("hbase094_hbase_default.xml"); > > > > > > where hbase.rootdir points to file:// > > > > > > > > How is the user supposed to plug in hbase-site.xml for the 0.94 > cluster ? > > > > > > > the resource gets loaded from the classpath by the Hadoop Configuration > > API. > > > > This is just a code sample to prove that a JVM can connect to two > > incompatible versions of HBase. > > > > you can plug in your configuration by loading a "Configuration" built by > > reading your configuration file. > > Personally I do not use the .xml files for configuration of clients but > use > > the 'Configuration#set" method to programmatically change the values > > > > > > > > > > For TableMigrationManager, I don't see where setBatchSize() is called. > > Does > > > this mean that batch size of 0 would be in effect ? > > > > > > > It is a bug, so the API uses its default values. See the HBase API docs > > > > > > > > > > Do you use any API specific to 1.2.4 ? Otherwise I suggest renaming > > > hbase124 > > > as hbase1 since 1.x releases are compatible. > > > > > > > +1 > > The only API that I use are the same as for 0.94 and 1.2 > > (scan,get,put,delete.....) > > > > > > > Thanks > > > > > > > > > > > > On Fri, Dec 30, 2016 at 6:33 AM, Enrico Olivelli <[email protected]> > > > wrote: > > > > > > > Thanks you Duo Zhang for the interest. > > > > > > > > The example comes from a Java class which actually "migrates" data > > > > from one table in a 0.94 cluster to a 1.2.4 cluster > > > > > > > > this is the file: > > > > https://github.com/eolivelli/multi-hbase/blob/master/multi- > > > > hbase-client/src/test/java/MigrateHBaseDataExample.java > > > > > > > > this can be the foundation for a future migration tool but at this > > > > stage that project is only a proof-of-concept > > > > > > > > > > > > 2016-12-30 14:22 GMT+01:00 张铎(Duo Zhang) <[email protected]>: > > > > > Nice try! > > > > > > > > > > Is this a migration tool? Or a thin layer on top of the original > > hbase > > > > > client to make the user be able to access both 0.94 and 1.2.4 with > > the > > > > same > > > > > API? > > > > > > > > > > Thanks. > > > > > > > > > > 2016-12-30 21:05 GMT+08:00 Enrico Olivelli <[email protected]>: > > > > > > > > > >> Hi all, > > > > >> The migration from 0.94 to 1.2.4 is really a pain and some time > > ago I > > > > >> asked to this list if there was a way to connect to both a 0.94 > > > > >> cluster and a 1.2.4 cluster. > > > > >> > > > > >> I have created a proof-of-concept project which is able to achieve > > my > > > > >> goal but "shading" most of the 0.94 client. > > > > >> > > > > >> This is the source code and the list of tricks to make it work: > > > > >> https://github.com/eolivelli/multi-hbase > > > > >> > > > > >> The above code simply performs a full scan from a 0.94 table and > > > > >> performs puts to a 1.2.4 cluster, but I think that most of the > basic > > > > >> API of the 0.94 client will work and the full API of the 1.2.4 is > > > > >> available. > > > > >> > > > > >> I'm going to implement a more stable version of the client to be > > used > > > > >> in production. > > > > >> > > > > >> I will appreciate a lot If anyone has some time to take a look and > > > > >> give me some feedback. > > > > >> > > > > >> Cheers > > > > >> Enrico Olivelli > > > > >> > > > > > > > > > >
