Thanks, that's good to know.
On Sun, Jun 29, 2014 at 10:20 AM, James Taylor <[email protected]> wrote: > FWIW, the upgrade script does not touch your data - it only modifies > the metadata in your SYSTEM.CATALOG table. > > On Sun, Jun 29, 2014 at 10:18 AM, Kristoffer Sjögren <[email protected]> > wrote: > > Thanks James for the clarification. I haven't tried the upgrade procedure > > yet. I just wanted to migrate our existing data first to CDH5 and try > out a > > few things on HBase 0.96. > > > > > > On Sun, Jun 29, 2014 at 9:50 AM, James Taylor <[email protected]> > > wrote: > >> > >> The default column family (i.e. the name of the column family used for > >> your table when one is not explicitly specified) was changed from _0 > >> to 0 between 2.2 and 3.0/4.0. You can override this in your CREATE > >> TABLE statement through the DEFAULT_COLUMN_FAMILY property. The > >> upgrade script modifies this property dynamically for any table being > >> upgraded. > >> > >> Was the upgrade script not working for you? > >> > >> Thanks, > >> James > >> > >> On Fri, Jun 27, 2014 at 6:44 PM, Kristoffer Sjögren <[email protected]> > >> wrote: > >> > After copying data from column _0 to column 0 phoenix is able to read > >> > the data (didn't try the catalog trick). > >> > > >> > I suppose this is the way Phoenix does the upgrade internally also > >> > (move data between columns)? > >> > > >> > On Wed, Jun 25, 2014 at 9:48 PM, Jeffrey Zhong < > [email protected]> > >> > wrote: > >> >> > >> >> No, you have to create column family name 0 and copy the data from > >> >> column > >> >> family "_0" to it or (personally I didn't try the following) you > might > >> >> able to change meta data in system.catalog to use column family "_0" > >> >> instead. > >> >> > >> >> For normal upgrade, you should follow instructions at > >> >> http://phoenix.apache.org/upgrade_from_2_2.html > >> >> > >> >> On 6/25/14 7:11 AM, "Kristoffer Sjögren" <[email protected]> wrote: > >> >> > >> >>>Yes, the import worked after creating the column family and I can see > >> >>>all the rows when doing scans. > >> >>> > >> >>>But I got nothing when using Phoenix 4.0 client, so after comparing > >> >>>old and new tables I saw that 4.0 tables have column family name 0 > >> >>>instead of _0. > >> >>> > >> >>>Now as far as I know there is no way to rename a column qualifier, > >> >>>right? So I cant simply remove the 0 column and rename _0 to 0 right? > >> >>> > >> >>> > >> >>> > >> >>>On Wed, Jun 25, 2014 at 3:08 AM, Jeffrey Zhong < > [email protected]> > >> >>>wrote: > >> >>>> > >> >>>> > >> >>>> You can try to use hbase shell to manually add "_0" column family > >> >>>> into > >> >>>> your destination hbase table. Phoenix 4.0 from Apache can't work on > >> >>>> hbase0.96. You can check discussions in > >> >>>> https://issues.apache.org/jira/browse/PHOENIX-848 to see if your > >> >>>> hbase > >> >>>>is > >> >>>> good for phoenix 4.0. > >> >>>> > >> >>>> Thanks, > >> >>>> -Jeffrey > >> >>>> > >> >>>> On 6/24/14 5:32 AM, "Kristoffer Sjögren" <[email protected]> wrote: > >> >>>> > >> >>>>>Hi > >> >>>>> > >> >>>>>We're currently running Phoenix 2.2 on HBase 0.94 CDH 4.4 and > slowly > >> >>>>>preparing to move to Phoenix 4 and HBase 0.96 CDH 5. > >> >>>>> > >> >>>>>For my first tests I wanted to simply copy data from 0.94 to 0.96, > >> >>>>>which works fine for regular hbase table using the following > >> >>>>> commands: > >> >>>>> > >> >>>>>$ hbase org.apache.hadoop.hbase.mapreduce.Export table /tmp/table > >> >>>>>$ hadoop distcp hftp://hbase94:50070/tmp/table > >> >>>>> hdfs://hbase96/tmp/table > >> >>>>>$ hbase -Dhbase.import.version=0.94 > >> >>>>>org.apache.hadoop.hbase.mapreduce.Import table /tmp/table > >> >>>>> > >> >>>>>This approach fail on the import for for phoenix tables tough (see > >> >>>>>below) - where I create an identical table in 0.96 using phoenix > 4.0 > >> >>>>>sqlline and then do the commands mentioned above. As I understand, > >> >>>>> the > >> >>>>>_0 column family was used to allow hbase empty rows. > >> >>>>> > >> >>>>>Are there any tricks that can be made to allow copy data between > >> >>>>> these > >> >>>>>two installations? > >> >>>>> > >> >>>>>Cheers, > >> >>>>>-Kristoffer > >> >>>>> > >> >>>>> > >> >>>>>2014-06-18 13:31:09,633 INFO [main] mapreduce.Job: Task Id : > >> >>>>>attempt_1403015236309_0015_m_000004_1, Status : FAILED > >> >>>>>Error: > >> > >>>>>org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: > >> >>>>>Failed 6900 actions: > >> >>>>>org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: > >> >>>>>Column family _0 does not exist in region > >> > >> >>>>> > >>>>>TABLE,\x1F\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x > >> >>>>>00 > >> > >> >>>>> > >>>>>\x00\x00\x00\x00\x00\x00,1403090967713.51a3eefa9b92568a87223aff7878cdcf. > >> >>>>>in table 'TABLE', {TABLE_ATTRIBUTES => {coprocessor$1 => > >> >>>>>'|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', > >> >>>>> coprocessor$2 > >> >>>>>=> > >> > >> >>>>> > >>>>>'|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', > >> >>>>>coprocessor$3 => > >> > >>>>>'|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|', > >> >>>>>coprocessor$4 => > >> >>>>>'|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', > >> >>>>>coprocessor$5 => > >> > >> >>>>> > >>>>>'|org.apache.phoenix.hbase.index.Indexer|1073741823|index.builder=org.ap > >> >>>>>ac > >> > >> >>>>> > >>>>>he.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hbase.index.codec > >> >>>>>.c > >> >>>>>lass=org.apache.phoenix.index.PhoenixIndexCodec'}, > >> >>>>>{NAME => '0', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => > >> >>>>>'false', KEEP_DELETED_CELLS => 'true', DATA_BLOCK_ENCODING => > >> >>>>>'FAST_DIFF', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS > => > >> >>>>>'0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE > => > >> >>>>>'0'} > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServ > >> >>>>>er > >> >>>>>.java:4056) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMuta > >> >>>>>ti > >> >>>>>on(HRegionServer.java:3361) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.j > >> >>>>>av > >> >>>>>a:3265) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2. > >> >>>>>ca > >> >>>>>llBlockingMethod(ClientProtos.java:26935) > >> >>>>>at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2175) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1879) > >> >>>>>: 6900 times, > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(As > >> >>>>>yn > >> >>>>>cProcess.java:187) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$500(Async > >> >>>>>Pr > >> >>>>>ocess.java:171) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.client.AsyncProcess.getErrors(AsyncProcess.java: > >> >>>>>88 > >> >>>>>2) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.client.HTable.backgroundFlushCommits(HTable.java > >> >>>>>:9 > >> >>>>>40) > >> >>>>>at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:903) > >> >>>>>at org.apache.hadoop.hbase.client.HTable.put(HTable.java:864) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.wr > >> >>>>>it > >> >>>>>e(TableOutputFormat.java:126) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.wr > >> >>>>>it > >> >>>>>e(TableOutputFormat.java:87) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask. > >> >>>>>ja > >> >>>>>va:635) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskIn > >> >>>>>pu > >> >>>>>tOutputContextImpl.java:89) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedM > >> >>>>>ap > >> >>>>>per.java:112) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.mapreduce.Import$Importer.writeResult(Import.jav > >> >>>>>a: > >> >>>>>167) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.mapreduce.Import$Importer.map(Import.java:136) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.hbase.mapreduce.Import$Importer.map(Import.java:118) > >> >>>>>at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) > >> >>>>>at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) > >> >>>>>at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) > >> >>>>>at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > >> >>>>>at java.security.AccessController.doPrivileged(Native Method) > >> >>>>>at javax.security.auth.Subject.doAs(Subject.java:422) > >> >>>>>at > >> > >> >>>>> > >>>>>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformatio > >> >>>>>n. > >> >>>>>java:1548) > >> >>>>>at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > >> >>>> > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> CONFIDENTIALITY NOTICE > >> >>>> NOTICE: This message is intended for the use of the individual or > >> >>>>entity to > >> >>>> which it is addressed and may contain information that is > >> >>>> confidential, > >> >>>> privileged and exempt from disclosure under applicable law. If the > >> >>>>reader > >> >>>> of this message is not the intended recipient, you are hereby > >> >>>> notified > >> >>>>that > >> >>>> any printing, copying, dissemination, distribution, disclosure or > >> >>>> forwarding of this communication is strictly prohibited. If you > have > >> >>>> received this communication in error, please contact the sender > >> >>>>immediately > >> >>>> and delete it from your system. Thank You. > >> >> > >> >> > >> >> > >> >> -- > >> >> CONFIDENTIALITY NOTICE > >> >> NOTICE: This message is intended for the use of the individual or > >> >> entity to > >> >> which it is addressed and may contain information that is > confidential, > >> >> privileged and exempt from disclosure under applicable law. If the > >> >> reader > >> >> of this message is not the intended recipient, you are hereby > notified > >> >> that > >> >> any printing, copying, dissemination, distribution, disclosure or > >> >> forwarding of this communication is strictly prohibited. If you have > >> >> received this communication in error, please contact the sender > >> >> immediately > >> >> and delete it from your system. Thank You. > > > > >
