I filed this JIRA: https://issues.apache.org/jira/browse/PHOENIX-1306
Thanks On Tue, Sep 30, 2014 at 8:53 AM, James Taylor <jamestay...@apache.org> wrote: > Thanks, Krishna. Please file a JIRA for the "option in CREATE TABLE & > CREATE INDEX clauses to indicate that underlying table is already a > phoenix table". This would be a reasonable thing to add. > > James > > On Mon, Sep 29, 2014 at 11:03 PM, Krishna <research...@gmail.com> wrote: > > Hi James/Lars, > > > > I did another run of bulk load, backup & restore on a new test cluster > but > > did not encounter the issue this time. It appears to be some specific > issue > > with that version of backup. I will pass on the steps, as soon as I can, > to > > replicate it with a small dataset. > > > > Having said that, it is still beneficial to have some kind of an option > in > > CREATE TABLE & CREATE INDEX clauses to indicate that underlying table is > > already a phoenix table. > > > > Thanks, > > Krishna > > > > > > > > > > On Monday, September 29, 2014, lars hofhansl <la...@apache.org> wrote: > >> > >> Not offhand. > >> > >> A few guesses/questions for Krihna: > >> - does the restore tool restore the exact timestamps? In fact how > exactly > >> do you backup and restore the tables? > >> - is it possible that you updated the CATALOG (via some DDL) while the > >> backup was running? (although I think that should be OK) > >> > >> - does it restore into a clean(ed) cluster? If cleaned, was the ZK state > >> removed? > >> - after you restore the two tables, can you scan SYSTEM.CATALOG from an > >> HBase shell? Or does that part hang? > >> - in the HBase UI, are there any regions in transition? (might be some > >> outdated coprocessors, etc, although unlikely) > >> - exact same version of Phoenix? (between the Phoenix that wrote the > data > >> and the one that restored it) > >> - same for HBase. Exact same version used for the restore? > >> > >> As James said, we need some logs from both the client and the region > >> server(s). > >> Also if possible, do you have some precise steps to reproduce with a > small > >> table? > >> > >> Lots of questions :) > >> > >> Thanks. > >> > >> -- Lars > >> > >> > >> ----- Original Message ----- > >> From: James Taylor <jamestay...@apache.org> > >> To: user <user@phoenix.apache.org>; lars hofhansl <la...@apache.org> > >> Cc: > >> Sent: Monday, September 29, 2014 4:26 PM > >> Subject: Re: Recreating SYSTEM.CATALOG metadata > >> > >> @Lars - any idea why Krishna may run into issues using Phoenix after a > >> restore from an HBase backup? > >> > >> > >> > >> > >> On Sun, Sep 28, 2014 at 9:00 PM, James Taylor <jamestay...@apache.org> > >> wrote: > >> > Hi Krishna, > >> > I think that's what we need to figure out - why is Phoenix having > >> > trouble when you restore the SYSTEM.CATALOG table. Any client or > >> > server-side exceptions in the logs when it hangs? > >> > Thanks, > >> > James > >> > > >> > On Sun, Sep 28, 2014 at 6:18 PM, Krishna <research...@gmail.com> > wrote: > >> >> Hi James, > >> >> > >> >> I did include SYSTEM.CATALOG table in the backup & restore process. I > >> >> dont > >> >> know why sqlline is having trouble using the backup version - it just > >> >> hangs, > >> >> unless catalog table is dropped from hbase shell. I will see if there > >> >> are > >> >> any errors in logs that's causing the behavior. > >> >> > >> >> Regards > >> >> Krishna > >> >> > >> >> > >> >> On Sunday, September 28, 2014, James Taylor <jamestay...@apache.org> > >> >> wrote: > >> >>> > >> >>> Hi Krishna, > >> >>> Any reason why the SYSTEM.CATALOG hbase table isn't restored as well > >> >>> from backup? Yes, if you try to re-create the SYSTEM.CATALOG by > >> >>> re-issuing your DDL statement, Phoenix won't know that the tables > were > >> >>> Phoenix tables before, so will try to add the empty key value to > each > >> >>> row. It's possible that an option could be made to avoid this, but > >> >>> it'd be good to understand a bit more why this is needed. > >> >>> Thanks, > >> >>> James > >> >>> > >> >>> On Sun, Sep 28, 2014 at 1:56 PM, Krishna <research...@gmail.com> > >> >>> wrote: > >> >>> > Hi, > >> >>> > > >> >>> > When I restore hbase from a backup, sqlline gets stuck unless > >> >>> > SYSTEM.CATALOG > >> >>> > table is dropped. It is automatically re-created via sqlline. > >> >>> > However, > >> >>> > metadata of previously created phoenix tables is lost. > >> >>> > > >> >>> > So, to restore the metadata, when a 'CREATE TABLE' statement is > >> >>> > re-issued, > >> >>> > Phoenix takes very, very long time to execute which I think, is > >> >>> > because > >> >>> > Phoenix is upserting null value for _0 column qualifier again for > >> >>> > every > >> >>> > row > >> >>> > (billions of rows in the table). > >> >>> > > >> >>> > Is there a work-around for this by somehow telling Phoenix that > the > >> >>> > underlying table is already a Phoenix table? Using views is not an > >> >>> > option > >> >>> > here. If my understanding is wrong, any pointers as to why it > takes > >> >>> > so > >> >>> > long > >> >>> > for executing 'CREATE TABLE' statement is appreciated. > >> >>> > > >> >>> > Thanks > >> >>> > > >> >>> > > >> > > >
