Ankit,
When tried executing the query, the following exception is thrown.
Error: ERROR 301 (23000): Concurrent modification to table.
tableName=SYSTEM.CATALOG (state=23000,code=301)
org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
(23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
at
org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
at
org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at
org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at
org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at
org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at
org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
Thank,
Arun
On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <ankitsingha...@gmail.com>
wrote:
> Arun,
> As per the system.catalog ,there seems no way that upgrade code(adding new
> columns and view upgrade for version < v4.5) doesn't execute.
>
> But anyways, let's add the required columns manually by following below
> step;
>
> > ./sqlline.py localhost;CurrentSCN=9
> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN;
> >!quit
>
> Quit the shell and start new session without CurrentSCN.
> > ./sqlline.py localhost
> > !describe system.catalog
>
> this should resolve the issue of missing column.
>
> Regards,
> Ankit Singhal
>
>
> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
> arun1...@gmail.com> wrote:
>
>> Ankit,James,
>>
>> Please find the results you have asked for in the attachment text
>> document.
>>
>> Thanks,
>> Arun
>>
>> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <ankitsingha...@gmail.com>
>> wrote:
>>
>>> Arun,
>>> the observations you had is understandable.
>>> So, that's why we again requested the output of
>>>
>>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
>>>
>>> So that we can help you in adding ROW_TIMESTAMP column in system.catalog
>>> table so that your phoenix setup works properly and you can also see it in
>>> !describe command.
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
>>> arun1...@gmail.com> wrote:
>>>
>>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>>>
>>>> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1
>>>> client,
>>>>
>>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>>>
>>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>>>>
>>>> Is this an expected behavior?
>>>>
>>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I
>>>> do? This issue is blocking us in the production environment. Any help to
>>>> resolve or workaround is highly appreciated.
>>>>
>>>> Thanks,
>>>> Arun
>>>>
>>>>
>>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <
>>>> ankitsingha...@gmail.com> wrote:
>>>>
>>>>> It's ok if you can just post after grep for CATALOG in a command
>>>>> output (scan 'SYSTEM.CATALOG', {RAW=>true}).
>>>>>
>>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>>>> arun1...@gmail.com> wrote:
>>>>>
>>>>>> One more question to add,
>>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>>>
>>>>>> I have limited the scan in HBase and here is the following data.
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>> type=DeleteFamily
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x05
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>>>> timestamp=1460354090655, value=default
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
>>>>>> value=\x00
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
>>>>>> value=\x00
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>>>> timestamp=1460354090655, value=\x01
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
>>>>>> value=\x00
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655,
>>>>>> value=u
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>> type=DeleteFamily
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x0C
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x01
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x02
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>> type=DeleteFamily
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x0C
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>> value=\x80\x01
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x02
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>> type=DeleteFamily
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x0C
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x01
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x02
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>> type=DeleteFamily
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x0C
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x01
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x02
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>> type=DeleteFamily
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x0C
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x01
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x02
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Arun
>>>>>>
>>>>>>
>>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>>>> arun1...@gmail.com> wrote:
>>>>>>
>>>>>>> James,
>>>>>>>
>>>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2
>>>>>>> =>
>>>>>>> '|org.apache
>>>>>>>
>>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>>>> coprocessor$3 =>
>>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>>>
>>>>>>> |', coprocessor$4 =>
>>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>>>
>>>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>>>
>>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>>>>>>> COLUMN FAMILIES DESCRIPTION
>>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER =>
>>>>>>> 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS =>
>>>>>>> '1000',
>>>>>>>
>>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
>>>>>>> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>>>>
>>>>>>> 1 row in 0.6060 seconds
>>>>>>>
>>>>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Arun
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <
>>>>>>> jamestay...@apache.org> wrote:
>>>>>>>
>>>>>>>> Arun,
>>>>>>>> Please run the command Ankit mentioned in an HBase shell and post
>>>>>>>> the output back here.
>>>>>>>> Thanks,
>>>>>>>> James
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>>>>> arun1...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>>>>>>>>>
>>>>>>>>> But we do see this column from select statement:
>>>>>>>>>
>>>>>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
>>>>>>>>> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Arun
>>>>>>>>>
>>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>>>> ankitsingha...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Arun,
>>>>>>>>>>
>>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
>>>>>>>>>> !describe on system.catalog.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> if not,
>>>>>>>>>> can you share the output of below command. As it seems
>>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp ,
>>>>>>>>>> and
>>>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>>>
>>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Ankit Singhal
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>> arun1...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> After further investigation, we found that Phoenix Upsert query
>>>>>>>>>>> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but
>>>>>>>>>>> PTableImpl.getColumn() is
>>>>>>>>>>> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP"
>>>>>>>>>>> . Does
>>>>>>>>>>> this mean that PTableImpl is reading from cached entity of
>>>>>>>>>>> SYSTEM.CATALOG
>>>>>>>>>>> before 4.6 upgrade?"
>>>>>>>>>>>
>>>>>>>>>>> We do see that clearCache() is being called for 4.7, and 4.7
>>>>>>>>>>> upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Arun
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>> arun1...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> James,
>>>>>>>>>>>>
>>>>>>>>>>>> To add more information on this issue, this happens in new
>>>>>>>>>>>> phoenix views associated with brand new tables as well. So, this
>>>>>>>>>>>> cannot be
>>>>>>>>>>>> an upgrade/migration issue. Not figured out a specific way to
>>>>>>>>>>>> reproduce
>>>>>>>>>>>> this issue yet. Could you throw some ideas on what direction this
>>>>>>>>>>>> problem
>>>>>>>>>>>> could be approached from this point?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Arun
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
