[
https://issues.apache.org/jira/browse/PHOENIX-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16320877#comment-16320877
]
Karan Mehta commented on PHOENIX-4523:
--------------------------------------
[~f.pompermaier] Thank you for the responses. I have a potential patch ready
but want to confirm with few more questions.
bq. I debugged a little more the code and it seems that the
ensureSystemTablesMigratedToSystemNamespace() is always called, causing a lock
to be created.
That method is always called, however if SYSTEM tables are already migrated to
SYSTEM namespace, then {{getSystemTableNames()}} will not return any tables and
hence the method would return from that point. Even if it did go inside, having
a row with the value {{UPGRADE_MUTEX_UNLOCKED}} means that the lock was not
acquired. Lock is only acquired during SYSTEM.CATALOG upgrade or SYSTEM table
migration. If both of them are done, then it never be acquired again.
This stack trace still doesn't tell where exactly the problem started from. I
checked out the tag {{v4.13.1-HBase-1.2-rc0}} and the only place
{{createSysMutexTable()}} method is called is inside
{{createOtherSystemTables()}} method. The master branch has multiple callers to
this method. Do you want me to test with master branch?
bq. it seems that TableExistsException is not catched within
createSysMutexTable() because is wrapped by a
org.apache.hadoop.ipc.RemoteException, and here I don't know why that exception
is not wrapped during local debug
This and the check for SYSMUTEX table seems to the valid issue. I will fix it
in the patch.
bq. So it appears that in the case of a cluster that hasn't been upgraded yet,
there are problems with the execution of the
ensureSystemTablesMigratedToSystemNamespace().
According to [~f.pompermaier], the cluster is already migrated. So the
{{ensureSystemTablesMigratedToSystemNamespace()}} method is almost
short-circuited.
bq. can we do a little refactoring and call a new method instead of
createSysMutexTable that doesn't again check for the existence of SYSTEM.MUTEX
(since we already check before calling createSysMutexTable
I will do some refactoring too.
bq. why in createSysMutexTable do we call admin.listTableNames() and then one
line later call admin.tableExists()? That seems like a complete waste.
We should not worry much about optimization here IMHO, since this code path is
not frequently executed. The code is already messy and trying to optimize might
result into some unexpected issues.
I will put up a patch once these things are confirmed.
> phoenix.schema.isNamespaceMappingEnabled problem
> ------------------------------------------------
>
> Key: PHOENIX-4523
> URL: https://issues.apache.org/jira/browse/PHOENIX-4523
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.13.1
> Reporter: Flavio Pompermaier
> Assignee: Karan Mehta
>
> I'm using Phoenix 4.13 for CDH 5.11.2 parcel and enabling schemas made my
> code unusable.
> I think that this is not a bug of the CDH release, but of all 4.13.x releases.
> I have many parallel Phoenix connections and I always get the following
> exception:
> {code:java}
> Caused by: java.sql.SQLException:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.TableExistsException):
> SYSTEM:MUTEX
> at
> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2492)
> at
> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2384)
> at
> org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:76)
> at
> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2384)
> at
> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:255)
> at
> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:150)
> at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
> at java.sql.DriverManager.getConnection(DriverManager.java:664)
> at java.sql.DriverManager.getConnection(DriverManager.java:270)
> {code}
> This is caused by the fact that all the times the SYSTEM tables are
> recreated, and this cannot be done simultaneously.
> Trying to debug the issue I found that in
> ConnectionQueryServicesImpl.createSysMutexTable() the call to
> getSystemTableNames() always return an empty array and the SYSTEM:MUTEX
> table is always recreated.
> This because getSystemTableNames() doesn't consider the case when system
> tables have namespace enabled. Right now that method tries to get all tables
> starting with *SYSTEM.\**, while it should try to get the list of *SYSTEM:\**
> tables..
> I hope this could get fixed very soon,
> Flavio
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
