Hi James, I've just upgraded and tested the RC6. Performance of bulk loading is pretty much stable now.
Thank you all of Phoenix team! Regards, Youngwoo 2016년 3월 3일 목요일, James Taylor<[email protected]>님이 작성한 메시지: > Youngwoo, > Prior to trying the new RC, you'll need to downgrade your SYSTEM.CATALOG > table to what it was before (at a minimum before you issued the manual > delete of the SYSTEM.STATS rows, but you can downgrade back to what it was > as of your previous Phoenix release). Phoenix bases it's upgrade on the > timestamp of the cells in the SYSTEM.CATALOG table, so future upgrades of > the SYSTEM.STATS table will get confused as the timestamp will be based on > wall clock time now in your cluster. > Thanks, > James > > On Wed, Mar 2, 2016 at 8:57 AM, James Taylor <[email protected] > <javascript:;>> wrote: > > > Canceling this vote due to PHOENIX-2728. New RC will be available with a > > fix today. > > > > > > On Wed, Mar 2, 2016 at 8:53 AM, James Taylor <[email protected] > <javascript:;>> > > wrote: > > > >> Thanks for testing this, Youngwoo. I've filed PHOENIX-2731 for this. > >> We'll roll a new RC this morning and we'll include a fix for that as > well. > >> Would be great if you could give the new RC a try. > >> > >> James > >> > >> On Wed, Mar 2, 2016 at 3:38 AM, 김영우 (YoungWoo Kim) <[email protected] > <javascript:;>> > >> wrote: > >> > >>> James, > >>> > >>> I'm trying to evaluate the new RC4. I built rpms from current master > and > >>> then installed the artifacts. as you may know, my env was installed > RC2, > >>> that is, I'm upgrading to RC4 from RC2. Unfortunately when I try to > >>> connect > >>> to Phoenix through QueryServer after restarting HBase and QueryServer, > >>> I'm > >>> stuck in errors from RC4 as follows: > >>> > >>> java.lang.RuntimeException: java.sql.SQLException: ERROR 1015 (42J04): > >>> Cannot add column to table when the last PK column is of type VARBINARY > >>> or > >>> ARRAY. columnName=GUIDE_POST_KEY > >>> at > >>> > >>> > org.apache.calcite.avatica.jdbc.JdbcMeta.openConnection(JdbcMeta.java:585) > >>> at > >>> > >>> > org.apache.calcite.avatica.remote.LocalService.apply(LocalService.java:263) > >>> at > >>> > >>> > org.apache.calcite.avatica.remote.Service$OpenConnectionRequest.accept(Service.java:1642) > >>> at > >>> > >>> > org.apache.calcite.avatica.remote.Service$OpenConnectionRequest.accept(Service.java:1625) > >>> at > >>> > >>> > org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:102) > >>> at > >>> > >>> > org.apache.calcite.avatica.remote.ProtobufHandler.apply(ProtobufHandler.java:38) > >>> at > >>> > >>> > org.apache.calcite.avatica.server.AvaticaProtobufHandler.handle(AvaticaProtobufHandler.java:68) > >>> at > >>> > org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52) > >>> at > >>> > >>> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) > >>> at org.eclipse.jetty.server.Server.handle(Server.java:497) > >>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) > >>> at > >>> > >>> > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:245) > >>> at > >>> > >>> > org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) > >>> at > >>> > >>> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) > >>> at > >>> > >>> > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) > >>> at java.lang.Thread.run(Thread.java:745) > >>> Caused by: java.sql.SQLException: ERROR 1015 (42J04): Cannot add column > >>> to > >>> table when the last PK column is of type VARBINARY or ARRAY. > >>> columnName=GUIDE_POST_KEY > >>> at > >>> > >>> > org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:422) > >>> at > >>> > >>> > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > >>> at > >>> > >>> > org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2625) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1021) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:338) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:326) > >>> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:325) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:1326) > >>> at > >>> > >>> > org.apache.phoenix.query.ConnectionQueryServicesImpl.addColumn(ConnectionQueryServicesImpl.java:2214) > >>> at > >>> > >>> > org.apache.phoenix.query.ConnectionQueryServicesImpl.addColumnsIfNotExists(ConnectionQueryServicesImpl.java:2242) > >>> at > >>> > >>> > org.apache.phoenix.query.ConnectionQueryServicesImpl.access$500(ConnectionQueryServicesImpl.java:211) > >>> at > >>> > >>> > org.apache.phoenix.query.ConnectionQueryServicesImpl$13.call(ConnectionQueryServicesImpl.java:2440) > >>> at > >>> > >>> > org.apache.phoenix.query.ConnectionQueryServicesImpl$13.call(ConnectionQueryServicesImpl.java:2248) > >>> at > >>> > >>> > org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:78) > >>> at > >>> > >>> > org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2248) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:233) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:135) > >>> at > org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:202) > >>> at java.sql.DriverManager.getConnection(DriverManager.java:571) > >>> at java.sql.DriverManager.getConnection(DriverManager.java:187) > >>> at > >>> > >>> > org.apache.calcite.avatica.jdbc.JdbcMeta.openConnection(JdbcMeta.java:582) > >>> ... 15 more > >>> > >>> Looks like upgrading the SYSTEM tables does not work for my case. the > >>> 'GUIDE_POST_KEY' column already exists. > >>> > >>> After that, I removed the SYSTEM.STATS table to connect Phoenix: > >>> - Disable the SYSTEM.STATS from HBase shell > >>> - Drop the SYSTEM.STATS on HBase > >>> - Delete rows related SYSTEM.STATS from SYSTEM.CATALOG > >>> > >>> Removing the SYSTEM.STATS table solves the connection problem. As of > >>> now, I > >>> can connect Phoenix in using QueryServer and also can run MR-based > >>> bulkload! but another problem popped up. I expected that when I run > >>> 'UPDATE > >>> STATISTICS' statement, Phoenix will create new SYSTEM.STATS table > because > >>> there is no SYSTEM.STATS on HBase and Phoenix CATALOG but updating > >>> statistics fails: > >>> > >>> java.lang.RuntimeException: > >>> org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03): > >>> Table > >>> undefined. tableName=SYSTEM.STATS > >>> at > org.apache.calcite.avatica.jdbc.JdbcMeta.propagate(JdbcMeta.java:651) > >>> at > >>> > >>> > org.apache.calcite.avatica.jdbc.JdbcMeta.prepareAndExecute(JdbcMeta.java:715) > >>> at > >>> > >>> > org.apache.calcite.avatica.remote.LocalService.apply(LocalService.java:186) > >>> at > >>> > >>> > org.apache.calcite.avatica.remote.Service$PrepareAndExecuteRequest.accept(Service.java:868) > >>> at > >>> > >>> > org.apache.calcite.avatica.remote.Service$PrepareAndExecuteRequest.accept(Service.java:842) > >>> at > >>> > >>> > org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:102) > >>> at > >>> > >>> > org.apache.calcite.avatica.remote.ProtobufHandler.apply(ProtobufHandler.java:38) > >>> at > >>> > >>> > org.apache.calcite.avatica.server.AvaticaProtobufHandler.handle(AvaticaProtobufHandler.java:68) > >>> at > >>> > org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52) > >>> at > >>> > >>> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) > >>> at org.eclipse.jetty.server.Server.handle(Server.java:497) > >>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) > >>> at > >>> > >>> > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:245) > >>> at > >>> > >>> > org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) > >>> at > >>> > >>> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) > >>> at > >>> > >>> > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) > >>> at java.lang.Thread.run(Thread.java:745) > >>> Caused by: org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 > >>> (42M03): Table undefined. tableName=SYSTEM.STATS > >>> at > >>> > >>> > org.apache.phoenix.compile.FromCompiler$BaseColumnResolver.createTableRef(FromCompiler.java:414) > >>> at > >>> > >>> > org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.<init>(FromCompiler.java:285) > >>> at > >>> > >>> > org.apache.phoenix.compile.FromCompiler.getResolverForQuery(FromCompiler.java:186) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:392) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:373) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:266) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:261) > >>> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:260) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:1313) > >>> at > >>> > >>> > org.apache.phoenix.schema.MetaDataClient.updateStatisticsInternal(MetaDataClient.java:919) > >>> at > >>> > >>> > org.apache.phoenix.schema.MetaDataClient.updateStatistics(MetaDataClient.java:857) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement$ExecutableUpdateStatisticsStatement$1.execute(PhoenixStatement.java:994) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:338) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:326) > >>> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:325) > >>> at > >>> > >>> > org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1345) > >>> at > >>> > >>> > org.apache.calcite.avatica.jdbc.JdbcMeta.prepareAndExecute(JdbcMeta.java:695) > >>> ... 15 more > >>> > >>> > >>> Updating the statistics creates SYSTEM.STATS table on HBase but does > not > >>> update SYSTEM.CATALOG so I run into a problem like above. I'm stuck in > >>> this > >>> status. Should I create the SYSTEM.STATS manually? if yes, please point > >>> me > >>> out the DDL for the STATS table. IMO, It would be nice to have the way > >>> upgrading SYSTEM.STATS gracefully (and sometimes manually) > >>> > >>> At this point, Should I downgrade the system catalogs? Is it possible? > If > >>> not, Can I create the SYSTEM.STATS table manually? > >>> > >>> Thanks, > >>> > >>> Youngwoo > >>> > >>> On Tue, Mar 1, 2016 at 5:11 AM, James Taylor <[email protected] > <javascript:;>> > >>> wrote: > >>> > >>> > Hello Everyone, > >>> > > >>> > This is a call for a vote on Apache Phoenix 4.7.0-HBase-1.1 RC4. This > >>> is > >>> > the next minor release of Phoenix 4, compatible with Apache HBase > >>> 1.1+. The > >>> > release includes both a source-only release and a convenience binary > >>> > release. > >>> > > >>> > This release has feature parity with our other pending 4.7.0 releases > >>> and > >>> > includes the following improvements: > >>> > - ACID transaction support (beta) [1] > >>> > - Statistics improvements [2][3][4] > >>> > - Performance improvements [5][6][7] > >>> > - 200+ other fixes/enhancements [8] > >>> > > >>> > The previous RC was sunk due to PHOENIX-2716 which is fixed in this > RC > >>> > (along with a number of others since the initial RC [9]). > >>> > > >>> > The source tarball, including signatures, digests, etc can be found > at: > >>> > > >>> > > >>> > https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.7.0-HBase-1.1-rc4/src/ > >>> > > >>> > The binary artifacts can be found at: > >>> > > >>> > > >>> > https://dist.apache.org/repos/dist/dev/phoenix/phoenix-4.7.0-HBase-1.1-rc4/bin/ > >>> > > >>> > For a complete list of changes, see: > >>> > > >>> > > >>> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120&version=12333998 > >>> > > >>> > Release artifacts are signed with the following key: > >>> > https://people.apache.org/keys/committer/mujtaba.asc > >>> > https://dist.apache.org/repos/dist/release/phoenix/KEYS > >>> > > >>> > The hash and tag to be voted upon: > >>> > > >>> > > >>> > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=58cf5fa2d2e147780258a1e88c79898fc199d61e > >>> > > >>> > > >>> > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.7.0-HBase-1.1-rc4 > >>> > > >>> > Vote will close on Thursday, March 3rd @ 5pm PST. Please vote: > >>> > > >>> > [ ] +1 approve > >>> > [ ] +0 no opinion > >>> > [ ] -1 disapprove (and reason why) > >>> > > >>> > Thanks, > >>> > The Apache Phoenix Team > >>> > > >>> > [1] https://phoenix.apache.org/transactions.html > >>> > [2] https://issues.apache.org/jira/browse/PHOENIX-2430 > >>> > [3] https://issues.apache.org/jira/browse/PHOENIX-2702 > >>> > [4] https://issues.apache.org/jira/browse/PHOENIX-2692 > >>> > [5] https://issues.apache.org/jira/browse/PHOENIX-1428 > >>> > [6] https://issues.apache.org/jira/browse/PHOENIX-2377 > >>> > [7] https://issues.apache.org/jira/browse/PHOENIX-2520 > >>> > [8] https://issues.apache.org/jira/issues/?filter=12334876 > >>> > [9] https://issues.apache.org/jira/issues/?filter=12334875 > >>> > > >>> > >> > >> > > >
