[
https://issues.apache.org/jira/browse/PHOENIX-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419808#comment-15419808
]
James Taylor edited comment on PHOENIX-2995 at 8/13/16 7:02 AM:
----------------------------------------------------------------
Thanks, [~tdsilva]. Looks very good. Couple of questions/comments:
- Would it be possible to move the {{new PhoenixConnection()}} outside of the
synchronize block?
{code}
@Override
public PhoenixConnection connect(String url, Properties info) throws
SQLException {
- checkClosed();
- PMetaData metadata = latestMetaData;
- if (metadata == null) {
- throwConnectionClosedException();
+ synchronized (latestMetaDataLock) {
+ checkClosed();
+ PMetaData metadata = latestMetaData;
+ if (metadata == null) {
+ throwConnectionClosedException();
+ }
+
+ return new PhoenixConnection(this, url, info, metadata);
}
-
- return new PhoenixConnection(this, url, info, metadata);
}
{code}
- Just curious on the updateCache changes to load the physical table - was that
being done differently before? I think we always load the physical table in the
update cache for a view and assume that we can find it in the cache. Maybe
that's a bad assumption?
Would you mind reviewing the new synchronization, [~samarthjain]?
was (Author: jamestaylor):
Thanks, [~tdsilva]. Looks very good. Couple of questions/comments:
- Would it be possible to move the {{new PhoenixConnection()}} outside of the
synchronize block?
{code}
@Override
public PhoenixConnection connect(String url, Properties info) throws
SQLException {
- checkClosed();
- PMetaData metadata = latestMetaData;
- if (metadata == null) {
- throwConnectionClosedException();
+ synchronized (latestMetaDataLock) {
+ checkClosed();
+ PMetaData metadata = latestMetaData;
+ if (metadata == null) {
+ throwConnectionClosedException();
+ }
+
+ return new PhoenixConnection(this, url, info, metadata);
}
-
- return new PhoenixConnection(this, url, info, metadata);
}
{code}
- Just curious on the updateCache changes to load the physical table - was that
being done differently before?
Would you mind reviewing the new synchronization, [~samarthjain]?
> Write performance severely degrades with large number of views
> ---------------------------------------------------------------
>
> Key: PHOENIX-2995
> URL: https://issues.apache.org/jira/browse/PHOENIX-2995
> Project: Phoenix
> Issue Type: Bug
> Reporter: Mujtaba Chohan
> Assignee: Thomas D'Silva
> Labels: Argus
> Fix For: 4.8.1
>
> Attachments: PHOENIX-2995-v2.patch, PHOENIX-2995-v3.patch,
> PHOENIX-2995.patch, create_view_and_upsert.png, image.png, image2.png,
> image3.png, upsert_rate.png
>
>
> Write performance for each 1K batch degrades significantly when there are
> *10K* views being written in random with default
> {{phoenix.client.maxMetaDataCacheSize}}. With all views created, upsert rate
> remains around 25 seconds per 1K batch i.e. ~2K rows/min upsert rate.
> When {{phoenix.client.maxMetaDataCacheSize}} is increased to 100MB+ then view
> does not need to get re-resolved and upsert rate gets back to normal ~60K
> rows/min.
> With *100K* views and {{phoenix.client.maxMetaDataCacheSize}} set to 1GB, I
> wasn't able create all 100K views as upsert time for each 1K batch keeps on
> steadily increasing.
> Following graph shows 1K batch upsert rate over time with variation of number
> of views. Rows are upserted to random views {{CREATE VIEW IF NOT EXISTS ...
> APPEND_ONLY_SCHEMA = true, UPDATE_CACHE_FREQUENCY=900000}} is executed before
> upsert statement.
> !upsert_rate.png!
> Base table is also created with {{APPEND_ONLY_SCHEMA = true,
> UPDATE_CACHE_FREQUENCY = 900000, AUTO_PARTITION_SEQ}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
