Mujtaba Chohan created PHOENIX-2995:
---------------------------------------
Summary: Write performance severely degrades with large number of
views
Key: PHOENIX-2995
URL: https://issues.apache.org/jira/browse/PHOENIX-2995
Project: Phoenix
Issue Type: Bug
Reporter: Mujtaba Chohan
Write performance for each 1K batch degrades significantly when there are *10K*
views being written in random with default
{{phoenix.client.maxMetaDataCacheSize}}. With all views created, upsert rate
remains around 25 seconds per 1K batch i.e. ~2K rows/min upsert rate.
When {{phoenix.client.maxMetaDataCacheSize}} is increased to 100MB+ then view
does not need to get re-resolved and upsert rate gets back to normal ~60K
rows/min.
With *100K* views and {{phoenix.client.maxMetaDataCacheSize}} set to 1GB, I
wasn't able create all 100K views as upsert time for each 1K batch keeps on
steadily increasing.
Following graph shows 1K batch upsert rate over time with variation of number
of views. Rows are upserted to random views {{CREATE VIEW IF NOT EXISTS ...
APPEND_ONLY_SCHEMA = true, UPDATE_CACHE_FREQUENCY=900000}} is executed before
upsert statement.
Base table is also created with {{APPEND_ONLY_SCHEMA = true,
UPDATE_CACHE_FREQUENCY = 900000, AUTO_PARTITION_SEQ}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
