[
https://issues.apache.org/jira/browse/PHOENIX-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15830645#comment-15830645
]
Josh Elser commented on PHOENIX-3583:
-------------------------------------
Went mad trying to find a JIRA issue because I know I had looked at this code
before, but turns out it was a mailing list thread :)
https://lists.apache.org/thread.html/059ea2ad18928c4383f47accc42a7a9e3c4a88c581863034327b45f1@%3Cdev.phoenix.apache.org%3E
bq. Do we have perf data that is pointing to this as an issue
Back in the day, I was at least able to show that the number of RPCs just for
IndexMaintainers exceeded those used to actually write the data table updates.
Granted, the length of those RPCs are quite different, but that many RPCs will
start to add up regardless.
bq. The IndexMaintainer is meant to be a very small, compiled form of only the
necessary meta data to maintain indexes. I don't think we need to prepare it on
the server side as it might be out of sync with the client. The client should
drive this.
Maybe a hybrid is possible? The client can create the IndexMaintainer but then
keep using the same entry in the ServerCache for multiple mini-batches? I like
the idea of re-using the {{updateCacheFrequency}}.
Granted, it's been a while since I've looked at this code. I should really
refresh myself..
> Prepare IndexMaintainer on server itself
> ----------------------------------------
>
> Key: PHOENIX-3583
> URL: https://issues.apache.org/jira/browse/PHOENIX-3583
> Project: Phoenix
> Issue Type: Bug
> Reporter: Ankit Singhal
> Assignee: Ankit Singhal
> Attachments: PHOENIX-3583.patch
>
>
> -- reuse the cache of PTable and it's lifecycle.
> -- With the new implementation, we will be doing RPC to meta table per mini
> batch which could be an overhead, but the same configuration
> "updateCacheFrequency" can be used to control a frequency of touching
> SYSTEM.CATALOG endpoint for updated Ptable or index maintainers.
> -- It is expected that 99% of the time the table is old and RPC will be
> returned with an empty result(so it may be less costly), as opposed to the
> current implementation where we have to send the index maintainer payload to
> each region server per upsert batch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
