[
https://issues.apache.org/jira/browse/PHOENIX-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881510#comment-15881510
]
James Taylor commented on PHOENIX-3218:
---------------------------------------
bq. When using `UPSERT` to write a large number of records, turn off autocommit
and batch records. Start with a batch size of 1000 and adjust as needed. Here's
some pseudocode showing one way to commit records in batches:
Josh Elser says:
_Recommend putting a caveat here that the use of commit() by Phoenix to control
batches of data written to HBase as being "non-standard" in terms of JDBC._
This is a common RDBMS pattern. Databases aren't happy if you apply too many
edits before issuing a commit. This isn't any different, except that on a non
transactional table, the commit isn't necessarily atomic. The other difference
with a typical transactional system is that you won't see your own updates
until you commit the data.
The doc says:
bq. When deleting a large data set, turn on autoCommit before issuing the
`DELETE` query so that the client does not need to remember the row keys of all
the keys as they are deleted.
Josh Elser says:
_Reasoning behind this one isn't clear to me. Batching DELETEs would have the
same benefit of batching UPSERTs, no? (I may just be missing an implementation
detail here..._
If you have auto commit off, then the DELETEs (i.e. the row keys of everything
returned by the statement if it was a SELECT instead of a DELETE) are buffered
on the client (much like executing UPSERT VALUE or UPSERT SELECT calls) until a
commit is performed. This is the only choice the driver has, since you haven't
told the driver to commit the data yet. If you have auto commit on, then
Phoenix is able to write the DELETEs directly on the region servers without
returning the rows back to the client (since it doesn't need to buffer them).
> First draft of Phoenix Tuning Guide
> -----------------------------------
>
> Key: PHOENIX-3218
> URL: https://issues.apache.org/jira/browse/PHOENIX-3218
> Project: Phoenix
> Issue Type: Improvement
> Reporter: Peter Conrad
> Attachments: Phoenix-Tuning-Guide-20170110.md,
> Phoenix-Tuning-Guide.md, Phoenix-Tuning-Guide.md
>
>
> Here's a first draft of a Tuning Guide for Phoenix performance.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
