[
https://issues.apache.org/jira/browse/HIVE-20147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542254#comment-16542254
]
Prasanth Jayachandran commented on HIVE-20147:
----------------------------------------------
If the clients of this API does not use async logging, there is high contention
in the logging done by the streaming API specifically logStats before/after
commit and close. Changed the log levels to DEBUG.
In one of the test application using async logging + this patch, the log
contention is no longer seen in the profiles.
> Hive streaming ingest is contented on synchronized logging
> ----------------------------------------------------------
>
> Key: HIVE-20147
> URL: https://issues.apache.org/jira/browse/HIVE-20147
> Project: Hive
> Issue Type: Bug
> Components: Streaming, Transactions
> Affects Versions: 4.0.0, 3.2.0
> Reporter: Prasanth Jayachandran
> Assignee: Prasanth Jayachandran
> Priority: Major
> Attachments: HIVE-20147.1.patch, Screen Shot 2018-07-11 at 4.17.27
> PM.png, sync-logger-contention.svg
>
>
> In one of the observed profile, >30% time spent on synchronized logging. See
> attachment.
> We should use async logging for hive streaming ingest by default. !Screen
> Shot 2018-07-11 at 4.17.27 PM.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
