----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71784/#review218679 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/hooks/HiveProtoLoggingHook.java Line 217 (original), 219 (patched) <https://reviews.apache.org/r/71784/#comment306546> how this can solve the issue ? Seems like it is doing the same thing `Executors.newSingleThreadScheduledExecutor` is the same as what you are doing ``` public static ScheduledExecutorService newSingleThreadScheduledExecutor(ThreadFactory threadFactory) { return new DelegatedScheduledExecutorService (new ScheduledThreadPoolExecutor(1, threadFactory)); } ``` ql/src/java/org/apache/hadoop/hive/ql/hooks/HiveProtoLoggingHook.java Lines 274 (patched) <https://reviews.apache.org/r/71784/#comment306547> looking at the java code i see that it is using a bounded queue so not sure what you mean by unbounded ? can you please clarify ? ql/src/java/org/apache/hadoop/hive/ql/hooks/HiveProtoLoggingHook.java Lines 279 (patched) <https://reviews.apache.org/r/71784/#comment306548> i am still not sure how this is going to work? the original code was dropping events when the queue is full that is the case where you see the `RejectedExecutionException` - Slim Bouguerra On Nov. 19, 2019, 3:43 p.m., Attila Magyar wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71784/ > ----------------------------------------------------------- > > (Updated Nov. 19, 2019, 3:43 p.m.) > > > Review request for hive, Laszlo Bodor, Harish Jaiprakash, Mustafa Iman, and > Panos Garefalakis. > > > Bugs: HIVE-22514 > https://issues.apache.org/jira/browse/HIVE-22514 > > > Repository: hive-git > > > Description > ------- > > HiveProtoLoggingHook uses a ScheduledThreadPoolExecutor to submit writer > tasks and to periodically handle rollover. The builtin > ScheduledThreadPoolExecutor uses a unbounded queue which cannot be replaced > from the outside. If log events are generated at a very fast rate this queue > can grow large. > > Since ScheduledThreadPoolExecutor does not support changing the default > unbounded queue to a bounded one, the queue capacity is checked manually by > the patch. > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a7687d59004 > ql/src/java/org/apache/hadoop/hive/ql/hooks/HiveProtoLoggingHook.java > 8eab54859bf > ql/src/test/org/apache/hadoop/hive/ql/hooks/TestHiveProtoLoggingHook.java > 450a0b544d6 > > > Diff: https://reviews.apache.org/r/71784/diff/1/ > > > Testing > ------- > > unittest > > > Thanks, > > Attila Magyar > >
