[
https://issues.apache.org/jira/browse/HUDI-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516187#comment-17516187
]
Alexey Kudinkin commented on HUDI-3774:
---------------------------------------
As could be seen from the dump retain HoodieLogFileReaders were reading
ColStats:
!image-2022-04-01-18-20-40-163.png!
Which is corroborated by CI logs mentioning HoodieLogFileReaders reading
ColStats logs in shutdown hooks exceptions:
{code:java}
2022-04-02T00:28:39.7628408Z java.lang.OutOfMemoryError: Java heap space
2022-04-02T00:28:39.7628972Z at
java.io.BufferedInputStream.<init>(BufferedInputStream.java:203)
2022-04-02T00:28:39.7629649Z at
org.apache.hadoop.fs.BufferedFSInputStream.<init>(BufferedFSInputStream.java:50)
2022-04-02T00:28:39.7630446Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.getFSDataInputStream(HoodieLogFileReader.java:483)
2022-04-02T00:28:39.7631224Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.<init>(HoodieLogFileReader.java:114)
2022-04-02T00:28:39.7631963Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.<init>(HoodieLogFileReader.java:103)
2022-04-02T00:28:39.7632698Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.<init>(HoodieLogFileReader.java:96)
2022-04-02T00:28:39.7633438Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.<init>(HoodieLogFileReader.java:91)
2022-04-02T00:28:39.7634171Z at
org.apache.hudi.common.table.log.HoodieLogFormat.newReader(HoodieLogFormat.java:277)
2022-04-02T00:28:39.7634928Z at
org.apache.hudi.common.table.TableSchemaResolver.readSchemaFromLogFile(TableSchemaResolver.java:515)
2022-04-02T00:28:39.7635770Z at
org.apache.hudi.client.functional.TestHoodieBackedMetadata.verifyMetadataColumnStatsRecords(TestHoodieBackedMetadata.java:2173)
2022-04-02T00:28:39.7636683Z at
org.apache.hudi.client.functional.TestHoodieBackedMetadata.lambda$validateMetadata$42(TestHoodieBackedMetadata.java:2159)
2022-04-02T00:28:39.7637490Z at
org.apache.hudi.client.functional.TestHoodieBackedMetadata$$Lambda$1539/899952308.accept(Unknown
Source)
2022-04-02T00:28:39.7638349Z at
java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:895)
2022-04-02T00:28:39.7639591Z at
org.apache.hudi.client.functional.TestHoodieBackedMetadata.validateMetadata(TestHoodieBackedMetadata.java:2144)
2022-04-02T00:28:39.7640466Z at
org.apache.hudi.client.functional.TestHoodieBackedMetadata.testTableOperationsImpl(TestHoodieBackedMetadata.java:1278)
2022-04-02T00:28:39.7641372Z at
org.apache.hudi.client.functional.TestHoodieBackedMetadata.testTableOperationsForMetaIndexImpl(TestHoodieBackedMetadata.java:447)
2022-04-02T00:28:39.7642292Z at
org.apache.hudi.client.functional.TestHoodieBackedMetadata.testTableOperationsWithMetadataIndex(TestHoodieBackedMetadata.java:442)
2022-04-02T00:28:39.7643208Z at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2022-04-02T00:28:39.7643829Z at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2022-04-02T00:28:39.7644539Z at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-04-02T00:28:39.7645163Z at
java.lang.reflect.Method.invoke(Method.java:498)
2022-04-02T00:28:39.7645806Z at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
2022-04-02T00:28:39.7646533Z at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
2022-04-02T00:28:39.7647338Z at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
2022-04-02T00:28:39.7648169Z at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
2022-04-02T00:28:39.7649633Z at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
2022-04-02T00:28:39.7650929Z at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:92)
2022-04-02T00:28:39.7669636Z at
org.junit.jupiter.engine.descriptor.TestTemplateInvocationTestDescriptor$$Lambda$896/427354807.apply(Unknown
Source)
2022-04-02T00:28:39.7670455Z at
org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
2022-04-02T00:28:39.7671232Z at
org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall$$Lambda$182/690339675.apply(Unknown
Source)
2022-04-02T00:28:39.7671899Z at
org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
2022-04-02T00:28:39.7672510Z at
org.junit.jupiter.engine.execution.ExecutableInvoker$$Lambda$333/2107443224.apply(Unknown
Source)
2022-04-02T00:28:39.7672794Z
2022-04-02T00:28:41.4231677Z 2058034 [Thread-53446] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.7_3-257-920',
fileLen=-1}
2022-04-02T00:28:41.4233522Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4234183Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4234861Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4235527Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4236309Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4236975Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4410326Z 2058034 [Thread-54236] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.3_3-58-218',
fileLen=-1}
2022-04-02T00:28:41.4427040Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4441860Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4442700Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4443811Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4444682Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4445416Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4447281Z 2058034 [Thread-54893] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.8_2-242-946',
fileLen=-1}
2022-04-02T00:28:41.4448457Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4448963Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4449489Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4450109Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4450779Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4451332Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4452527Z 2058034 [Thread-53569] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.6_2-199-757',
fileLen=-1}
2022-04-02T00:28:41.4453367Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4453834Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4454351Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4454896Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4455557Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4456114Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4457268Z 2058034 [Thread-54696] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.6_3-176-751',
fileLen=-1}
2022-04-02T00:28:41.4458075Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4458562Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4459080Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4459611Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4460286Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4460843Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4461975Z 2058034 [Thread-53875] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.2_3-25-120',
fileLen=-1}
2022-04-02T00:28:41.4462790Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4463272Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4463781Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4464327Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4464996Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4465531Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4466675Z 2058034 [Thread-54694] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.7_1-219-899',
fileLen=-1}
2022-04-02T00:28:41.4467602Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4468084Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4468587Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4469202Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4469875Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4470410Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4471567Z 2058034 [Thread-54234] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.4_1-103-440',
fileLen=-1}
2022-04-02T00:28:41.4472395Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4472862Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4473380Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4473928Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4474588Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4475136Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4476277Z 2058034 [Thread-53918] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.2_3-25-120',
fileLen=-1}
2022-04-02T00:28:41.4477096Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4477562Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4478547Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4479095Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4479753Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4480310Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4481464Z 2058034 [Thread-52837] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.2_3-25-120',
fileLen=-1}
2022-04-02T00:28:41.4482265Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4482743Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4483262Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4483793Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4484463Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4485014Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4486145Z 2058035 [Thread-54039] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.2_3-25-120',
fileLen=-1}
2022-04-02T00:28:41.4487295Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4487790Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4488412Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4488960Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4489630Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4490165Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4491407Z 2058034 [Thread-53216] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.2_3-25-120',
fileLen=-1}
2022-04-02T00:28:41.4492302Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4492784Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4493293Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4493838Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4494511Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4495047Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4496200Z 2058035 [Thread-53448] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.6_2-199-757',
fileLen=-1}
2022-04-02T00:28:41.4497026Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4497494Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4498008Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4498559Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4499214Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4499766Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4500961Z 2058035 [Thread-52950] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.3_2-58-220',
fileLen=-1}
2022-04-02T00:28:41.4501780Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4502246Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4502762Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4503309Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4503967Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4504519Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4505656Z 2058035 [Thread-54037] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.3_3-58-218',
fileLen=-1}
2022-04-02T00:28:41.4506455Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4506942Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4507458Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4507992Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4508667Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4509293Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4510434Z 2058035 [Thread-53573] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.4_3-103-443',
fileLen=-1}
2022-04-02T00:28:41.4511250Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4511732Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4512313Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4512845Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4513513Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4514060Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4515200Z 2058035 [Thread-53450] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.5_3-149-662',
fileLen=-1}
2022-04-02T00:28:41.4516018Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4516501Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4517003Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4517555Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4518410Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4518949Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4520105Z 2058035 [Thread-53571] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.5_3-149-662',
fileLen=-1}
2022-04-02T00:28:41.4520920Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4521402Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4521901Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4522447Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4523121Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4523657Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4524808Z 2058035 [Thread-54891] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.9_3-297-1111',
fileLen=-1}
2022-04-02T00:28:41.4525631Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4526101Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4526623Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4527170Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4527840Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4528382Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4606123Z 2058035 [Thread-52952] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.2_3-25-120',
fileLen=-1}
2022-04-02T00:28:41.4607598Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4608116Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4609709Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4610264Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4610942Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4611596Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4612817Z 2058034 [Thread-53567] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.7_3-257-920',
fileLen=-1}
2022-04-02T00:28:41.4613644Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4614117Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4614638Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4615189Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4615847Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4616405Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4617659Z 2058035 [Thread-54232] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.5_1-149-651',
fileLen=-1}
2022-04-02T00:28:41.4618476Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4618958Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4619467Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4620013Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4620682Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4621215Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4622362Z 2058035 [Thread-54238] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.2_3-25-120',
fileLen=-1}
2022-04-02T00:28:41.4623177Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4623659Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4624164Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4624714Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4625386Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4625919Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4627061Z 2058035 [Thread-53210] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.5_3-149-662',
fileLen=-1}
2022-04-02T00:28:41.4627881Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4628346Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4628864Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4629405Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4630142Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4630690Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4631839Z 2058035 [Thread-54895] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.7_1-219-899',
fileLen=-1}
2022-04-02T00:28:41.4632713Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4633175Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4633689Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4634233Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4634888Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4635442Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4636594Z 2058035 [Thread-54464] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.4_1-103-440',
fileLen=-1}
2022-04-02T00:28:41.4637396Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4637878Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4638806Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4639340Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4640010Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4640566Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4641719Z 2058035 [Thread-53452] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.4_3-103-443',
fileLen=-1}
2022-04-02T00:28:41.4642536Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4643016Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4643526Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4644072Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4644740Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4645274Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4646415Z 2058035 [Thread-54702] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.3_3-58-218',
fileLen=-1}
2022-04-02T00:28:41.4647230Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4647709Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4648213Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4648762Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4649429Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4649967Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4651109Z 2058035 [Thread-54466] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.3_3-58-218',
fileLen=-1}
2022-04-02T00:28:41.4652022Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4652497Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4653012Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4653558Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4654279Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4654831Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4657390Z 2058035 [Thread-53577] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.2_3-25-120',
fileLen=-1}
2022-04-02T00:28:41.4658225Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4658697Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4659215Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4659760Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4660417Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4660973Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4662118Z 2058035 [Thread-54700] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.4_1-103-440',
fileLen=-1}
2022-04-02T00:28:41.4662919Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4663398Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4663912Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4664446Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4665117Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4665664Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4666793Z 2058035 [Thread-53456] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.2_3-25-120',
fileLen=-1}
2022-04-02T00:28:41.4667611Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4668098Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4668603Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4669146Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4669814Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4670349Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4671485Z 2058035 [Thread-54468] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.2_3-25-120',
fileLen=-1}
2022-04-02T00:28:41.4672297Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4672774Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4673374Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4673923Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4674595Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4675131Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4676283Z 2058035 [Thread-54897] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.6_3-176-751',
fileLen=-1}
2022-04-02T00:28:41.4677163Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4677631Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4678332Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4678889Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4679542Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4680092Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4681260Z 2058035 [Thread-54692] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.8_2-242-946',
fileLen=-1}
2022-04-02T00:28:41.4682078Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4682543Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4683061Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4683593Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4684269Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4684819Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4685943Z 2058035 [Thread-54698] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.5_1-149-651',
fileLen=-1}
2022-04-02T00:28:41.4686749Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4687231Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4687747Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4688278Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4688947Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4689499Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4690629Z 2058035 [Thread-53212] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.4_3-103-443',
fileLen=-1}
2022-04-02T00:28:41.4691441Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4691920Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4692428Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4692976Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4693645Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4694177Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4695420Z 2058035 [Thread-52794] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.2_3-25-120',
fileLen=-1}
2022-04-02T00:28:41.4696228Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4696707Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4697211Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4697829Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4698501Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4699036Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4700185Z 2058035 [Thread-54460] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.6_3-176-751',
fileLen=-1}
2022-04-02T00:28:41.4700995Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4701462Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4701976Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4702520Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4703176Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4703725Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4704861Z 2058035 [Thread-53575] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.3_2-58-220',
fileLen=-1}
2022-04-02T00:28:41.4705677Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4706141Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4706655Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4707185Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4707852Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4708406Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4709528Z 2058035 [Thread-53454] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.3_2-58-220',
fileLen=-1}
2022-04-02T00:28:41.4710345Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4710823Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4711335Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4711867Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4712533Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4713084Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4714206Z 2058035 [Thread-53214] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit6911931651394939909/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.3_2-58-220',
fileLen=-1}
2022-04-02T00:28:41.4715012Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4715557Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4716061Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4716604Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4717268Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4717804Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4719179Z 2058035 [Thread-54462] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.5_1-149-651',
fileLen=-1}
2022-04-02T00:28:41.4719986Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4720464Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4720970Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4721516Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4722186Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4722719Z at java.lang.Thread.run(Thread.java:750)
2022-04-02T00:28:41.4723852Z 2058035 [Thread-54704] WARN
org.apache.hudi.common.table.log.HoodieLogFileReader - unable to close input
stream for log file
HoodieLogFile{pathStr='file:/tmp/junit5699402806563629075/dataset/.hoodie/metadata/column_stats/.col-stats-0000_00000000000000.log.2_3-25-120',
fileLen=-1}
2022-04-02T00:28:41.4724664Z java.lang.IllegalStateException: Shutdown in
progress
2022-04-02T00:28:41.4725130Z at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
2022-04-02T00:28:41.4725645Z at
java.lang.Runtime.removeShutdownHook(Runtime.java:239)
2022-04-02T00:28:41.4726196Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.close(HoodieLogFileReader.java:338)
2022-04-02T00:28:41.4726855Z at
org.apache.hudi.common.table.log.HoodieLogFileReader.lambda$addShutDownHook$0(HoodieLogFileReader.java:139)
2022-04-02T00:28:41.4727406Z at java.lang.Thread.run(Thread.java:750) {code}
> Fix memory leaks through shutdown hooks
> ---------------------------------------
>
> Key: HUDI-3774
> URL: https://issues.apache.org/jira/browse/HUDI-3774
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Alexey Kudinkin
> Assignee: Alexey Kudinkin
> Priority: Blocker
> Fix For: 0.11.0
>
> Attachments: Screen Shot 2022-04-01 at 11.58.14 AM.png,
> image-2022-04-01-18-20-40-163.png
>
>
> Our tests occasionally crash w/ OOM in Azure.
> I re-run the tests locally under the same conditions trying to reproduce it
> and took a few heap dumps.
> I see that out of about 1.1Gb about 272Mb are the HoodieLogFileReaders
> retained t/h shutdown hooks. This means that those HoodieLogFileReader has
> not been properly closed (potentially under conditions that there were some
> exceptions thrown)
>
> To make sure that all readers get closed upon reaching their end of life,
> we'd use try-resource-statement wherever possible, which will guarantee
> proper closing for the reader under any possible control-flow
> {code:java}
> try (Reader r = new Reader) {
> // ...
> }
> {code}
>
> !Screen Shot 2022-04-01 at 11.58.14 AM.png!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
