ly923976094 opened a new issue #3874: consumer out of memory URL: https://github.com/apache/incubator-pinot/issues/3874 When the consumer is consuming, when the state is transitioning, memory out of memory, consumption stops. What's going on here server configuration（6+32G, jvm16G） java.lang.reflect.InvocationTargetException at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:430) at org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:344) at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:96) at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:47) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.OutOfMemoryError: Direct buffer memory at java.nio.Bits.reserveMemory(Bits.java:693) at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) at com.linkedin.pinot.core.segment.memory.PinotByteBuffer.allocateDirect(PinotByteBuffer.java:36) at com.linkedin.pinot.core.segment.memory.PinotByteBuffer.loadFile(PinotByteBuffer.java:40) at com.linkedin.pinot.core.segment.memory.PinotDataBuffer.loadFile(PinotDataBuffer.java:144) at com.linkedin.pinot.core.segment.store.SingleFileIndexDirectory.mapAndSliceFile(SingleFileIndexDirectory.java:284) at com.linkedin.pinot.core.segment.store.SingleFileIndexDirectory.mapBufferEntries(SingleFileIndexDirectory.java:265) at com.linkedin.pinot.core.segment.store.SingleFileIndexDirectory.load(SingleFileIndexDirectory.java:190) at com.linkedin.pinot.core.segment.store.SingleFileIndexDirectory.<init>(SingleFileIndexDirectory.java:86) at com.linkedin.pinot.core.segment.store.SegmentLocalFSDirectory.loadData(SegmentLocalFSDirectory.java:198) at com.linkedin.pinot.core.segment.store.SegmentLocalFSDirectory.load(SegmentLocalFSDirectory.java:179) at com.linkedin.pinot.core.segment.store.SegmentLocalFSDirectory.<init>(SegmentLocalFSDirectory.java:77) at com.linkedin.pinot.core.segment.store.SegmentDirectory.createFromLocalFS(SegmentDirectory.java:108) at com.linkedin.pinot.core.indexsegment.immutable.ImmutableSegmentLoader.load(ImmutableSegmentLoader.java:108) at com.linkedin.pinot.core.data.manager.realtime.RealtimeTableDataManager.replaceLLSegment(RealtimeTableDataManager.java:291) at com.linkedin.pinot.core.data.manager.realtime.RealtimeTableDataManager.downloadAndReplaceSegment(RealtimeTableDataManager.java:279) at com.linkedin.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.downloadSegmentAndReplace(LLRealtimeSegmentDataManager.java:945) at com.linkedin.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.goOnlineFromConsuming(LLRealtimeSegmentDataManager.java:934) at com.linkedin.pinot.server.starter.helix.SegmentOnlineOfflineStateModelFactory$SegmentOnlineOfflineStateModel.onBecomeOnlineFromConsuming(SegmentOnlineOfflineStateModelFactory.java:118)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
