Hi everyone!
We've been investigating some unpredictably long GC pauses in Hadoop Datanode (2.5.1, Oracle JRE8_25) and noticed strange behaviour in DataBlockScanner. DataBlockScanner initiates block verification tasks during DN startup and every 504 hours after. It holds some structures in memory heap (HashMap, TreeSet & LightWeightGSet) which, in our case, are around 330Mb total (over 1.5m blocks). Are these structures useful after block verification task completes? Tenuring and promoting these objects to OldGen causes some terrible STW pauses we would like to avoid. Kind regards, Sergey Kazakov
