Hello. I would appreciate any help on this matter. Thanks On Wed, Sep 13, 2023 at 1:30 PM Mohammad Aghanabi <m.webs...@gmail.com> wrote:
> Hello. > > I read in a few articles like [1] that we can obtain data block stats from > "historical data access recorder from the NameNode log file" or in another > paper it's stated that frequent data blocks can be determined using > NameNode provided logs. > > I searched for related information on hadoop.apache.org but didn't find > anything. I read about job counters, fsimage, edit logs, audit logs... but > nothing related to a metric that represents "frequently accessed data > blocks" of DataNodes. > > I'd appreciate any help on whether this kind of stat is being collected by > a component or not. > > Thank you > > > [1] Jia-xuan Wu, Chang-sheng Zhang, Bin Zhang, Peng Wang, "A new > data-grouping-aware dynamic data placement method that take into account > jobs execute frequency for Hadoop", Microprocessors and Microsystems, > Volume 47, Part A, 2016, Pages 161-169 >
