I recently upgraded to Hadoop 2.7.1 and HBase 1.1.2. Since the upgrade, the HBase master logs have been filling and rolling over about every 5 minutes, filled with variations of the following (modified to obscure internal details):
2015-12-27 12:24:52,481 DEBUG [SnapshotManifestLoader-pool878551-t3] snapshot.SnapshotManifestV1: Creating references for hfiles 2015-12-27 12:24:52,482 DEBUG [SnapshotManifestLoader-pool878551-t8] snapshot.SnapshotManifestV1: Storing region-info for snapshot. 2015-12-27 12:24:52,482 DEBUG [SnapshotManifestLoader-pool878551-t8] snapshot.SnapshotManifestV1: Creating references for hfiles 2015-12-27 12:24:52,482 DEBUG [SnapshotManifestLoader-pool878551-t6] snapshot.SnapshotManifestV1: Storing region-info for snapshot. 2015-12-27 12:24:52,482 DEBUG [SnapshotManifestLoader-pool878551-t6] snapshot.SnapshotManifestV1: Creating references for hfiles 2015-12-27 12:24:52,482 DEBUG [SnapshotManifestLoader-pool878551-t5] snapshot.SnapshotManifestV1: Adding snapshot references for [hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/efeae4f976778ed5b87fd53fd82f9f8f/p/b73e5e35e6cd4699bc48b67427173246 <hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/efeae4f976778ed5b87fd53fd82f9f8f/p/b73e5e35e6cd4699bc48b67427173246>] hfiles 2015-12-27 12:24:52,482 DEBUG [SnapshotManifestLoader-pool878551-t5] snapshot.SnapshotManifestV1: Adding reference for file (1/1): hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/efeae4f976778ed5b87fd53fd82f9f8f/p/b73e5e35e6cd4699bc48b67427173246 <hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/efeae4f976778ed5b87fd53fd82f9f8f/p/b73e5e35e6cd4699bc48b67427173246> 2015-12-27 12:24:52,482 DEBUG [SnapshotManifestLoader-pool878551-t7] snapshot.SnapshotManifestV1: Storing region-info for snapshot. 2015-12-27 12:24:52,482 DEBUG [SnapshotManifestLoader-pool878551-t7] snapshot.SnapshotManifestV1: Creating references for hfiles 2015-12-27 12:24:52,482 DEBUG [SnapshotManifestLoader-pool878551-t4] snapshot.SnapshotManifestV1: Adding snapshot references for [hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f061f0ad27a1bb55b0f00be2c78779fb/p/54f2f56e456d451a9a3601bc58540472 <hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f061f0ad27a1bb55b0f00be2c78779fb/p/54f2f56e456d451a9a3601bc58540472>, hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f061f0ad27a1bb55b0f00be2c78779fb/p/a75caa4c2da2485db941769be491792d <hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f061f0ad27a1bb55b0f00be2c78779fb/p/a75caa4c2da2485db941769be491792d>, hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f061f0ad27a1bb55b0f00be2c78779fb/p/ead9c7f82c554c93a8c7b3361e7550a4] <hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f061f0ad27a1bb55b0f00be2c78779fb/p/ead9c7f82c554c93a8c7b3361e7550a4]> hfiles 2015-12-27 12:24:52,482 DEBUG [SnapshotManifestLoader-pool878551-t4] snapshot.SnapshotManifestV1: Adding reference for file (1/3): hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f061f0ad27a1bb55b0f00be2c78779fb/p/54f2f56e456d451a9a3601bc58540472 <hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f061f0ad27a1bb55b0f00be2c78779fb/p/54f2f56e456d451a9a3601bc58540472> 2015-12-27 12:24:52,483 DEBUG [SnapshotManifestLoader-pool878551-t4] snapshot.SnapshotManifestV1: Adding reference for file (2/3): hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f061f0ad27a1bb55b0f00be2c78779fb/p/a75caa4c2da2485db941769be491792d <hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f061f0ad27a1bb55b0f00be2c78779fb/p/a75caa4c2da2485db941769be491792d> 2015-12-27 12:24:52,483 DEBUG [SnapshotManifestLoader-pool878551-t4] snapshot.SnapshotManifestV1: Adding reference for file (3/3): hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f061f0ad27a1bb55b0f00be2c78779fb/p/ead9c7f82c554c93a8c7b3361e7550a4 <hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f061f0ad27a1bb55b0f00be2c78779fb/p/ead9c7f82c554c93a8c7b3361e7550a4> 2015-12-27 12:24:52,483 DEBUG [SnapshotManifestLoader-pool878551-t3] snapshot.SnapshotManifestV1: Adding snapshot references for [hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f19b1b94666c38d6c3585b3 <hdfs://mycluster/apps/hbase/data/.hbase-snapshot/Foo-ru-20151223/f19b1b94666c38d6c3585b3> This is also causing my HDFS audit logs to grow at a rate of about 1.5MB/s. Has anyone observed this behavior or know a fix? Thanks Brian
