Re: [gpfsug-discuss] mmbackup file selections

2022-01-26 Thread Skylar Thompson
Awesome, glad that you found them (I missed them the first time too).

As for the anomalous changed files, do you have these options set in your
client option file?

skipacl yes
skipaclupdatecheck yes
updatectime yes

We had similar problems where metadata and ACL updates were interpreted as
data changes by mmbackup/dsmc.

We also have a case open with IBM where mmbackup will both expire and
backup a file in the same run, even in the absence of mtime changes, but
it's unclear whether that's program error or something with our
include/exclude rules. I'd be curious if you're running into that as
well.

On Wed, Jan 26, 2022 at 03:55:48PM +, Paul Ward wrote:
> Good call!
> 
> Yes they are dot files.
> 
> 
> New issue.
> 
> Mmbackup seems to be backup up the same files over and over without them 
> changing:
> areas are being backed up multiple times.
> The example below is a co-resident file, the only thing that has changed 
> since it was created 20/10/21, is the file has been accessed for backup.
> This file is in the 'changed' list in mmbackup:
> 
> This list has just been created:
> -rw-r--r--. 1 root root 6591914 Jan 26 11:12 
> mmbackupChanged.ix.197984.22A38AA7.39.nhmfsa
> 
> Listing the last few files in the file (selecting the last one)
> 11:17:52 [root@scale-sk-pn-1 .mmbackupCfg]# tail 
> mmbackupChanged.ix.197984.22A38AA7.39.nhmfsa
> "/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604556977.png"
> "/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557039.png"
> "/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557102.png"
> "/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557164.png"
> "/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557226.png"
> "/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557288.png"
> "/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557351.png"
> "/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557413.png"
> "/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557476.png"
> "/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png"
> 
> Check the file stats (access time just before last backup)
> 11:18:05 [root@scale-sk-pn-1 .mmbackupCfg]# stat 
> "/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png"
>   File: 
> '/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png'
>   Size: 545 Blocks: 32 IO Block: 4194304 regular file
> Device: 2bh/43d Inode: 212618897   Links: 1
> Access: (0644/-rw-r--r--)  Uid: (1399613896/NHM\edwab)   Gid: 
> (1399647564/NHM\dg-mbl-urban-nature-project-rw)
> Context: unconfined_u:object_r:unlabeled_t:s0
> Access: 2022-01-25 06:40:58.334961446 +
> Modify: 2020-12-01 15:20:40.122053000 +
> Change: 2021-10-20 17:55:18.265746459 +0100
> Birth: -
> 
> Check if migrated
> 11:18:16 [root@scale-sk-pn-1 .mmbackupCfg]# dsmls 
> "/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png"
> File name   : 
> /gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png
> On-line size: 545
> Used blocks : 16
> Data Version: 1
> Meta Version: 1
> State   : Co-resident
> Container Index : 1
> Base Name   : 
> 34C0B77D20194B0B.EACEB2055F6CAA58.56D56C5F140C8C9D..2197396D.0CAC4E91
> 
> Check if immutable
> 11:18:26 [root@scale-sk-pn-1 .mmbackupCfg]# mstat 
> "/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png"
> file name:
> /gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png
> metadata replication: 2 max 2
> data replication: 2 max 2
> immutable:no
> appendOnly:   no
> flags:
> storage pool name:data
> fileset name: hpc-workspaces-fset
> snapshot name:
> creation time:Wed Oct 20 17:55:18 2021
> Misc attributes:  ARCHIVE
> Encrypted:no
> 
> Check active and inactive backups (it was backed up yesterday)
> 11:18:52 [root@scale-sk-pn-1 .mmbackupCfg]# dsmcqbi 
> 

Re: [gpfsug-discuss] mmbackup file selections

2022-01-26 Thread Paul Ward
Good call!

Yes they are dot files.


New issue.

Mmbackup seems to be backup up the same files over and over without them 
changing:
areas are being backed up multiple times.
The example below is a co-resident file, the only thing that has changed since 
it was created 20/10/21, is the file has been accessed for backup.
This file is in the 'changed' list in mmbackup:

This list has just been created:
-rw-r--r--. 1 root root 6591914 Jan 26 11:12 
mmbackupChanged.ix.197984.22A38AA7.39.nhmfsa

Listing the last few files in the file (selecting the last one)
11:17:52 [root@scale-sk-pn-1 .mmbackupCfg]# tail 
mmbackupChanged.ix.197984.22A38AA7.39.nhmfsa
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604556977.png"
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557039.png"
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557102.png"
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557164.png"
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557226.png"
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557288.png"
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557351.png"
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557413.png"
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557476.png"
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png"

Check the file stats (access time just before last backup)
11:18:05 [root@scale-sk-pn-1 .mmbackupCfg]# stat 
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png"
  File: 
'/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png'
  Size: 545 Blocks: 32 IO Block: 4194304 regular file
Device: 2bh/43d Inode: 212618897   Links: 1
Access: (0644/-rw-r--r--)  Uid: (1399613896/NHM\edwab)   Gid: 
(1399647564/NHM\dg-mbl-urban-nature-project-rw)
Context: unconfined_u:object_r:unlabeled_t:s0
Access: 2022-01-25 06:40:58.334961446 +
Modify: 2020-12-01 15:20:40.122053000 +
Change: 2021-10-20 17:55:18.265746459 +0100
Birth: -

Check if migrated
11:18:16 [root@scale-sk-pn-1 .mmbackupCfg]# dsmls 
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png"
File name   : 
/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png
On-line size: 545
Used blocks : 16
Data Version: 1
Meta Version: 1
State   : Co-resident
Container Index : 1
Base Name   : 
34C0B77D20194B0B.EACEB2055F6CAA58.56D56C5F140C8C9D..2197396D.0CAC4E91

Check if immutable
11:18:26 [root@scale-sk-pn-1 .mmbackupCfg]# mstat 
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png"
file name:
/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png
metadata replication: 2 max 2
data replication: 2 max 2
immutable:no
appendOnly:   no
flags:
storage pool name:data
fileset name: hpc-workspaces-fset
snapshot name:
creation time:Wed Oct 20 17:55:18 2021
Misc attributes:  ARCHIVE
Encrypted:no

Check active and inactive backups (it was backed up yesterday)
11:18:52 [root@scale-sk-pn-1 .mmbackupCfg]# dsmcqbi 
"/gpfs/nhmfsa/bulk/share/data/mbl/share/workspaces/groups/urban-nature-project/audiowaveform/300_40/unp-grounds-01-1604557538.png"
IBM Spectrum Protect
Command Line Backup-Archive Client Interface
  Client Version 8, Release 1, Level 10.0
  Client date/time: 01/26/2022 11:19:02
(c) Copyright by IBM Corporation and other(s) 1990, 2020. All Rights Reserved.

Node Name: SC-PN-SK-01
Session established with server TSM-JERSEY: Windows
  Server Version 8, Release 1, Level 10.100
  Server date/time: 01/26/2022 11:19:02  Last access: 01/26/2022 11:07:05

Accessing as node: SCALE
   SizeBackup DateMgmt Class   A/I File
   -----   --- 
   545  B  01/25/2022 06:41:17 DEFAULT  A