Re: [Bacula-users] Restore job forward space volume - usb file volume

2023-08-25 Thread Martin Simmons 
> On Fri, 25 Aug 2023 13:22:47 -0400, Josh Fisher said:
> 
> On 8/25/23 12:06, Martin Simmons wrote:
> >> On Thu, 24 Aug 2023 15:51:18 -0400, Josh Fisher via Bacula-users said:
> >>
> >>Probably you have compression and/or encryption turned on. In
> >> that case, Bacula cannot simply fseek to the offset. It has to
> >> decompress and/or decrypt all data in order to find it, making restores
> >> far slower than backups.
> > The compression and/or encryption is done within each block, so tnat doesn't
> > affect seek time.
> 
> 
> Interesting. So after decompression and decryption, does the 
> uncompressed/decrypted data contain a partial block(s), or are the 
> compressed/encrypted blocks originally written with variable block sizes 
> so that the original data is handled as fixed size blocks?

Except for the last block in a backup, I think file volumes use a fixed block
size (the default is 64512 bytes), but it doesn't really matter.

The compression/encryption occurs within the records, which are then packed
into blocks.  If you run 'bls -v -v -k ..path.to.a.file.volume..' then it will
show the blocks and records like this:

Blk=0 blen=246 bVer=2 SessId=16 SessTim=1691737690
bls: block_util.c:105-0 Dump block  800d68200: adata=0 size=246 BlkNum=0
   Hdrcksum=8adb2881 cksum=8adb2881
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=VOL_LABEL Strm=0 
len=210 reclen=0
Blk=1 blen=64512 bVer=2 SessId=16 SessTim=1691737690
bls: block_util.c:105-0 Dump block  800d68200: adata=0 size=64512 BlkNum=1
   Hdrcksum=d01c4958 cksum=d01c4958
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=SOS_LABEL Strm=98043 
len=177 reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=1 Strm=UATTR len=102 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=1 Strm=GZIP len=226 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=1 Strm=MD5 len=16 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=2 Strm=UATTR len=104 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=2 Strm=GZIP len=1633 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=2 Strm=MD5 len=16 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=UATTR len=113 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=5748 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=5978 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=4677 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=4378 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=5937 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=4950 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=6003 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=5222 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=4809 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=5911 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=5435 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=5339 
reclen=0
Blk=2 blen=64512 bVer=2 SessId=16 SessTim=1691737690
bls: block_util.c:105-0 Dump block  800d68200: adata=0 size=64512 BlkNum=2
   Hdrcksum=d28c863e cksum=d28c863e
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=contGZIP 
len=2526 reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=5652 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=6024 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=6674 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=GZIP len=4829 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=3 Strm=MD5 len=16 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=4 Strm=UATTR len=121 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=4 Strm=GZIP len=19296 
reclen=0
bls: block_util.c:130-0Rec: VId=16 VT=1691737690 FI=4 Strm=GZIP len=23650 
reclen=0

__Martin


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Restore job forward space volume - usb file volume

2023-08-25 Thread Josh Fisher via Bacula-users 



On 8/25/23 12:06, Martin Simmons wrote:

On Thu, 24 Aug 2023 15:51:18 -0400, Josh Fisher via Bacula-users said:


   Probably you have compression and/or encryption turned on. In
that case, Bacula cannot simply fseek to the offset. It has to
decompress and/or decrypt all data in order to find it, making restores
far slower than backups.

The compression and/or encryption is done within each block, so tnat doesn't
affect seek time.



Interesting. So after decompression and decryption, does the 
uncompressed/decrypted data contain a partial block(s), or are the 
compressed/encrypted blocks originally written with variable block sizes 
so that the original data is handled as fixed size blocks?





__Martin



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Restore job forward space volume - usb file volume

2023-08-25 Thread Martin Simmons 
> On Thu, 24 Aug 2023 15:51:18 -0400, Josh Fisher via Bacula-users said:
> 
> On 8/24/23 05:32, Lionel PLASSE wrote:
> > Hello,
> >
> > For usb harddrive and file media volume,   when I do a restore job I get a 
> > long waiting  step : "Forward spacing Volume "T1" to addr=249122650420"
> > I remember I managed to configure the storage resource to quickly restore 
> > sdd drives.
> >
> > Should I use fastforward, blockpositionning and HardwareEndOfFile for USB 
> > disks and file volume?
> > How to avoid this long forwarding when not a tape device?
> 
> 
> I don't think any of those affect file type storage (random access 
> devices).

Agreed.

>   Probably you have compression and/or encryption turned on. In 
> that case, Bacula cannot simply fseek to the offset. It has to 
> decompress and/or decrypt all data in order to find it, making restores 
> far slower than backups.

The compression and/or encryption is done within each block, so tnat doesn't
affect seek time.

__Martin


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Restore job forward space volume - usb file volume

2023-08-25 Thread Martin Simmons 
> On Thu, 24 Aug 2023 09:32:38 +, Lionel PLASSE said:
> 
> Hello,
> 
> For usb harddrive and file media volume,   when I do a restore job I get a 
> long waiting  step : "Forward spacing Volume "T1" to addr=249122650420" 
> I remember I managed to configure the storage resource to quickly restore sdd 
> drives.
> 
> Should I use fastforward, blockpositionning and HardwareEndOfFile for USB 
> disks and file volume?
> How to avoid this long forwarding when not a tape device?

How do you know it is slow in the forwarding?  It might have done that quickly
and is slow in the next operation.

How much data are you restoring?

You can try using 'status storage' and 'status client' a few times in bconsole
to monitor the progress of the restore.

__Martin


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users