Hello,
I'm sorry, please disregard my last post (the two filesets are identical).
Following is a more detailed test (and this is not a cosmetic error of the
backup log):
1) compression works.
FileSet {
Name = "Full Set"
Include {
Options {
exclude = yes
wildfile = "*.data*"
}
Options {
signature = MD5
compression = GZIP
}
File = /opt/bacula/bin
}
Build OS: x86_64-unknown-linux-gnu ubuntu 14.04
JobId: 62
Job: BackupClient1.2016-03-26_01.37.17_04
Backup Level: Full
Client: "bacula-server-fd" 7.4.0 (16Jan16)
x86_64-unknown-linux-gnu,ubuntu,14.04
FileSet: "Full Set" 2016-02-19 23:35:17
...
FD Files Written: 17
SD Files Written: 17
FD Bytes Written: 7,015,847 (7.015 MB)
SD Bytes Written: 7,017,616 (7.017 MB)
Rate: 3507.9 KB/s
Software Compression: 58.0% 2.4:1
...
Volume name(s): Vol-0001
Volume Session Id: 1
Volume Session Time: 1458952340
Last Volume Bytes: 7,025,678 (7.025 MB)
...
Termination: Backup OK
-rw-r----- 1 root root 7025678 mar 26 01:37 Vol-0001
2) compression do not work:
FileSet {
Name = "Full Set"
Include {
Options {
signature = MD5
compression = GZIP
}
Options {
exclude = yes
wildfile = "*.data*"
}
File = /opt/bacula/bin
}
Build OS: x86_64-unknown-linux-gnu ubuntu 14.04
JobId: 63
Job: BackupClient1.2016-03-26_01.41.46_03
Backup Level: Full
Client: "bacula-server-fd" 7.4.0 (16Jan16)
x86_64-unknown-linux-gnu,ubuntu,14.04
FileSet: "Full Set" 2016-02-19 23:35:17
...
FD Files Written: 17
SD Files Written: 17
FD Bytes Written: 16,691,953 (16.69 MB)
SD Bytes Written: 16,693,466 (16.69 MB)
Rate: 16692.0 KB/s
Software Compression: None
...
Volume name(s): Vol-0001
Volume Session Id: 1
Volume Session Time: 1458952842
Last Volume Bytes: 16,706,736 (16.70 MB)
...
Termination: Backup OK
-rw-r----- 1 root root 16706736 mar 26 01:41 Vol-0001
3) compression works:
FileSet {
Name = "Full Set"
Include {
Options {
signature = MD5
compression = GZIP
}
Options {
exclude = yes
compression = GZIP
wildfile = "*.data*"
}
File = /opt/bacula/bin
}
Build OS: x86_64-unknown-linux-gnu ubuntu 14.04
JobId: 65
Job: BackupClient1.2016-03-26_01.50.53_03
Backup Level: Full
Client: "bacula-server-fd" 7.4.0 (16Jan16)
x86_64-unknown-linux-gnu,ubuntu,14.04
FileSet: "Full Set" 2016-02-19 23:35:17
...
FD Files Written: 17
SD Files Written: 17
FD Bytes Written: 7,015,847 (7.015 MB)
SD Bytes Written: 7,017,360 (7.017 MB)
Rate: 3507.9 KB/s
Software Compression: 58.0% 2.4:1
...
Volume name(s): Vol-0001
Volume Session Id: 1
Volume Session Time: 1458953395
Last Volume Bytes: 7,025,230 (7.025 MB)
...
Termination: Backup OK
-rw-r----- 1 root root 7025230 mar 26 01:50 Vol-0001
Notes:
- I created some *.data* files in /opt/bacula/bin for testing purposes
- all three Filesets correctly exclude the *.data* files
- I cannot see the need of a compressed clause in the options with the
exclude option set to yes. But as Heitor suggested, it works.
Randy, I have not run tests with your Fileset configuration. We should have
a close look in bacula client version, linux version, zlib devel version,
maybe verify carefully if there is really no difference in Fileset
configurations, check if without the exclude clause compression works and
finally check if the selected files for this "compression not working
client" can be compressed.
Best regards,
Ana
On Sat, Mar 26, 2016 at 12:44 AM, Randy Katz <rk...@simplicityhosting.com>
wrote:
> My expressions are default, here is mine:
>
> FileSet {
> Name = "client-28-filesetdefinition"
> Include {
> Options {
> signature = MD5
> compression = GZIP
> }
> File = /
> }
> Exclude {
> File = /tmp
> File = /proc
> File = /.journal
> File = /.fsck
> File = /home/virtfs
>
> }
> }
>
>
> On 3/25/2016 3:38 PM, Ana Emília M. Arruda wrote:
>
> Hello Ankush,
>
> The expression works correctly but for some reason, the following occurs:
>
> 1) Software compression none, FD and SD bytes written shows correct values
> with the bellow configuration:
>
> FileSet {
> Name = "chrome02 Set"
> Include {
> Options {
> exclude = yes
> wildfile = "*.data*"
> }
> Options {
> signature = MD5
> compression = GZIP
> }
> File = /rman
> }
>
>
> 2) Software compression shows correct value for compression, FD and SD
> bytes written shows incorrect values with the bellow configuration:
>
> FileSet {
> Name = "chrome02 Set"
> Include {
> Options {
> exclude = yes
> wildfile = "*.data*"
> }
> Options {
> signature = MD5
> compression = GZIP
> }
> File = /rman
> }
>
> The estimate listing command shows that both configurations
> work correctly. Maybe this is a cosmetic error.
>
> Best regards,
> Ana
>
>
> On Fri, Mar 25, 2016 at 6:21 PM, Ana Emília M. Arruda <
> <emiliaarr...@gmail.com>emiliaarr...@gmail.com> wrote:
>
>> Hello Ankush and Randy,
>>
>> It seems that there is a problem with the use of the wildfile expression.
>> Regardless the "*.data*" is correct (tested with bwild utility), it seems
>> that when this expression is used, the backup do not work correctly and the
>> result shows a "none" software compression.
>>
>> bacula-fd: job.c:1646-0 Fileset: WF *.data*
>> bacula-fd: job.c:1252-0 WF *.data*
>> bacula-fd: job.c:1280-0 Set state=error or double code.
>>
>> Could you please try this filset without the exclude option that excludes
>> "*.data*" (the same for Randy, I don't know if your client problem uses a
>> fileset like this one).
>>
>> Best regards,
>> Ana
>>
>> On Thu, Mar 24, 2016 at 9:18 PM, Randy Katz <
>> <rk...@simplicityhosting.com>rk...@simplicityhosting.com> wrote:
>>
>>> Yes, her client compilation confirmed compression for gzip and lzo...so
>>> now I would think there is an issue here. I also have one client out of
>>> around 40 that is not compressing. Dir and fd compiled from source version
>>> 7.4.0.
>>>
>>> On March 25, 2016 5:29:20 AM GMT+07:00, "Ana Emília M. Arruda" <
>>> <emiliaarr...@gmail.com>emiliaarr...@gmail.com> wrote:
>>>
>>>> Hello Ankush,
>>>>
>>>> As Heitor said in a previous post, zlib development libraries must also
>>>> be installed on clients. Have you checked this? Compression is working on
>>>> your bacula server (and client) for your catalog backup because the zlib
>>>> development libraries are installed on the bacula director host. Sorry if
>>>> you have already answered this, but I did not found in previous posts.
>>>>
>>>> Best regards,
>>>> Ana
>>>>
>>>> On Thu, Mar 24, 2016 at 5:45 PM, More, Ankush <
>>>> <ankush.m...@capgemini.com>ankush.m...@capgemini.com> wrote:
>>>>
>>>>> Hello Randy,
>>>>>
>>>>> Bacula server(7.x) is installed with Linux RPM package.
>>>>>
>>>>> Catalog backup on bacula server work with compression = GZIP option.
>>>>>
>>>>> Job log:
>>>>>
>>>>> Software Compression: 84.9% 6.6:1
>>>>>
>>>>> But clients does not work
>>>>>
>>>>> Job log:
>>>>>
>>>>> Software Compression: None
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Ankush More
>>>>>
>>>>>
>>>>>
>>>>> *From:* Randy Katz [mailto: <rk...@simplicityhosting.com>
>>>>> rk...@simplicityhosting.com]
>>>>> *Sent:* Thursday, March 24, 2016 8:10 PM
>>>>> *To:* More, Ankush
>>>>> *Subject:* Re: [Bacula-users] Client backup with compress
>>>>>
>>>>>
>>>>>
>>>>> Then the only other thing I could think of is if the director is
>>>>> compiled with
>>>>> the gzip/lzo libraries, have you recompiled that? Should not be too
>>>>> hard, you don't
>>>>> need that long configure line, if using mysql all you need is:
>>>>>
>>>>> ./configure --with-mysql
>>>>>
>>>>>
>>>>> On 3/24/2016 6:54 AM, More, Ankush wrote:
>>>>>
>>>>> Hi Randy,
>>>>>
>>>>>
>>>>>
>>>>> Backing up are Database files which has very high compression ratio.
>>>>>
>>>>> I have check same files with linux command ‘gzip’ on client working
>>>>> fine with good compression ratio.
>>>>>
>>>>> But Bacula does not work.
>>>>>
>>>>>
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Ankush More
>>>>>
>>>>>
>>>>>
>>>>> *From:* Randy Katz [ <rk...@simplicityhosting.com>
>>>>> mailto:rk...@simplicityhosting.com <rk...@simplicityhosting.com>]
>>>>> *Sent:* Thursday, March 24, 2016 1:18 PM
>>>>> *To:* <bacula-users@lists.sourceforge.net>
>>>>> bacula-users@lists.sourceforge.net
>>>>> *Subject:* Re: [Bacula-users] Client backup with compress
>>>>>
>>>>>
>>>>>
>>>>> Hi Ankush, are you sure the files you are backing up can be
>>>>> compressed? You might
>>>>> want to try a separate backup on the same client of something like
>>>>> /usr/local /etc
>>>>> just to test, these should definitely compress. Sometimes video, audio
>>>>> and other dense
>>>>> files do not compress.
>>>>>
>>>>> On 3/23/2016 11:44 AM, More, Ankush wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>>
>>>>>
>>>>> I re-compile client with LZO package.
>>>>>
>>>>> Compression does not work both GZIP or LZO.
>>>>>
>>>>>
>>>>>
>>>>> Large file support: yes
>>>>>
>>>>> Bacula conio support: yes -lncurses
>>>>>
>>>>> readline support: no
>>>>>
>>>>> TCP Wrappers support: no
>>>>>
>>>>> TLS support: yes
>>>>>
>>>>> Encryption support: yes
>>>>>
>>>>> ZLIB support: yes
>>>>>
>>>>> LZO support: yes
>>>>>
>>>>> enable-smartalloc: yes
>>>>>
>>>>> enable-lockmgr: no
>>>>>
>>>>> bat support: no
>>>>>
>>>>> client-only: yes
>>>>>
>>>>> build-dird: no
>>>>>
>>>>> build-stored: no
>>>>>
>>>>> Plugin support: yes
>>>>>
>>>>> AFS support: no
>>>>>
>>>>> ACL support: yes
>>>>>
>>>>> XATTR support: yes
>>>>>
>>>>> systemd support: no
>>>>>
>>>>> Batch insert enabled: None
>>>>>
>>>>>
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Ankush More
>>>>>
>>>>>
>>>>>
>>>>> *From:* More, Ankush
>>>>> *Sent:* Wednesday, March 23, 2016 8:46 PM
>>>>> *To:* 'Heitor Faria'
>>>>> *Cc:* <comp...@hotrodpc.com>comp...@hotrodpc.com;
>>>>> <bacula-users@lists.sourceforge.net>bacula-users@lists.sourceforge.net
>>>>> *Subject:* RE: [Bacula-users] Client backup with compress
>>>>>
>>>>>
>>>>>
>>>>> Hello Faria,
>>>>>
>>>>>
>>>>>
>>>>> Sorry for confusion , please ignore previous mail. I have re-compile
>>>>> the client with source which version 7.x on Linux.
>>>>>
>>>>> Resending debug –d500 output for same.
>>>>>
>>>>>
>>>>>
>>>>> Which support ZLIB, but still compression is not working.
>>>>>
>>>>>
>>>>>
>>>>> ZLIB support: yes
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Ankush More
>>>>>
>>>>>
>>>>>
>>>>> *From:* Heitor Faria [ <hei...@bacula.com.br>
>>>>> mailto:hei...@bacula.com.br <hei...@bacula.com.br>]
>>>>> *Sent:* Wednesday, March 23, 2016 8:31 PM
>>>>> *To:* More, Ankush
>>>>> *Cc:* <comp...@hotrodpc.com>comp...@hotrodpc.com;
>>>>> <bacula-users@lists.sourceforge.net>bacula-users@lists.sourceforge.net
>>>>> *Subject:* Re: [Bacula-users] Client backup with compress
>>>>>
>>>>>
>>>>>
>>>>> Hello, Akush: according to the debug you sent me this is the FileSet
>>>>> you are using for this job:
>>>>>
>>>>>
>>>>>
>>>>> bacula-fd: job.c:273-415 Executing fileset command.
>>>>> bacula-fd: job.c:753-415 I
>>>>> bacula-fd: job.c:753-415 O MZ6
>>>>> bacula-fd: job.c:753-415 N
>>>>> bacula-fd: job.c:753-415 O e
>>>>> bacula-fd: job.c:753-415 WF *.apm
>>>>> bacula-fd: job.c:781-415 Set state=error or double code.
>>>>> bacula-fd: job.c:753-415 N
>>>>> bacula-fd: job.c:753-415 F /oracle
>>>>> bacula-fd: job.c:753-415 N
>>>>> bacula-fd: job.c:753-415 N
>>>>> bacula-fd: job.c:753-415 E
>>>>> bacula-fd: job.c:753-415 F /var/spool/bacula
>>>>> bacula-fd: job.c:753-415 F /tmp
>>>>> bacula-fd: job.c:753-415 F /proc
>>>>> bacula-fd: job.c:753-415 F /tmp
>>>>> bacula-fd: job.c:753-415 F /sys
>>>>> bacula-fd: job.c:753-415 F /.journal
>>>>> bacula-fd: job.c:753-415 F /.fsck
>>>>> bacula-fd: job.c:753-415 N
>>>>>
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> *From: *"More, Ankush" < <ankush.m...@capgemini.com>
>>>>> ankush.m...@capgemini.com>
>>>>> *To: *"Heitor Faria" < <hei...@bacula.com.br>hei...@bacula.com.br>
>>>>> *Cc: * <comp...@hotrodpc.com>comp...@hotrodpc.com,
>>>>> <bacula-users@lists.sourceforge.net>bacula-users@lists.sourceforge.net
>>>>> *Sent: *Wednesday, March 23, 2016 11:57:12 AM
>>>>> *Subject: *RE: [Bacula-users] Client backup with compress
>>>>>
>>>>> Hello Faria,
>>>>>
>>>>>
>>>>>
>>>>> Below is DIR config file.
>>>>>
>>>>>
>>>>>
>>>>> FileSet {
>>>>>
>>>>> Name = "chrome02 Set"
>>>>>
>>>>> Include {
>>>>>
>>>>> Options {
>>>>>
>>>>> signature = MD5
>>>>>
>>>>> compression = GZIP
>>>>>
>>>>> }
>>>>>
>>>>>
>>>>>
>>>>> Not sure what you mean Compression=gzip in all of them.
>>>>>
>>>>>
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Ankush More
>>>>>
>>>>>
>>>>>
>>>>> *From:* Heitor Faria [ <hei...@bacula.com.br>
>>>>> mailto:hei...@bacula.com.br <hei...@bacula.com.br>]
>>>>> *Sent:* Wednesday, March 23, 2016 8:14 PM
>>>>> *To:* More, Ankush
>>>>> *Cc:* <comp...@hotrodpc.com>comp...@hotrodpc.com;
>>>>> <bacula-users@lists.sourceforge.net>bacula-users@lists.sourceforge.net
>>>>> *Subject:* Re: [Bacula-users] Client backup with compress
>>>>>
>>>>>
>>>>>
>>>>> Hello, Ankush: ZLIB support: yes
>>>>>
>>>>> Your client is fine. It must be a FileSet configuration thing, since
>>>>> you seems to have multiple Options sub-resources. Just put
>>>>> Compression=gzip
>>>>> in all of them.
>>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>> ------------------------------
>>>>>
>>>>> *From: *"More, Ankush" < <ankush.m...@capgemini.com>
>>>>> ankush.m...@capgemini.com>
>>>>> *To: * <comp...@hotrodpc.com>comp...@hotrodpc.com,
>>>>> <bacula-users@lists.sourceforge.net>bacula-users@lists.sourceforge.net
>>>>> *Sent: *Wednesday, March 23, 2016 11:32:34 AM
>>>>> *Subject: *Re: [Bacula-users] Client backup with compress
>>>>>
>>>>> Hello,
>>>>>
>>>>>
>>>>>
>>>>> Please see below, this is build for bacula client.
>>>>>
>>>>>
>>>>>
>>>>> Configuration on Wed Mar 23 04:37:28 EDT 2016:
>>>>>
>>>>>
>>>>>
>>>>> Host: x86_64-unknown-linux-gnu -- redhat
>>>>> Enterprise release
>>>>>
>>>>> Bacula version: Bacula 7.0.4 (04 June 2014)
>>>>>
>>>>> Source code location: .
>>>>>
>>>>> Install binaries: /usr/sbin
>>>>>
>>>>> Install libraries: /usr/lib64
>>>>>
>>>>> Install config files: /etc/bacula
>>>>>
>>>>> Scripts directory: /usr/libexec/bacula/
>>>>>
>>>>> Archive directory: /tmp
>>>>>
>>>>> Working directory: /var/spool/bacula/
>>>>>
>>>>> PID directory: /var/run
>>>>>
>>>>> Subsys directory: /var/lock/subsys
>>>>>
>>>>> Man directory: ${datarootdir}/man
>>>>>
>>>>> Data directory: /usr/share
>>>>>
>>>>> Plugin directory: /usr/lib64
>>>>>
>>>>> C Compiler: gcc 4.1.2
>>>>>
>>>>> C++ Compiler: /usr/bin/g++ 4.1.2
>>>>>
>>>>> Compiler flags: -g -O2 -Wall -fno-strict-aliasing
>>>>> -fno-exceptions -fno-rtti
>>>>>
>>>>> Linker flags:
>>>>>
>>>>> Libraries: -lpthread -ldl -ldl
>>>>>
>>>>> Statically Linked Tools: no
>>>>>
>>>>> Statically Linked FD: no
>>>>>
>>>>> Statically Linked SD: no
>>>>>
>>>>> Statically Linked DIR: no
>>>>>
>>>>> Statically Linked CONS: no
>>>>>
>>>>> Database backends: None
>>>>>
>>>>> Database port:
>>>>>
>>>>> Database name: bacula
>>>>>
>>>>> Database user: bacula
>>>>>
>>>>>
>>>>>
>>>>> Job Output Email: root@localhost
>>>>>
>>>>> Traceback Email: root@localhost
>>>>>
>>>>> SMTP Host Address: localhost
>>>>>
>>>>>
>>>>>
>>>>> Director Port: 9101
>>>>>
>>>>> File daemon Port: 9102
>>>>>
>>>>> Storage daemon Port: 9103
>>>>>
>>>>>
>>>>>
>>>>> Director User:
>>>>>
>>>>> Director Group:
>>>>>
>>>>> Storage Daemon User:
>>>>>
>>>>> Storage DaemonGroup:
>>>>>
>>>>> File Daemon User:
>>>>>
>>>>> File Daemon Group:
>>>>>
>>>>>
>>>>>
>>>>> Large file support: yes
>>>>>
>>>>> Bacula conio support: yes -lncurses
>>>>>
>>>>> readline support: no
>>>>>
>>>>> TCP Wrappers support: no
>>>>>
>>>>> TLS support: yes
>>>>>
>>>>> Encryption support: yes
>>>>>
>>>>> ZLIB support: yes
>>>>>
>>>>> LZO support: no
>>>>>
>>>>> enable-smartalloc: yes
>>>>>
>>>>> enable-lockmgr: no
>>>>>
>>>>> bat support: no
>>>>>
>>>>> client-only: yes
>>>>>
>>>>> build-dird: no
>>>>>
>>>>> build-stored: no
>>>>>
>>>>> Plugin support: yes
>>>>>
>>>>> AFS support: no
>>>>>
>>>>> ACL support: yes
>>>>>
>>>>> XATTR support: yes
>>>>>
>>>>> systemd support: no
>>>>>
>>>>> Batch insert enabled: None
>>>>>
>>>>>
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Ankush More
>>>>>
>>>>>
>>>>>
>>>>> *From:* compdoc [ <comp...@hotrodpc.com>mailto:comp...@hotrodpc.com
>>>>> <comp...@hotrodpc.com>]
>>>>> *Sent:* Wednesday, March 23, 2016 7:48 PM
>>>>> *To:* <bacula-users@lists.sourceforge.net>
>>>>> bacula-users@lists.sourceforge.net
>>>>> *Subject:* Re: [Bacula-users] Client backup with compress
>>>>>
>>>>>
>>>>>
>>>>> >#./configure --enable-client-only --enable-build-dird=no
>>>>> --enable-build-stored=no --bindir=/usr/bin --sbindir=/usr/sbin
>>>>> --with-scriptdir=/usr/libexec/bacula/
>>>>> --with-working-dir=/var/spool/bacula/
>>>>> --with-logdir=/var/log --enable-smartalloc
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ...
>
> [Message clipped]
>
> ------------------------------------------------------------------------------
> Transform Data into Opportunity.
> Accelerate data analysis in your applications with
> Intel Data Analytics Acceleration Library.
> Click to learn more.
> http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
> _______________________________________________
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
