[Bacula-users] Found: IO waiting bacula-sd process, Re: Migration from tape to disk is hanging
Am 16.07.21 um 13:42 schrieb Pierre Bernhardt: It was hard to find the reason, but it looks like on one tape on special position the bacula-sd process is waiting for a non-interruptible IO from the drive. The interesting point is, that it is the penultimate job of the tape. The situation was so hard, if the bacula-sd would be restarted the process is left as a zombie process (State Z) so only a full reboot of the system helps to fix the issue. I migrated all other jobs from this tape to another one (via a disk migration step) so only the problem job is still active on this tape. I hope I won't need this weekly backup job so in one year I will put the tape in the trash bin. So the reason is found and I don't want to find the exact reason why the bacula-sd process never comes back. But it is reproducible so I can make investigations more in detail if it is needed. PS: Normally I mean a problem should never bring the system in such a state. The process should die by itself in such a situation e.b. by a timeout or something else and leave the system intact. Cheers, Pierre ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] S3 cloud plugin does not load on FreeBSD 13.0
Hello All,
>> The error in bacula-sd I am getting is the following:
>> > my-sd: init_dev.c:488-0 Driver=cloud entry point=800d4fe60
>> > my-sd: cloud_dev.c:876-0 Open SD driver at
>> > /usr/local/lib/bacula-sd-cloud-s3-driver-11.0.5.so
>> > my-sd: cloud_dev.c:879-0 Driver=s3 handle=800d19408
>> > my-sd: cloud_dev.c:881-0 Lookup "BaculaCloudDriver" in driver=s3
>> > my-sd: cloud_dev.c:883-0 Driver=s3 entry point=0
>> > my-sd: cloud_dev.c:889-0 Lookup of symbol "BaculaCloudDriver"
>> > driver=/usr/local/lib/bacula-sd-cloud-s3-driver-11.0.5.so failed:
>> > ERR=Undefined symbol "BaculaCloudDriver"
> This line above ...
>> > my-sd: message.c:1831-0 cloud_dev.c:1001 Could not open Cloud driver
>> > type=1 for Device=CloudStorage.
>> > my-sd: message.c:1831-0 cloud_dev.c:1032 Cloud driver initialization
>> > error Cloud driver not properly loaded
>> > my-sd: htable.c:67-0 malloc buf=801c71ae8 size=100 rem=76
>> > my-sd: init_dev.c:235-0 init_dev allocated: 801c1cc28
>> > my-sd: init_dev.c:437-0 init_dev: tape=0
>> > dev_name=/backup/backupdisk1/bacula-storage
>> > my-sd: dev.c:1143-0 DEVICE::register_metrics called. 0x801c1cc28
>> > collector=0x80101c938
>> > my-sd: stored.c:706-0 SD init done CloudStorage (0x801c1cc28)
>> I comppiled the cloud-driver and the cloud-s3-driver with the following
>> commands:
>> > $ /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/src/stored
>> > > make install-cloud
>> > LT Compiling cloud_parts.c
>> > LT Compiling cloud_transfer_mgr.c
>> > LT Compiling cloud_dev.c
>> > cloud_dev.c:1818:17: warning: address of array 'tpkt->m_hash64' will
>> > always evaluate to 'true' [-Wpointer-bool-conversion]
>> > if (tpkt->m_hash64) {
>> > ~~ ~~^~~~
>> > 1 warning generated.
>> > LT Compiling file_driver.c
>> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool
>> > --silent --tag=CXX --mode=link /usr/bin/c++ -L/usr/local/lib
>> > -L/usr/local/lib -L/usr/local/lib -fstack-protector-strong -shared
>> > cloud_parts.lo cloud_transfer_mgr.lo cloud_dev.lo file_driver.lo -o
>> > bacula-sd-cloud-driver.la -R /usr/local/lib -rpath /usr/local/lib
>> > -module -export-dynamic -release 11.0.5
>> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/autoconf/mkinstalldirs
>> > /usr/local/lib
>> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool
>> > --silent --tag=CXX --mode=install /usr/bin/install -c -m 755
>> > bacula-sd-cloud-driver.la /usr/local/lib
>> > /bin/rm -f /usr/local/lib/bacula-sd-cloud-driver.la
>> > $ /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/src/stored
>> > > make install-s3-cloud
>> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/autoconf/mkinstalldirs
>> > /usr/local/lib
>> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool
>> > --silent --tag=CXX --mode=install /usr/bin/install -c -m 755
>> > bacula-sd-cloud-driver.la /usr/local/lib
>> > /bin/rm -f /usr/local/lib/bacula-sd-cloud-driver.la
>> > LT Compiling s3_driver.c
>> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool
>> > --silent --tag=CXX --mode=link /usr/bin/c++ -L/usr/local/lib
>> > -L/usr/local/lib -L/usr/local/lib -fstack-protector-strong -shared
>> > cloud_parts.lo cloud_transfer_mgr.lo s3_driver.lo -o
>> > bacula-sd-cloud-s3-driver.la -R /usr/local/lib -rpath /usr/local/lib
>> > -module -export-dynamic -release 11.0.5
>> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/autoconf/mkinstalldirs
>> > /usr/local/lib
>> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool
>> > --silent --tag=CXX --mode=install /usr/bin/install -c -m 755
>> > bacula-sd-cloud-s3-driver.la /usr/local/lib
>> > /bin/rm -f /usr/local/lib/bacula-sd-cloud-s3-driver.la
>> But when I start bacula-sd with bacula-sd -d 255, I am getting the above
>> > ERR=Undefined symbol "BaculaCloudDriver"
>> error message.
> Matches this line. My thoughts: don't try to start bacula-sd until that
> compile
> time error is fixed.
I reported a similar bug in the 11.0.2 version for CentOS. Maybe is the FreeBSD
part not fixed.
If you are feeling adventurous you can try to fix it from the thread
information .
Regards,
--
MSc Heitor Faria (Miami/USA)
Bacula LATAM CEO
mobile1: + 1 909 655-8971
mobile2: + 55 61 98268-4220
[ https://www.linkedin.com/in/msc-heitor-faria-5ba51b3 ]
[ http://www.bacula.com.br/ ]
América Latina
[ http://bacula.lat/ | bacula.lat ] | [ http://www.bacula.com.br/ |
bacula.com.br ]
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] S3 cloud plugin does not load on FreeBSD 13.0
On Sun, Jul 18, 2021, at 6:32 AM, Ronny Forberger wrote:
> Hi,
>
> I have a problem that I cannot load the s3 cloud plugin on FreeBSD
> 13.0-RELEASE-p3 amd64.
>
> The error in bacula-sd I am getting is the following:
>
> > my-sd: init_dev.c:488-0 Driver=cloud entry point=800d4fe60
> > my-sd: cloud_dev.c:876-0 Open SD driver at
> > /usr/local/lib/bacula-sd-cloud-s3-driver-11.0.5.so
> > my-sd: cloud_dev.c:879-0 Driver=s3 handle=800d19408
> > my-sd: cloud_dev.c:881-0 Lookup "BaculaCloudDriver" in driver=s3
> > my-sd: cloud_dev.c:883-0 Driver=s3 entry point=0
> > my-sd: cloud_dev.c:889-0 Lookup of symbol "BaculaCloudDriver"
> > driver=/usr/local/lib/bacula-sd-cloud-s3-driver-11.0.5.so failed:
> > ERR=Undefined symbol "BaculaCloudDriver"
This line above ...
> > my-sd: message.c:1831-0 cloud_dev.c:1001 Could not open Cloud driver
> > type=1 for Device=CloudStorage.
> > my-sd: message.c:1831-0 cloud_dev.c:1032 Cloud driver initialization
> > error Cloud driver not properly loaded
> > my-sd: htable.c:67-0 malloc buf=801c71ae8 size=100 rem=76
> > my-sd: init_dev.c:235-0 init_dev allocated: 801c1cc28
> > my-sd: init_dev.c:437-0 init_dev: tape=0
> > dev_name=/backup/backupdisk1/bacula-storage
> > my-sd: dev.c:1143-0 DEVICE::register_metrics called. 0x801c1cc28
> > collector=0x80101c938
> > my-sd: stored.c:706-0 SD init done CloudStorage (0x801c1cc28)
> I comppiled the cloud-driver and the cloud-s3-driver with the following
> commands:
> > $ /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/src/stored
> > > make install-cloud
> >
> > LT Compiling cloud_parts.c
> > LT Compiling cloud_transfer_mgr.c
> > LT Compiling cloud_dev.c
> > cloud_dev.c:1818:17: warning: address of array 'tpkt->m_hash64' will
> > always evaluate to 'true' [-Wpointer-bool-conversion]
> > if (tpkt->m_hash64) {
> > ~~ ~~^~~~
> > 1 warning generated.
> > LT Compiling file_driver.c
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool
> > --silent --tag=CXX --mode=link /usr/bin/c++ -L/usr/local/lib
> > -L/usr/local/lib -L/usr/local/lib -fstack-protector-strong -shared
> > cloud_parts.lo cloud_transfer_mgr.lo cloud_dev.lo file_driver.lo -o
> > bacula-sd-cloud-driver.la -R /usr/local/lib -rpath /usr/local/lib
> > -module -export-dynamic -release 11.0.5
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/autoconf/mkinstalldirs
> >
> > /usr/local/lib
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool
> > --silent --tag=CXX --mode=install /usr/bin/install -c -m 755
> > bacula-sd-cloud-driver.la /usr/local/lib
> > /bin/rm -f /usr/local/lib/bacula-sd-cloud-driver.la
> >
> > $ /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/src/stored
> > > make install-s3-cloud
> >
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/autoconf/mkinstalldirs
> >
> > /usr/local/lib
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool
> > --silent --tag=CXX --mode=install /usr/bin/install -c -m 755
> > bacula-sd-cloud-driver.la /usr/local/lib
> > /bin/rm -f /usr/local/lib/bacula-sd-cloud-driver.la
> > LT Compiling s3_driver.c
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool
> > --silent --tag=CXX --mode=link /usr/bin/c++ -L/usr/local/lib
> > -L/usr/local/lib -L/usr/local/lib -fstack-protector-strong -shared
> > cloud_parts.lo cloud_transfer_mgr.lo s3_driver.lo -o
> > bacula-sd-cloud-s3-driver.la -R /usr/local/lib -rpath /usr/local/lib
> > -module -export-dynamic -release 11.0.5
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/autoconf/mkinstalldirs
> >
> > /usr/local/lib
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool
> > --silent --tag=CXX --mode=install /usr/bin/install -c -m 755
> > bacula-sd-cloud-s3-driver.la /usr/local/lib
> > /bin/rm -f /usr/local/lib/bacula-sd-cloud-s3-driver.la
> >
> But when I start bacula-sd with bacula-sd -d 255, I am getting the above
>
> > ERR=Undefined symbol "BaculaCloudDriver"
> error message.
Matches this line. My thoughts: don't try to start bacula-sd until that
compile time error is fixed.
>
> Any ideas what cloud be wrong?
Let's see what the list comes up with over the next few days. If nothing comes
of it, try the devel mailing list.
--
Dan Langille
d...@langille.org
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problem disable all schedules but jobs starts
On 16/07/2021 21:36, Pierre Bernhardt wrote: Hello, I have bacula on debian buster. The configured jobs will start at the estimate times althoug the schedules are manually disabled and in bconsole status dir shown no scheduled jobs. Only if I disable manually the jobs one by one they will not start. I don't know why they start after schedules has been disabled. Is this a normal behavior? A cause of this is changing the bacula-dir.conf file and not using the "reload" command to tell the director about it. Cheers, GaryB-) ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Migration from tape to disk is hanging
Hello, any idea how I can get more information why my job is hanging? It looks like it has started but not all copy the data to the disk. Both jobs for migration are still running since hours. I tried also to use data spooling to tmp but then the spool file will be filled, but never the data will be send to the disk (the disk file will not increase and won't change the mtime) Before I used for latest test setdebug level=99 trace=1 to change the debug for sd and dir. But I did not find information. Maybe there is another option which give more details? Cheers, Pierre ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Problem disable all schedules but jobs starts
Hello, I have bacula on debian buster. The configured jobs will start at the estimate times althoug the schedules are manually disabled and in bconsole status dir shown no scheduled jobs. Only if I disable manually the jobs one by one they will not start. I don't know why they start after schedules has been disabled. Is this a normal behavior? Cheers, Pierre ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] S3 cloud plugin does not load on FreeBSD 13.0
Hi,I have a problem that I cannot load the s3 cloud plugin on FreeBSD 13.0-RELEASE-p3 amd64.The error in bacula-sd I am getting is the following:> my-sd: init_dev.c:488-0 Driver=cloud entry point=800d4fe60> my-sd: cloud_dev.c:876-0 Open SD driver at > /usr/local/lib/bacula-sd-cloud-s3-driver-11.0.5.so> my-sd: cloud_dev.c:879-0 Driver=s3 handle=800d19408> my-sd: cloud_dev.c:881-0 Lookup "BaculaCloudDriver" in driver=s3> my-sd: cloud_dev.c:883-0 Driver=s3 entry point=0> my-sd: cloud_dev.c:889-0 Lookup of symbol "BaculaCloudDriver" > driver=/usr/local/lib/bacula-sd-cloud-s3-driver-11.0.5.so failed: > ERR=Undefined symbol "BaculaCloudDriver"> my-sd: message.c:1831-0 cloud_dev.c:1001 Could not open Cloud driver > type=1 for Device=CloudStorage.> my-sd: message.c:1831-0 cloud_dev.c:1032 Cloud driver initialization > error Cloud driver not properly loaded> my-sd: htable.c:67-0 malloc buf=801c71ae8 size=100 rem=76> my-sd: init_dev.c:235-0 init_dev allocated: 801c1cc28> my-sd: init_dev.c:437-0 init_dev: tape=0 > dev_name=/backup/backupdisk1/bacula-storage> my-sd: dev.c:1143-0 DEVICE::register_metrics called. 0x801c1cc28 > collector=0x80101c938> my-sd: stored.c:706-0 SD init done CloudStorage (0x801c1cc28)I comppiled the cloud-driver and the cloud-s3-driver with the following commands:> $ /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/src/stored > > make install-cloud>> LT Compiling cloud_parts.c> LT Compiling cloud_transfer_mgr.c> LT Compiling cloud_dev.c> cloud_dev.c:1818:17: warning: address of array 'tpkt->m_hash64' will > always evaluate to 'true' [-Wpointer-bool-conversion]> if (tpkt->m_hash64) {> ~~ ~~^~~~> 1 warning generated.> LT Compiling file_driver.c> /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool > --silent --tag=CXX --mode=link /usr/bin/c++ -L/usr/local/lib > -L/usr/local/lib -L/usr/local/lib -fstack-protector-strong -shared > cloud_parts.lo cloud_transfer_mgr.lo cloud_dev.lo file_driver.lo -o > bacula-sd-cloud-driver.la -R /usr/local/lib -rpath /usr/local/lib > -module -export-dynamic -release 11.0.5> /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/autoconf/mkinstalldirs > /usr/local/lib> /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool > --silent --tag=CXX --mode=install /usr/bin/install -c -m 755 > bacula-sd-cloud-driver.la /usr/local/lib> /bin/rm -f /usr/local/lib/bacula-sd-cloud-driver.la>> $ /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/src/stored > > make install-s3-cloud>> /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/autoconf/mkinstalldirs > /usr/local/lib> /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool > --silent --tag=CXX --mode=install /usr/bin/install -c -m 755 > bacula-sd-cloud-driver.la /usr/local/lib> /bin/rm -f /usr/local/lib/bacula-sd-cloud-driver.la> LT Compiling s3_driver.c> /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool > --silent --tag=CXX --mode=link /usr/bin/c++ -L/usr/local/lib > -L/usr/local/lib -L/usr/local/lib -fstack-protector-strong -shared > cloud_parts.lo cloud_transfer_mgr.lo s3_driver.lo -o > bacula-sd-cloud-s3-driver.la -R /usr/local/lib -rpath /usr/local/lib > -module -export-dynamic -release 11.0.5> /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/autoconf/mkinstalldirs > /usr/local/lib> /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool > --silent --tag=CXX --mode=install /usr/bin/install -c -m 755 > bacula-sd-cloud-s3-driver.la /usr/local/lib> /bin/rm -f /usr/local/lib/bacula-sd-cloud-s3-driver.la>But when I start bacula-sd with bacula-sd -d 255, I am getting the above> ERR=Undefined symbol "BaculaCloudDriver"error message.Any ideas what cloud be wrong?Best regards,Ronny Forberger-- ___Ronny Forbergerronnyforberger at ronnyforberger.dePGP: http://www.ronnyforberger.de/pgp/email-encryption.html
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
