On Thu, Jul 8, 2021 at 10:33 AM 'Christian Svensson' via bareos-users < [email protected]> wrote:
> Instead, I would like to propose a change where the spooling process creates two spool files (each bound to 50% of Maximum Spool Size), Spool-A and Spool-B. > When Spool-A is filled up, Bareos-SD starts to despool that file while continuing to spool data to Spool-B. > If Spool-B fills up while Spool-A is running, the spooling is paused until the spooling of Spool-A is done. At that moment Spool-B is despooled and Spool-A is used to spool data. I like this idea. My configuration for example has a single machine which is much larger than all the others put together. The throughput (since I turned spooling on on 2021-06-04) is lower for the longest jobs, and that's where I really want to save time. On the other hand the clients with throughput small enough to need spooling have backups that don't run for a long time, and so there would not be much shoe-shining. bareos=> SELECT CASE WHEN starttime < '2021-06-04' THEN 'OFF' WHEN starttime > '2021-06-05' THEN 'ON' ELSE 'UNKNOWN' END AS spooling, ROUND(AVG(JobFiles),0) AS AverageFiles, ROUND(AVG(JobBytes/1024.0/1024.0/1024.0),1) AS AverageGB, AVG(endtime-starttime) AS avg_elapsed, AVG(JobBytes/1024.0/1024.0/EXTRACT(EPOCH FROM (endtime-starttime))) AS avg_throughput_mbps, Name AS job FROM Job WHERE level = 'F' AND endtime != starttime GROUP by Name, spooling ORDER BY job, spooling, avg_throughput_mbps DESC; spooling | averagefiles | averagegb | avg_elapsed | avg_throughput_mbps | job ----------+--------------+-----------+------------------+---------------------+--------------------------------- ON | 95392 | 6.1 | 00:12:27.5 | 6.35006567352883 | backup-Big-in-Japan-fd-all UNKNOWN | 145402 | 10.7 | 00:56:15.666667 | 9.85466560430505 | backup-Big-in-Japan-fd-all OFF | 56 | 0.4 | 00:02:41 | 2.3997043586406 | BackupCatalog ON | 84 | 2.1 | 00:02:02.578947 | 22.6342882288465 | BackupCatalog UNKNOWN | 84 | 0.7 | 00:02:22 | 4.74277415074093 | BackupCatalog OFF | 843949 | 30.0 | 00:07:28 | 68.6448513461011 | backup-horizon-fd-all ON | 781091 | 29.4 | 00:08:42.666667 | 61.3886728985652 | backup-horizon-fd-all OFF | 876132 | 13276.2 | 1 day 07:00:21 | 121.794402469763 | backup-jupiter-fd-all ON | 2654049 | 10568.3 | 1 day 16:30:13 | 72.0647217388582 | backup-jupiter-fd-all OFF | 2616869 | 1405.3 | 07:37:43 | 52.3996788929579 | backup-jupiter-fd-home ON | 2874971 | 1416.3 | 08:12:06 | 49.1188398851964 | backup-jupiter-fd-home ON | 304803 | 1630.5 | 07:37:05 | 60.8783728477318 | backup-jupiter-fd-nas ON | 339038 | 9564.7 | 1 day 13:12:14.5 | 73.137894083823 | backup-jupiter-fd-nas-video OFF | 116521 | 7.8 | 00:08:35 | 15.4905055259038 | backup-substrate-fd-all ON | 109400 | 7.7 | 00:10:21.666667 | 13.2956005905249 | backup-substrate-fd-all ON | 71047 | 3.0 | 00:02:08 | 23.6317118629813 | backup-terminator-fd-all OFF | 75407 | 3.2 | 00:00:56 | 58.7631317887987 | backup-terminator-fd-everything ON | 75419 | 3.2 | 00:02:17 | 24.1265843106012 | backup-terminator-fd-everything ON | 0 | 0.0 | 00:04:44.333333 | 0 | RestoreFiles UNKNOWN | 1 | 0.2 | 00:06:14 | 0.366163643718768 | RestoreFiles (20 rows) This analysis would be better if I could figure out the write throughput while despooling, but I don't know how to extract that from the Bareos catalog (though as another thread pointed out, you can get this from tapestats or in the future, Prometheus). James. -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/CAGwX7666tG3XW6pRX1x-P88XFZDozg527ZK6975_roVjsXT7KQ%40mail.gmail.com.
