Hi Kern,
I'm 100% sure I've checked for extra indexes on the File table in the past
but now I checked again and then dropped the index I had on PathId in the
File table.
The time to build the build the restore tree is again manageable now for a
test case I had run earlier today and also for a different client specified
just to be sure it wasn't improved due to database/disk caching.
The index was named idxPIchk which I don't think I would have come up with
on my own, so I'm not sure where it came from. Is that the name dbcheck
uses to create the index that's supposed to be temporary?
Bob
On 4/1/2011 2:57 AM, Kern Sibbald wrote:
> Hello Bob,
>
> I agree with you (as well as with Phil): it would probably take some major
> rewrite to try to use multiple threads to improve performance, and second
> doing so would be unlikely to improve the overall speed.
>
> I would go one step further than you did toward understanding the "problem"
> and say that it is very unlikely that it is caused by the way that the SQL
> query is written or that it has anything to do with the Accurate changes (I
> believe we proved that with tests). The major performance bottleneck is most
> probably with MySQL itself or the MySQL configuration.
>
> With an SQL backend that is properly tuned, this SQL request even for 20
> million files has quite reasonable performance (i.e. order of minutes).
> Depending on the number of files involved one needs to tune the SQL conf to
> handle such requests. Also users often add additional seemly inoculous
> database indexes (or have old historic ones that are no longer recommended),
> which totally destroy performance.
>
> Bacula Systems provides professional help (solutions) for these kinds of
> problems :-)
>
> Regards,
>
> Kern
>
> On Thursday 31 March 2011 23:29:20 Bob Hetzel wrote:
>> On 3/30/2011 9:43 PM, Phil Stracchino wrote:
>>> On 03/30/11 17:05, Kern Sibbald wrote:
>>>> Hello,
>>>>
>>>> Thank you for your feature request. Unfortunately, I have no idea how
>>>> to implement it, so unless you can provide some new insight, it will not
>>>> help adding it to the projects file.
>>>>
>>>> Best regards,
>>>>
>>>> Kern
>>>>
>>>> On Wednesday 30 March 2011 15:23:31 peterfl...@users.sourceforge.net
> wrote:
>>>>> Item n: Add multi-threading to batch console
>>>>> Origin: Peter Flass<peter_fl...@yahoo.com>
>>>>> Date: 30 Mar 2011
>>>>> Status:
>>>>>
>>>>> What: When doing a restore building the directory tree
>>>>> takes an inordinate amount of time. Multi core systems are
>>>>> common and could benefit from this. On my dual-core 2.93GHz
>>>>> system looking at a full backup takes more than an hour
>>>>> (haven't actually timed it). System monitor shows one CPU
>>>>> 100% loaded and the other less than 5%.
>>>>>
>>>>> Why: This would speed up restores at lot, especially
>>>>> important for critical files.
>>>
>>> Yeah, this doesn't seem like a task that'd parallelize well without a
>>> major rewrite.
>>
>> I don't know for sure what Peter was seeing but the issue I've been seeing
>> is that the query Bacula uses to build the restore tree isn't well
>> optimized under MySQL since the v5.x bacula introduced the accurate backup
>> feature.
>>
>> Unfortunately it's a rather complex query so it's understandable that it
>> isn't perfect, but perhaps somebody can fully describe in words the
>> goal/pseudocode/methodology/whatever of that query, so it could be
>> re-looked at by other people.
>>
>> If Peter was seeing the same issue I've seen, then it's not even Bacula
>> consuming the cpu time but MySQL, and therefore the huge changes required
>> to more parallel-ize that section of the Bacula app code would not be
>> necessary.
>>
>> ---------------------------------------------------------------------------
>> --- Create and publish websites with WebMatrix
>> Use the most popular FREE web apps or write code yourself;
>> WebMatrix provides all the features you need to develop and
>> publish your website. http://p.sf.net/sfu/ms-webmatrix-sf
>> _______________________________________________
>> Bacula-devel mailing list
>> Bacula-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-devel
>
>
------------------------------------------------------------------------------
Create and publish websites with WebMatrix
Use the most popular FREE web apps or write code yourself;
WebMatrix provides all the features you need to develop and
publish your website. http://p.sf.net/sfu/ms-webmatrix-sf
_______________________________________________
Bacula-devel mailing list
Bacula-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-devel
