Re: [Firebird-devel] Lock manager's hash calculations

2015-12-30 Thread Dimitry Sibiryakov
30.12.2015 14:43, Dmitry Yemanov wrote: > The first one is in lock.cpp and you've found two others, three in total. Ok, then it is done now. Should I prepare a patch for Firebird 3 or leave it in Avalerion and let you adopt it at will? -- WBR, SD.

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-30 Thread Dmitry Yemanov
30.12.2015 16:16, Dimitry Sibiryakov wrote: > I have found two: one in HashJoin.cpp and other in lck.cpp. Can't find third > one. Where > it is? The first one is in lock.cpp and you've found two others, three in total. Dmitry --

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-30 Thread Dimitry Sibiryakov
29.12.2015 18:16, Dmitry Yemanov wrote: > I didn't speak about Hash.h, but about other three copies that are the same. I have found two: one in HashJoin.cpp and other in lck.cpp. Can't find third one. Where it is? -- WBR, SD. -

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-29 Thread Wols Lists
On 29/12/15 10:23, Dimitry Sibiryakov wrote: > 24.12.2015 18:29, Alex Peshkoff wrote: >> If we have one chain with 12 objects, three empty chains and chains >> with 1, 2 or 3 objects - it's not bad. >> I.e. existing stat-s is not enough for analysis. > >I've got following data from Igor V

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-29 Thread Dimitry Sibiryakov
29.12.2015 17:29, Alex Peshkoff wrote: > Remains to check on something like tpcc does it affect CS performance. If someone has a powerful computer with SSE4.2 support and ready to - fresh builds for Windows are available at http://www.ibphoenix.com/ibpr_devel/ -- WBR, SD. --

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-29 Thread Dmitry Yemanov
29.12.2015 20:11, Dimitry Sibiryakov wrote: > Yep. Look into Hash.h. It is much different (and much slower). I didn't speak about Hash.h, but about other three copies that are the same. Dmitry -- Firebird-Devel mailin

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-29 Thread Dimitry Sibiryakov
29.12.2015 18:08, Dmitry Yemanov wrote: > Really? Yep. Look into Hash.h. It is much different (and much slower). -- WBR, SD. -- Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/li

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-29 Thread Dmitry Yemanov
28.12.2015 16:03, Dimitry Sibiryakov wrote: Yes. But in this case the hash is only used internally and not stored anywhere. Because of this, different platforms can use different algorithms for it. >> Of course, but then it should be encapsulated and moved to /common to >> avoid th

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-29 Thread Alex Peshkoff
On 12/29/2015 07:05 PM, Dimitry Sibiryakov wrote: > 29.12.2015 11:43, Alex Peshkoff wrote: >> That I do not expect to get something better than this bell curve: > > I was unable to create a good load on database with my weak > notebook, but oltp-emul with 5 windows gave me these numbers for lock

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-29 Thread Tommi Prami
I stumbled upon quite evesome Hash-algorithm xxHash. There are lightning fast implementations. https://github.com/Cyan4973/xxHash Noted that usually the Hash algorithm is not the bottleneck, but still pretty interesting... -Tee- On Tue, Dec 29, 2015 at 6:05 PM, Dimitry Sibiryakov wrote: > 29

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-29 Thread Dimitry Sibiryakov
29.12.2015 11:43, Alex Peshkoff wrote: That I do not expect to get something better than this bell curve: I was unable to create a good load on database with my weak notebook, but oltp-emul with 5 windows gave me these numbers for lock manager using CRC32: Hash slots: 8191, Hash lengths (

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-29 Thread Alex Peshkoff
On 12/29/2015 01:23 PM, Dimitry Sibiryakov wrote: 24.12.2015 18:29, Alex Peshkoff wrote: If we have one chain with 12 objects, three empty chains and chains with 1, 2 or 3 objects - it's not bad. I.e. existing stat-s is not enough for analysis. I've got following data from Igor Valchen

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-29 Thread Dimitry Sibiryakov
24.12.2015 18:29, Alex Peshkoff wrote: > If we have one chain with 12 objects, three empty chains and chains > with 1, 2 or 3 objects - it's not bad. > I.e. existing stat-s is not enough for analysis. I've got following data from Igor Valchenko: > Hash slots: 65521, Hash lengths (min/avg/

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-28 Thread Dimitry Sibiryakov
24.12.2015 18:22, Dmitry Yemanov wrote: >> >Yes. But in this case the hash is only used internally and not stored >> >anywhere. Because >> >of this, different platforms can use different algorithms for it. > Of course, but then it should be encapsulated and moved to /common to > avoid the same #if

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-25 Thread Dimitry Sibiryakov
25.12.2015 14:14, Vlad Khorsun wrote: > crc32" used _mm_crc32_u32, _mm_crc32_u16 and _mm_crc32_u8 when appropriate Cool. I'm surprised that these instructions work even with unaligned data. -- WBR, SD. -- Firebird

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-25 Thread Vlad Khorsun
25.12.2015 13:40, Dimitry Sibiryakov wrote: > 25.12.2015 12:12, Dimitry Sibiryakov wrote: >> You'd better try final attached variant yourself. > > BTW, MSVC2010 32 bits showed the worst result: only 2.5x speed up while > GCC -O3 gave 5x. > Vlad, could you try it with MSVC2013?.. > MSV

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-25 Thread Alex Peshkoff
On 12/25/2015 02:12 PM, Dimitry Sibiryakov wrote: > 24.12.2015 17:58, Alex Peshkoff wrote: >> BTW, please try it in such form: > > I can't believe my eyes. You'd better try final attached variant > yourself. > Telling true have not noticed great difference. Looks like it's highly HW-dependent.

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-25 Thread Dimitry Sibiryakov
25.12.2015 12:12, Dimitry Sibiryakov wrote: > You'd better try final attached variant yourself. BTW, MSVC2010 32 bits showed the worst result: only 2.5x speed up while GCC -O3 gave 5x. Vlad, could you try it with MSVC2013?.. -- WBR, SD.

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-25 Thread Wols Lists
On 24/12/15 17:22, Dmitry Yemanov wrote: >> I suspect that "true hash" could make distribution of hash values more >> equival >> > and thus improve overall timing for lock table search. > A couple of years ago I played a bit with different hash functions that > are considered good nowadays. Quit

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-25 Thread Dimitry Sibiryakov
24.12.2015 17:58, Alex Peshkoff wrote: BTW, please try it in such form: I can't believe my eyes. You'd better try final attached variant yourself. -- WBR, SD. HashTest.cpp.7z Description: Binary data -- Firebird-

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-24 Thread Alex Peshkoff
On 12/24/2015 08:24 PM, Dimitry Sibiryakov wrote: > 24.12.2015 18:19, Alex Peshkoff wrote: >> This should be checked. At first one should show that current hash >> provides bad distribution. > In fb_lock_print output I often see something like this: > >> Hash lengths (min/avg/max):0/ 2/

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-24 Thread Leyne, Sean
>On the other hand, CS is going down, so lock manager performance may be > not a critical part anymore? It will take groups like mine a while before they embrace the new SMP functionality of v3 -- there is application/deployment testing to be performed... So, CS will be around for a while

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-24 Thread Dimitry Sibiryakov
24.12.2015 18:19, Alex Peshkoff wrote: > This should be checked. At first one should show that current hash > provides bad distribution. In fb_lock_print output I often see something like this: > Hash lengths (min/avg/max):0/ 2/ 12 Isn't it bad? -- WBR, SD. ---

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-24 Thread Dmitry Yemanov
24.12.2015 20:07, Dimitry Sibiryakov wrote: > > Yes. But in this case the hash is only used internally and not stored > anywhere. Because > of this, different platforms can use different algorithms for it. Of course, but then it should be encapsulated and moved to /common to avoid the same #ifde

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-24 Thread Alex Peshkoff
On 12/24/2015 08:07 PM, Dimitry Sibiryakov wrote: > 24.12.2015 17:48, Alex Peshkoff wrote: >> Use of CPU-accelerated code appears suspicious from non-intel ports POV. > Yes. But in this case the hash is only used internally and not stored > anywhere. Because > of this, different platforms can

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-24 Thread Dimitry Sibiryakov
24.12.2015 17:58, Alex Peshkoff wrote: > For me (AMD CPU) it behaves much better for length in 11,15,19,etc. I have AMD too, so we need someone with Intel to see if these conditions won't defeat its jump prediction algorithms there. -- WBR, SD. --

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-24 Thread Dimitry Sibiryakov
24.12.2015 17:48, Alex Peshkoff wrote: > Use of CPU-accelerated code appears suspicious from non-intel ports POV. Yes. But in this case the hash is only used internally and not stored anywhere. Because of this, different platforms can use different algorithms for it. I suspect that "true ha

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-24 Thread Alex Peshkoff
On 12/24/2015 07:48 PM, Alex Peshkoff wrote: > On 12/24/2015 04:19 PM, Dimitry Sibiryakov wrote: >> Hello, All. >> >>I played a little with hash calculation code from lock manager and >> found that there is a room for improvement. >>Attached test program compiled by GCC 4.7.1 with switches

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-24 Thread Alex Peshkoff
On 12/24/2015 04:19 PM, Dimitry Sibiryakov wrote: > Hello, All. > > I played a little with hash calculation code from lock manager and > found that there is a room for improvement. > Attached test program compiled by GCC 4.7.1 with switches "-O3 > -msse4.2" shown that simple unroll of the loo

Re: [Firebird-devel] Lock manager's hash calculations

2015-12-24 Thread Dimitry Sibiryakov
24.12.2015 14:19, Dimitry Sibiryakov wrote: > simple unroll of the loop makes it 2-3 times faster. Actually, even replace of USHORT with ULONG in the loop wins about 15%. -- WBR, SD. -- Firebird-Devel mailing list

[Firebird-devel] Lock manager's hash calculations

2015-12-24 Thread Dimitry Sibiryakov
Hello, All. I played a little with hash calculation code from lock manager and found that there is a room for improvement. Attached test program compiled by GCC 4.7.1 with switches "-O3 -msse4.2" shown that simple unroll of the loop makes it 2-3 times faster. Current code is even slower t