Jim, > The problem with specialized processors is that they are a scarce > resource that must be managed rather than shared. They're just dandy > when a server has a single specialized load, but on a server with > multiple clients, one guy gets the specialized processor and everyone one > else waits. > > The best way for Firebird to do parallism is to move towards fine > grain multi- threading so at least all available processors can keep > busy doing useful work. Trying to accelerate a single operation with > non-shared hardware is almost always a net loss.
To my way of thinking/understanding the current OpenCL/GPU/PHI solutions can be used as a shared resource. To be clear, I am proposing that the development focus should be on refactoring the existing code with the view of improving multi-threading support, first, and then parallelism -- thru libraries like the Intel C++ Compiler or Intel Parallel Studio which can automatically detect and use available hardware (CPU/threads, MPI, GPU or coprocessor) to the greatest extent possible. Granted for some of the solutions, some resource management/queuing may be required, but the overall performance payoff could be worth it. In the case of the PHI, having up to 61* "helper" processors which could be responsible for performing sorting/grouping for *any* running query (so a shared resource) would provide significant benefit. In the case of the PHI, each processor is a full x86 instance, so unlike GPU based solutions no specialized instructions set would be required. * 57 to 61 cores (@1.053 to 1.238Ghz) per PHI card, a server could have several cards -- up to 8 cards per server. > You might remember that the original DEC JRD was the core for what was > eventually be the DEC database machine. We had people on specialized > hardware and specialized microcode. Granted I am now an old man with a 20yr old son, but I wasn't around in the Neolithic era! ;-] > The Falcon group at MySQL had a long meeting with the Intel > parallelization tool group. My I ask, when were those discussions? Was the XEON PHI discussed? > At the end of the day, everyone was in agreement that with more > processes than cores, fine grain multi-threading using user mode > interlocked instructions for thread synchronization was by far the > best solution. Again, there are some operations (ie. Sorting/merging/grouping) where interlocked instructions are not necessary. Sean ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
