Andy used to tell us that the original SQL spooling was slower than Berkeley DB spooling and thus it was not used. However, the original SQL spooling was written for PostgreSQL, which is a client/server DB. An embedded SQLite DB should be much faster, as SQLite runs in the same process as the qmaster, the latency is much lower and thus should lead to better speed.
I just did a very quick test with SQLite 3.6.22, some of the SQL statements used by SGE were able to run against an SQLite DB without any changes - in the end, we are doing simple INSERT/UPDATE/DELETE SQL, plus some table maintenance code & table/index creation code... most SQL DBs could be used as the SGE spooling backend with min. changes to the SQL spooling code. (BTW, talking about Andy, I exchanged emails with him earlier this month, he and the Oracle Grid Engine team are doing good at Oracle.) Rayson On Fri, Sep 9, 2011 at 3:42 PM, "Hung-Sheng Tsao (Lao Tsao 老曹) Ph.D." <[email protected]> wrote: > > recently oracle introduce a SQLlite frontend to berkleyDB > not sure you can take advantage this new FE to query the berkleyDB in GE > regards > > On 9/9/2011 2:51 PM, Chi Chan wrote: >> >> It is good that Ron Chen said that there is new features (IPV6, GPU >> integration) -- but any forks working on SQL spooling? >> >> We don't need GPU integration now (with AMD bulldozer shipping soon, the >> need for a seperate GPU for handling GPGPU type of workload may be even >> less), and our clusters will not be migrated to IPV6 any time soon. >> >> However, SQL spooling is really useful, and I can query the job DB and >> find out what is there. In 2003 the code was in the source tree, but I >> looked at the SGE forks yesterday and found that the code is gone. >> >> So what has happened to SQL spooling, and any one planning to get it back >> to life? >> >> >> --Chi >> >> _______________________________________________ >> users mailing list >> [email protected] >> https://gridengine.org/mailman/listinfo/users > > _______________________________________________ > users mailing list > [email protected] > https://gridengine.org/mailman/listinfo/users > > _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
