OK, I get that the definition of the SQLite API is a (large) set of C function calls. And that changing the way they work under the covers would break something. (Check out the IDEA below!)
I'm wondering if your use case is - in the same application - that you dynamically change from option SQLITE_CONFIG_SINGLETHREAD to SQLITE_CONFIG_MULTITHREAD to SQLITE_CONFIG_SERIALIZED while your application is running? If that is the case, then your application knows which option to use dynamically; Otherwise, your application doesn't know and doesn't care which option is in effect. I am suggesting that if the we added the global calls to the underlying functions to the API - that is, the functions that are called by the function table indirection - then one could code the application to call the underlying functions. If the application knows it's single-thread, then code it that way and get a 25% improvement (see the talk). If the application makes the choice of thread option dynamically, then the penalty for single-thread is at least double (application choice, SQLite indirection), so calling the (new) underlying function once you made the choice, performs better for that path. I grant you that probably will see little improvement on the threaded path. If you are going to tell me that you need to maintain two versions of your application if you run it in a single-thread environment or a multi-thread environment, then let's define the (new) API to use a preprocessor macro to generate the right code for the option selected based on the values of SQLITE_CONFIG_SINGLETHREAD, el al. So now you have a single source but multiple executables corresponding to that source. And the choice of which executable to use becomes a configuration problem at application deployment time, or at application run time. ---- IDEA! ---- Thinking about it, I'm surprised that the C API isn't just a set of macros already. I can visualize a C API composed of a set of macro definitions _identical_ to the current C function calls. They just use the extra knowledge of SQLITE_THREADSAFE and other SQLite compiler options to decide what application code to generate. Then the formal API doesn't change from a coding point of view. The generated code calls a set of under-the-cover functions which are not part of the API. The change doesn't require a new layer of testing; presumably, we already have test cases that test the same code using different compiler options. What about that? Best, Doug > -----Original Message----- > From: sqlite-users <sqlite-users-boun...@mailinglists.sqlite.org> > On Behalf Of Jens Alfke > Sent: Friday, December 27, 2019 2:22 PM > To: SQLite mailing list <sqlite-users@mailinglists.sqlite.org> > Subject: Re: [sqlite] Causal profiling > > > > > On Dec 26, 2019, at 3:45 PM, Doug <dougf....@comcast.net> wrote: > > > > Can you explain what the API is that you use to "set concurrency > levels per connection", please? Is it a parameter on open() or its > own function call? And how would the API break, exactly? > > sqlite3_config(), specifically the SQLITE_CONFIG_SINGLETHREAD, > SQLITE_CONFIG_MULTITHREAD, SQLITE_CONFIG_SERIALIZED options. > > This API would break because configuring those options at runtime > would have no effect on behavior; the only thing that would change > threading behavior would be the compile-time flags > SQLITE_MUTEX_OMIT, etc. > > (This is actually global, not per-connection, but that doesn't > invalidate what I said.) > > > The talk suggested removing the SQLite virtual table of > functions (specifically the call to free a mutex). The user calls > the function directly. How does that break an API? > > If SQLite's implementation directly called the mutex lock/unlock > functions, instead of indirecting, then there would be no way to > control whether or not mutexes were used. In other words, it would > be impossible to change any of the above options at runtime. > > > The talk suggested removing the SQLite virtual table of > functions (specifically the call to free a mutex). The user calls > the function directly. How does that break an API? > > No, the user does not call those functions directly. The code > shown in the video is deep inside SQLite itself and not visible > through the API. (You say you're using a TCL wrapper … so you may > not be aware of what is or isn't in the C API. Trust me, I use the > C API a lot.) > > —Jens > > _______________________________________________ > sqlite-users mailing list > sqlite-users@mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite- > users _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
