The two-part proposal of 1.) table-level self-identification of experimental status and 2.) a global config flag that determines what to do when querying those might work. I guess the only thing you can't do there is ignore warnings from a specific experimental table, since that's controlled in code and not in configuration? You'd probably have to do that as a custom argument in the native protocol, similar to the client instructions for what to do on overload.
On Wed, May 29, 2024 at 6:11 PM David Capwell <dcapw...@apple.com> wrote: > As another option, we could add an @Experimetal annotation (or another name) > and a configuration parameter experimental_virtula_tables_enabled > > > Since these are tables we have TableProps so we could always store an > experimental = true there and if Config.experimental_virtula_tables_enabled > == false we just skip those tables when we load them. > > That way authors would just do the following… > > CreateTableStatement.parse(query, keyspace) > .comment(comment) > .kind(TableMetadata.Kind.VIRTUAL) > .experimental(true) > .build(); > > And VirtualKeyspace would skip it when > Config.experimental_virtula_tables_enabled > == false > > This feels very similar to the JDK… some features require you to opt into > experimental things twice… > > What about putting such a table to system_experimental keyspace? > > > I like it as it makes this apart of its name, but is sad that we must > break compatibility once we mark it stable (as we move the keyspace). > > > On May 29, 2024, at 1:23 PM, Štefan Miklošovič < > stefan.mikloso...@gmail.com> wrote: > > I think that 2) (client warning on accessing vtable) is pretty obtrusive / > irritating, that means that every single time I would e.g. SELECT from such > a table, it would emit a warning to a client / cqlsh? I do not think that > is necessary. > > What about putting such a table to system_experimental keyspace? Then it > would be about deciding when it is not considered experimental anymore and > once that happens, we just move it to the appropriate keyspace. I think > that it is better if users (rather admins) execute a query against > system_experimental.accord_epochs so they immediately know what they access > (something experimental) rather than being constantly reminded about that > by a client warning. > > Maxim's idea about "experimental_virtual_tables_enabled" property is also > viable. However, I can't help it but that somehow reminds me a property for > a guardrail. Just a thought. I do not see yet how we could leverage it, > just saying how I perceive it. > > On Wed, May 29, 2024 at 9:02 PM David Capwell <dcapw...@apple.com> wrote: > >> We agreed a long time ago that all new features are disabled by default, >> but I wanted to try to flesh out what we “should” do with something that >> might still be experimental and subject to breaking changes; I would prefer >> we keep this thread specific to vtables as the UX is different for >> different types of things… >> >> So, lets say we are adding a set of vtables but we are not 100% sure what >> the schema should be and we learn after the release that changes should be >> made, but that would end up breaking the table… we currently define >> everything as “don’t break this” so if we publish a table that isn’t 100% >> baked we are kinda stuck with it for a very very long time… I would like to >> define a way to expose vtables that are subject to change (breaking schema >> changes) across different release and rules around them (only in minor? >> Maybe even in patch?). >> >> Lets try to use a concrete example so everyone is on the same page. >> >> Accord is disabled by default (it is a new feature), so the vtables to >> expose internals would be expected to be undefined and not present on the >> instance. >> >> When accord is enabled (accord.enabled = true) we add a set of vtables: >> >> Epochs - shows what epochs are known to accord >> Cache - shows how the internal caches are performing >> Etc. >> >> Using epochs as an example it currently only shows a single column: the >> long epoch >> >> CREATE VIRTUAL TABLE system_accord.epochs (epoch bigint PRIMARY KEY); >> >> Lets say we find that this table isn’t enough and we really need to scope >> it to each of the “stores” (threads for processing accord tasks) >> >> CREATE VIRTUAL TABLE system_accord.epochs (epoch bigint, store_id int, >> PRIMARY KEY (epoch, store_id)); >> >> In this example the table changed the schema in a way that could break >> users, so this normally is not allowed. >> >> Since we don’t really have a way to define something experimental other >> than NEWS.txt, we kinda get stuck with this table and are forced to make >> new versions and maintain them for a long time (in this example we would >> have epochs and epochs_v2)… it would be nice if we could define a way to >> express that tables are free to be changed (modified or even deleted) and >> the life cycle for them…. >> >> I propose that we allow such a case and make changes to the UX (as best >> as we can) to warn about this: >> >> 1) update NEWS.txt to denote that the feature is experimental >> 2) when you access an experimental table you get a ClientWarning stating >> that this is free to change >> 3) the tables comments starts with “[EXPERIMENTAL]” >> >> What do others think? >> >> >> >
