We agreed a long time ago that all new features are disabled by default, but I wanted to try to flesh out what we “should” do with something that might still be experimental and subject to breaking changes; I would prefer we keep this thread specific to vtables as the UX is different for different types of things…
So, lets say we are adding a set of vtables but we are not 100% sure what the schema should be and we learn after the release that changes should be made, but that would end up breaking the table… we currently define everything as “don’t break this” so if we publish a table that isn’t 100% baked we are kinda stuck with it for a very very long time… I would like to define a way to expose vtables that are subject to change (breaking schema changes) across different release and rules around them (only in minor? Maybe even in patch?). Lets try to use a concrete example so everyone is on the same page. Accord is disabled by default (it is a new feature), so the vtables to expose internals would be expected to be undefined and not present on the instance. When accord is enabled (accord.enabled = true) we add a set of vtables: Epochs - shows what epochs are known to accord Cache - shows how the internal caches are performing Etc. Using epochs as an example it currently only shows a single column: the long epoch CREATE VIRTUAL TABLE system_accord.epochs (epoch bigint PRIMARY KEY); Lets say we find that this table isn’t enough and we really need to scope it to each of the “stores” (threads for processing accord tasks) CREATE VIRTUAL TABLE system_accord.epochs (epoch bigint, store_id int, PRIMARY KEY (epoch, store_id)); In this example the table changed the schema in a way that could break users, so this normally is not allowed. Since we don’t really have a way to define something experimental other than NEWS.txt, we kinda get stuck with this table and are forced to make new versions and maintain them for a long time (in this example we would have epochs and epochs_v2)… it would be nice if we could define a way to express that tables are free to be changed (modified or even deleted) and the life cycle for them…. I propose that we allow such a case and make changes to the UX (as best as we can) to warn about this: 1) update NEWS.txt to denote that the feature is experimental 2) when you access an experimental table you get a ClientWarning stating that this is free to change 3) the tables comments starts with “[EXPERIMENTAL]” What do others think?
