Would also vote to remove when TCM is available/stable. More words on the create, drop, re-create, in case it’s not familiar to people:
- Create assigns an ID - Drop removes definition and data - Create assigns an ID If non-deterministic, we had races where an ID was assigned concurrently in two processes, and then we’d see CFID mismatches (and in some cases, data loss) where different nodes had different opinions of table ID, which really only got resolved after a full cluster bounce (the state could be inconsistent in memory and on disk, so directory would change on reboot in some horrible cases). If the ID is deterministic, two threads making the same table at the same time are guaranteed to think the table is the same, but if you have a node offline, drop the table, create the table again, node comes online, the offline node had situations where it would handle that “poorly” (TCM is the right fix in both cases and fixes the first problem instantly, so once it’s here, let’s just get rid of the remaining problem of deterministic IDs masking a drop during partitions) > On Jul 31, 2024, at 11:47 AM, David Capwell <dcapw...@apple.com> wrote: > >> what’s the motivation for removal? > > One thing that also makes it better to remove is that this adds the ABA > problem into our tables, where as removing it fixes that… > > If you create a table “foo”, drop it, then recreate “foo”, the 2 “foo” tables > share the same tableId, which can trick things to think they are the same… > > When the table ID was random and could conflict on different nodes, this > feature was useful to make sure to avoid such an issue, but with TCM that > problem is solved, so this just adds more complexity as the “real” ID of a > table is (TableId, Epoch)… and that… is annoying... > >> On Jul 30, 2024, at 3:47 PM, Jordan West <jorda...@gmail.com> wrote: >> >> Understood. Nits aside I have no objection to your plan. >> >> Jordan >> >> On Tue, Jul 30, 2024 at 15:42 Caleb Rackliffe <calebrackli...@gmail.com >> <mailto:calebrackli...@gmail.com>> wrote: >>> I think the main motivation for ignoring the YAML option and removing down >>> the line is that we probably never would have created it if TCM existed at >>> that point of creation. I'd liken it to what we did w/ some >>> no-longer-relevant options for the batch commit log. >>> >>> On Tue, Jul 30, 2024 at 5:19 PM Jordan West <jorda...@gmail.com >>> <mailto:jorda...@gmail.com>> wrote: >>>> Generally no disagreement but more of a curiosity: what’s the motivation >>>> for removal? Just that it’s not needed? Otherwise it’s relatively cheap >>>> and DDL aren’t high throughput (or at least shouldn’t be since we can only >>>> deal with so many tables) >>>> >>>> Jordan >>>> >>>> On Tue, Jul 30, 2024 at 15:04 Caleb Rackliffe <calebrackli...@gmail.com >>>> <mailto:calebrackli...@gmail.com>> wrote: >>>>> To summarize all this noise I've created, the plan would be... >>>>> >>>>> 1.) Leave CQL WITH id intact. >>>>> 2.) Deprecate and WARN on use_deterministic_table_id in 5.0.x. >>>>> 3.) Ignore and WARN on use_deterministic_table_id in 5.1. >>>>> 4.) Profit >>>>> >>>>> On Tue, Jul 30, 2024 at 4:46 PM Caleb Rackliffe <calebrackli...@gmail.com >>>>> <mailto:calebrackli...@gmail.com>> wrote: >>>>>> No intention of touching WITH id in CQL >>>>>> >>>>>> On Tue, Jul 30, 2024 at 4:10 PM Caleb Rackliffe >>>>>> <calebrackli...@gmail.com <mailto:calebrackli...@gmail.com>> wrote: >>>>>>> To clarify, my plan was to deprecate in Config/JMX and ignore it, not >>>>>>> remove it entirely so it breaks existing YAMLs and JMX clients. >>>>>>> >>>>>>> This should be fine, if I'm reading the upgrade notes correctly, as no >>>>>>> table or view creation operations will be allowed on 5.1 nodes until >>>>>>> upgrade is complete and the CMS has been initialized. >>>>>>> >>>>>>> On Tue, Jul 30, 2024 at 3:54 PM J. D. Jordan <jeremiah.jor...@gmail.com >>>>>>> <mailto:jeremiah.jor...@gmail.com>> wrote: >>>>>>>> +1 to deprecate it. What does removing it buy us? >>>>>>>> >>>>>>>>> On Jul 30, 2024, at 3:52 PM, David Capwell <dcapw...@apple.com >>>>>>>>> <mailto:dcapw...@apple.com>> wrote: >>>>>>>>> >>>>>>>>> Users can provide ids and TCM can manage to make them safe, so agree >>>>>>>>> we don’t really need the feature anymore. I am fine with deprecating >>>>>>>>> the feature, but removing would be a breaking change for anyone that >>>>>>>>> had that config in place, so not a fan of breaking the config >>>>>>>>> interface. >>>>>>>>> >>>>>>>>>> On Jul 30, 2024, at 1:38 PM, Caleb Rackliffe >>>>>>>>>> <calebrackli...@gmail.com <mailto:calebrackli...@gmail.com>> wrote: >>>>>>>>>> >>>>>>>>>> I'd like to propose removing deterministic table IDs for new user >>>>>>>>>> tables and views in trunk. With TCM in place, it looks like the >>>>>>>>>> reason we added use_deterministic_table_id, concurrent table >>>>>>>>>> creations, is no longer a concern. >>>>>>>>>> >>>>>>>>>> Thoughts? Objections? >>>>>>>>> >
