Great discussion.
It would be an implementation detail but we could imagine having a single
interface for reading config and providing readers for old or/and new
configuration formats. If we can streamline reading of all configs from
that single interface we would have much centralized (following the Single
Responsibility Principle) changes to track/review and test.
Thanks
Maulin
On Wed, Feb 23, 2022 at 11:47 AM Jeremy Hanna <jeremy.hanna1...@gmail.com>
wrote:
> If we support both formats for a time, I just would want to make
> absolutely sure that it will read only one or the other so there's no
> uncertainty about the server configuration. Perhaps to avoid unforeseen
> migration problems, we only read the old format if a specific flag is set?
> So with version 5, we only read the new format by default. So if you only
> have the old format and you try to start 5.0, it will fail with a log
> message about a JVM option to be used ("READ_CASSANDRA_YAML" or
> something). So if you enable that, you *only* read the old config. It
> would be one or the other so you don't have weird dilemmas of which one to
> choose.
>
> On Feb 23, 2022, at 11:30 AM, Caleb Rackliffe <calebrackli...@gmail.com>
> wrote:
>
>
> Continuing to parse the old format for some time seems unavoidable, and
> allowing dot-separated options in the old format seems reasonable.
>
> There will certainly be some interesting problems when we move into
> implementation space with this. One approach might be to implement a clean
> object model that corresponds to the new format, work out how it's
> parsed/populated from the file, and then have some kind of converter from
> the old Config object to the new object model that allows us to provide
> values to DatabaseDescriptor from only the new one (thereby avoiding any
> changes to the places all over the codebase that use DD).
>
> On Wed, Feb 23, 2022 at 4:46 AM Bowen Song <bo...@bso.ng> wrote:
>
>> I agree with Benedict, there's legit use cases for both the flat and
>> structured config file format. The operator should be able to choose which
>> one is best suited for their own use case. It will also make the upgrade
>> process easier if both formats are supported by future versions of
>> Cassandra.
>> On 23/02/2022 07:52, bened...@apache.org wrote:
>>
>> I agree that a new configuration layout should be introduced once only,
>> not incrementally.
>>
>>
>>
>> However, I disagree that we should immediately deprecate the old config
>> file and refuse to parse it. We can maintain compatibility indefinitely at
>> low cost, so we should do so.
>>
>>
>>
>> Users of the old format, when using new configuration options, can simply
>> use dot separators to specify them. Since most settings are not required,
>> this is by far the least painful upgrade process.
>>
>>
>>
>>
>>
>> *From: *Berenguer Blasi <berenguerbl...@gmail.com>
>> <berenguerbl...@gmail.com>
>> *Date: *Wednesday, 23 February 2022 at 06:53
>> *To: *dev@cassandra.apache.org <dev@cassandra.apache.org>
>> <dev@cassandra.apache.org>
>> *Subject: *Re: [DISCUSS] CASSANDRA-17292 Move cassandra.yaml toward a
>> nested structure around major database concepts
>>
>> +1 to a non-incremental approach as well.
>>
>> On 23/2/22 1:27, Caleb Rackliffe wrote:
>> > @Patrick I’m absolutely intending for this to be a 5.0 concern. The
>> only reason why it would have any bearing on 4.x is the case where we’re
>> adding new config that could fit into the v2 structure now and not require
>> any later changes.
>> >
>> >> On Feb 22, 2022, at 3:22 PM, Bernardo Sanchez
>> <bernard...@pointclickcare.com> <bernard...@pointclickcare.com> wrote:
>> >>
>> >> unsubscribe
>> >>
>> >> -----Original Message-----
>> >> From: Stefan Miklosovic <stefan.mikloso...@instaclustr.com>
>> <stefan.mikloso...@instaclustr.com>
>> >> Sent: Tuesday, February 22, 2022 3:53 PM
>> >> To: dev@cassandra.apache.org
>> >> Subject: Re: [DISCUSS] CASSANDRA-17292 Move cassandra.yaml toward a
>> nested structure around major database concepts
>> >>
>> >> "EXTERNAL EMAIL" - This email originated from outside of the
>> organization. Do not click or open attachments unless you recognize the
>> sender and know the content is safe. If you are unsure, please contact
>> hel...@pointclickcare.com.
>> >>
>> >> I want to add that to, however, on the other hand, we also do have
>> dtests in Python and they need to run with old configs too. That is what
>> Ekaterina was doing - supporting old configuration while introducing new
>> one. If we make "a big cut" and old way of doing things would not be
>> possible, how are we going to treat this in dtests when we will have stuff
>> for 3.11, 4 on old configs and 5 on new configs?
>> >>
>> >>> On Tue, 22 Feb 2022 at 21:48, Stefan Miklosovic
>> <stefan.mikloso...@instaclustr.com> <stefan.mikloso...@instaclustr.com>
>> wrote:
>> >>>
>> >>> +1 to what Patrick says.
>> >>>
>> >>>> On Tue, 22 Feb 2022 at 21:40, Patrick McFadin <pmcfa...@gmail.com>
>> <pmcfa...@gmail.com> wrote:
>> >>>>
>> >>>> I'm going to put up a red flag of making config file changes of this
>> scale on a dot release. This should really be a 5.0 consideration.
>> >>>>
>> >>>> With that, I would propose a #5. 5.0 nodes will only read the new
>> config files and reject old config files. If any of you went through the
>> config file changes from Apache HTTPd 1.3 -> 2.0 you know how much of a
>> lifesaver that can be for ops. Make it a part of the total upgrade to a new
>> major version, not a radical change inside of a dot version, and make it a
>> clean break. No "legacy config" laying around. That's just a recipe for
>> surprises later if there are new required config values and somebody
>> doesn't even realize they have some old 4.x yaml files laying around.
>> >>>>
>> >>>> Patrick
>> >>>>
>> >>>> On Tue, Feb 22, 2022 at 11:51 AM Tibor Répási
>> <tibor.rep...@anzix.org> <tibor.rep...@anzix.org> wrote:
>> >>>>> Glad to be agree on #4. That feature could be add anytime.
>> >>>>>
>> >>>>> If a version element is added to the YAML, then it is not necessary
>> to change the filename, thus we could end up with #3. The value of the
>> version element could default to 1 in the first phase, which does not need
>> any change for legacy format configuration. New config format must include
>> version: 2. When in some later version the support for legacy configuration
>> is removed, the default for the version element could be changed to 2 or
>> removed.
>> >>>>>
>> >>>>> On 22. Feb 2022, at 19:30, Caleb Rackliffe
>> <calebrackli...@gmail.com> <calebrackli...@gmail.com> wrote:
>> >>>>>
>> >>>>> My initial preference would be something like combining #1 and #4.
>> We could add something like a simple "version: <1|2>" element to the YAML
>> that would eliminate any possible confusion about back-compat within a
>> given file.
>> >>>>>
>> >>>>> Thanks for enumerating these!
>> >>>>>
>> >>>>> On Tue, Feb 22, 2022 at 10:42 AM Tibor Répási
>> <tibor.rep...@anzix.org> <tibor.rep...@anzix.org> wrote:
>> >>>>>> Hi,
>> >>>>>>
>> >>>>>> I like the idea of having cassandra.yaml better structured, as an
>> operator, my primer concern is the transition. How would we change the
>> config structure from legacy to the new one during a rolling upgrade? My
>> thoughts on this:
>> >>>>>>
>> >>>>>> 1. Legacy and new configuration is stored in different files.
>> Cassandra will read the legacy file on startup if it exists, the new one
>> otherwise. May raise warning on startup when legacy was used.
>> >>>>>> pros:
>> >>>>>> - separate files for separate formats
>> >>>>>> - clean and operator controlled switch to new format
>> >>>>>> - already known procedure, e.g. change from PropertyFileSnitch
>> to GossipingPropertyFileSnitch
>> >>>>>> cons:
>> >>>>>> - name of the config file would change from cassandra.yaml to
>> something else (cassandra_v2.yaml, config.yaml ???)
>> >>>>>> - would need considerable work to get config to the new format
>> >>>>>> - format translation not solved
>> >>>>>>
>> >>>>>> 2. Offline configuration converter tool may be available to
>> convert legacy format to new one. During package upgrade, if a legacy
>> config is found, the upgrade process should convert the config file to the
>> new format.
>> >>>>>> pros:
>> >>>>>> - seamless upgrade process
>> >>>>>> - tool can be tested properly before
>> >>>>>> cons:
>> >>>>>> - may interact badly with configuration management tools
>> controlling the contents of cassandra.yaml
>> >>>>>> - poor transparency for operators
>> >>>>>>
>> >>>>>> 3. Cassandra could read both formats, may warn on startup when
>> legacy format found.
>> >>>>>> pros:
>> >>>>>> - no filename change
>> >>>>>> - operator controlled switch to new format
>> >>>>>> cons:
>> >>>>>> - higher complexity at implementation and testing
>> >>>>>> - format translation not solved
>> >>>>>>
>> >>>>>> 4. An online tool, e.g. nodetool command to export the
>> configuration the Cassandra node is currently running with, with filtering
>> option to suppress default settings.
>> >>>>>> pros:
>> >>>>>> - such a nodetool command would be useful independently from
>> changing the config format, could be added before and support any format
>> >>>>>> - the bare information is already available in
>> system_views.settings
>> >>>>>> - could be combined with #1 or #3 to support the format
>> translation
>> >>>>>> cons: ?
>> >>>>>>
>> >>>>>>
>> >>>>>> My favourite would be #3 + #4, while I would most dislike #2.
>> >>>>>>
>> >>>>>> Tibor
>> >>>>>>
>> >>>>>>
>> >>>>>> On 17. Feb 2022, at 23:13, Caleb Rackliffe
>> <calebrackli...@gmail.com> <calebrackli...@gmail.com> wrote:
>> >>>>>>
>> >>>>>> Hey everyone,
>> >>>>>>
>> >>>>>> There has already been some Slack discussion around this, but for
>> anyone who doesn't follow that closely, I'd like to lobby more widely for
>> my proposal in CASSANDRA-17292 to eventually move cassandra.yaml toward a
>> more nested structure.
>> >>>>>>
>> >>>>>> The proposal itself is here, and there has already been some
>> inline discussion, but feel free to drop any feedback there, in the Jira,
>> or here, depending on what you're most comfortable with.
>> >>>>>>
>> >>>>>> Given where we are in the lead-up to 4.1, I have no intention of
>> pushing to adopt any of this for existing config in that release. However,
>> what I think would be nice is if we could come to a rough consensus in time
>> to inform work on new parameters, like those we're planning to add in
>> CASSANDRA-17188.
>> >>>>>>
>> >>>>>> Thanks!
>> >>>>>>
>> >>>>>>
>> >> No PHI in Email: PointClickCare and Collective Medical, A
>> PointClickCare Company, policies prohibit sending protected health
>> information (PHI) by email, which may violate regulatory requirements. If
>> sending PHI is necessary, please contact the sender for secure delivery
>> instructions.
>> >>
>> >> Confidentiality Notice: This email message, including any attachments,
>> is for the sole use of the intended recipient(s) and may contain
>> confidential and privileged information. Any unauthorized review, use,
>> disclosure or distribution is prohibited. If you are not the intended
>> recipient, please contact the sender by reply email and destroy all copies
>> of the original message.
>>
>>
