I am just wondering how to represent in properties things like lists of
non-scalar values?
- - -- --- ----- -------- -------------
Jacek Lewandowski
On Mon, Nov 22, 2021 at 5:16 PM Joseph Lynch <joe.e.ly...@gmail.com> wrote:
> Isn't one of the primary reasons to have a YAML configuration instead
> of a properties file is to allow typed and structured (implies nested)
> configuration? I think it makes a lot of sense to group related
> configuration options (e.g. a feature) into a typed class when we're
> talking about more than one or two related options.
>
> It's pretty standard elsewhere in the JVM ecosystem to encode YAMLs to
> period encoded key->value pairs when required (usually when providing
> a property or override layer), Spring and Elasticsearch yamls both
> come to mind. It seems pretty reasonable to support dot encoding and
> decoding, for example {"a": {"b": 12}} -> '"a.b": 12'.
>
> Regarding quickly telling what configuration a node is running I think
> we should lean on virtual tables for "what is the current
> configuration" now that we have them, as others have said the written
> cassandra.yaml is not necessarily the current configuration ... and
> also grep -C or -A exist for this reason.
>
> -Joey
>
> On Mon, Nov 22, 2021 at 4:14 AM Benjamin Lerer <ble...@apache.org> wrote:
> >
> > I do not have a strong opinion for one or the other but wanted to raise
> the
> > issue I see with the "Settings" virtual table.
> >
> > Currently the "Settings" virtual table converts nested options into flat
> > options using a "_" separator. For those options it allows a user to
> query
> > the all set of options through some hack.
> > If we decide to move to more nesting (more than one level), it seems to
> me
> > that we need to change the way this table is behaving and how we can
> query
> > its data.
> >
> > We would need to start using "." as a nesting separator to ensure that
> > things are consistent between the configuration and the table and add
> > support for LIKE restrictions for filtering queries to allow operators to
> > be able to select the precise set of settings that the operator is
> looking
> > for.
> >
> > Doing so is not really complicated in itself but might impact some users.
> >
> > Le ven. 19 nov. 2021 à 22:39, David Capwell <dcapw...@apple.com.invalid>
> a
> > écrit :
> >
> > > > it is really handy to grep
> > > > cassandra.yaml on some config key and you know the value instantly.
> > >
> > > You can still do that
> > >
> > > $ grep -A2 coordinator_read_size conf/cassandra.yaml
> > > # coordinator_read_size:
> > > # warn_threshold_kb: 0
> > > # abort_threshold_kb: 0
> > >
> > > I was also arguing we should support nested and flat, so if your infra
> > > works better with flat then you could use
> > >
> > > track_warnings.coordinator_read_size.warn_threshold_kb: 0
> > > track_warnings.coordinator_read_size.abort_threshold_kb: 0
> > >
> > > > On Nov 19, 2021, at 1:34 PM, David Capwell <dcapw...@apple.com>
> wrote:
> > > >
> > > >> With the flat structure it turns into properties file - would it be
> > > >> possible to support both formats - nested yaml and flat properties?
> > > >
> > > >
> > > > For majority of our configs yes, but there are a subset where flat
> > > properties is annoying
> > > >
> > > > hinted_handoff_disabled_datacenters - set type, so you could do
> > > hinted_handoff_disabled_datacenters=“a,b,c,d” but we would need to deal
> > > with separators as the format doesn’t support
> > > > seed_provider.parameters - this is a map type… so would need to do
> > > something like seed_provider.parameters=“{\”a\”: \a\”}” …. Maybe we
> special
> > > case maps as dynamic fields? Then seed_provider.parameters.a=a? We
> have
> > > ParameterizedClass all over the code
> > > >
> > > > So, as long as we define how to deal with java collections; we could
> in
> > > theory support properties files (not arguing for that in this thread)
> as
> > > well as system properties.
> > > >
> > > >
> > > >> On Nov 19, 2021, at 1:22 PM, Jacek Lewandowski <
> > > lewandowski.ja...@gmail.com> wrote:
> > > >>
> > > >> With the flat structure it turns into properties file - would it be
> > > >> possible to support both formats - nested yaml and flat properties?
> > > >>
> > > >>
> > > >> - - -- --- ----- -------- -------------
> > > >> Jacek Lewandowski
> > > >>
> > > >>
> > > >> On Fri, Nov 19, 2021 at 10:08 PM Caleb Rackliffe <
> > > calebrackli...@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> If it's nested, "track_warnings" would still work if you're
> grepping
> > > around
> > > >>> vim or less.
> > > >>>
> > > >>> I'd have to concede the point about grep output, although there are
> > > tools
> > > >>> like https://github.com/kislyuk/yq that could probably be bent to
> do
> > > what
> > > >>> you want.
> > > >>>
> > > >>> On Fri, Nov 19, 2021 at 1:08 PM Stefan Miklosovic <
> > > >>> stefan.mikloso...@instaclustr.com> wrote:
> > > >>>
> > > >>>> Hi David,
> > > >>>>
> > > >>>> while I do not oppose nested structure, it is really handy to grep
> > > >>>> cassandra.yaml on some config key and you know the value
> instantly.
> > > >>>> This is not possible when it is nested (easily & fastly) as it is
> on
> > > >>>> two lines. Or maybe my grepping is just not advanced enough to
> cover
> > > >>>> this case? If it is flat, I can just grep "track_warnings" and I
> have
> > > >>>> them all.
> > > >>>>
> > > >>>> Can you elaborate on your last bullet point? Parsing layer ...
> What do
> > > >>>> you mean specifically?
> > > >>>>
> > > >>>> Thanks
> > > >>>>
> > > >>>> On Fri, 19 Nov 2021 at 19:36, David Capwell <dcapw...@gmail.com>
> > > wrote:
> > > >>>>>
> > > >>>>> This has been brought up in a few tickets, so pushing to the dev
> > > list.
> > > >>>>>
> > > >>>>> CASSANDRA-15234 - Standardise config and JVM parameters
> > > >>>>> CASSANDRA-16896 - hard/soft limits for queries
> > > >>>>> CASSANDRA-17147 - Guardrails prototype
> > > >>>>>
> > > >>>>> In short, do we as a project wish to move "new features" into
> nested
> > > >>>>> YAML when the feature has "enough" to justify the nesting? I
> would
> > > >>>>> really like to focus this discussion on new features rather than
> > > >>>>> retroactively grouping (leaving that to CASSANDRA-15234), as
> there is
> > > >>>>> already a place to talk about that.
> > > >>>>>
> > > >>>>> To get things started, let's start with the track-warning feature
> > > >>>>> (hard/soft limits for queries), currently the configs look as
> follows
> > > >>>>> (assuming 15234)
> > > >>>>>
> > > >>>>> track_warnings:
> > > >>>>> enabled: true
> > > >>>>> coordinator_read_size:
> > > >>>>> warn_threshold: 10kb
> > > >>>>> abort_threshold: 1mb
> > > >>>>> local_read_size:
> > > >>>>> warn_threshold: 10kb
> > > >>>>> abort_threshold: 1mb
> > > >>>>> row_index_size:
> > > >>>>> warn_threshold: 100mb
> > > >>>>> abort_threshold: 1gb
> > > >>>>>
> > > >>>>> or should this be "flat"
> > > >>>>>
> > > >>>>> track_warnings_enabled: true
> > > >>>>> track_warnings_coordinator_read_size_warn_threshold: 10kb
> > > >>>>> track_warnings_coordinator_read_size_abort_threshold: 1mb
> > > >>>>> track_warnings_local_read_size_warn_threshold: 10kb
> > > >>>>> track_warnings_local_read_size_abort_threshold: 1mb
> > > >>>>> track_warnings_row_index_size_warn_threshold: 100mb
> > > >>>>> track_warnings_row_index_size_abort_threshold: 1gb
> > > >>>>>
> > > >>>>> For me I prefer nested for a few reasons
> > > >>>>> * easier to enforce consistency as the configs can use shared
> types;
> > > >>>>> in the track warnings patch I had mismatches cross configs (warn
> vs
> > > >>>>> warns, fail vs abort, etc.) before going nested, now everything
> > > reuses
> > > >>>>> the same types
> > > >>>>> * even though it is longer, things can be more clear how they are
> > > >>> related
> > > >>>>> * parsing layer can add support for mixed or purely flat
> depending on
> > > >>>>> user preference (example:
> > > >>>>> track_warnings.row_index_size.abort_threshold, using the '.'
> notation
> > > >>>>> to represent nested structures)
> > > >>>>>
> > > >>>>> Thoughts?
> > > >>>>>
> > > >>>>>
> ---------------------------------------------------------------------
> > > >>>>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > >>>>> For additional commands, e-mail: dev-h...@cassandra.apache.org
> > > >>>>>
> > > >>>>
> > > >>>>
> ---------------------------------------------------------------------
> > > >>>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > >>>> For additional commands, e-mail: dev-h...@cassandra.apache.org
> > > >>>>
> > > >>>>
> > > >>>
> > > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>
