RE: [RFC 00/15] Device Tree schemas and validation
Rob Herring in devicetree/msg06598.html This is interesting approach using the dts syntax, Benoit Cousson in devicetree/msg06617.html it has the big advantage of re-using the parser already included in DTC for free. In term or readability, it avoids to re-defining a brand new syntax for people who are already familiar with the DTS one. Stephen Warren wrote in devicetree/msg06676.html: DT is a language for representing data. The validation checks described by schemas are rules, or code, and not static data. The syntax of the existing device-tree source is strikingly similar to that of the widely used JavaScript Object Notation, better known as JSON. JSON has many parsers, validators and schemata already in existence. Assuredly as many coders know how to program JSON as know C. JSON makes some sense for representation of device-trees, because, as the authoritative json.org explains, JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages . . . JSON is a natural representation of data for the C family of programming languages. Benoit Cousson remarks in devicetree/msg06617.html: The bindings definition being quite open, there is no easy way to ensure proper schema / bindings without careful review of the schema. Stephen Warren continues in devicetree/msg06676.html: Overall, I believe perhaps the single most important aspect of any DT schema is schema inheritance or instancing, David Gibson comments in devicetree/msg06725.html: define the notion of dt patterns or templates. A dt pattern is to a dt node or subtree as a regex is to a string - it provides a reasonably expressive way of defining a family of dt nodes. These would be defined in an extension / superset of dt syntax. I violently agree with Stephen and David and believe that inheritance offers a partial solution to the problem Benoit describes. What about improving compliance by explicitly making use of inheritance with device-tree include files? Suppose we consider supplementing the ARM tree's skeleton.dtsi tree-root with board.dtsi, cpu.dtsi, daughtercard.dtsi . . . Suppose then we require board-level dts files to include board.dtsi, and furthermore that the CPU node for the board to be described in a DTSI file that must itself include cpu.dtsi. Then the dtc itself could in effect check perform some constraint checking. Device vendors could offer arch.dtsi and particular chip.dtsi that users of those products would be required to include. A related question: what DTS files will a validator compare against a schema? Assuredly given the existing ability of nodes and properties in a hierarchy of include files to override and modify one another, the post C-preprocessed and compiled single-file is the one wanted, that is, the output from $CC $CPP_FLAGS -o foo.tmp foo.dts dtc -O dts -i arch/arm/boot/dts foo.tmp Some other suggestions: Let's not make the documentation derived from schemata and DTS files tree-structured.Otherwise we end up with GNU info, ahem. Stephen Warren previously contributed a useful bindings checklist. We should try to roll that checklist into any validator. Benoit promises in devicetree/msg06617.html: Being the very first one, you might get a free beer... meaning there might be such thing as a free beer :-) I'm presenting a talk about device-tree for Embedded Linux Conference Europe, http://sched.co/17ozhPE, and hope that some of you come flame me in person there. If so, I will actually buy you a free beer, but not until after the second seminar I must give that afternoon! -- Alison Chaiken Mentor Embedded Software Division Fremont, CA GMT-8 -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC 00/15] Device Tree schemas and validation
On 02/10/2013 15:54, David Gibson wrote:
On Tue, Oct 01, 2013 at 03:54:20PM -0500, Rob Herring wrote:
On Tue, Oct 1, 2013 at 10:06 AM, Benoit Cousson bcous...@baylibre.com wrote:
Hi Rob,
On 01/10/2013 15:17, Rob Herring wrote:
On 10/01/2013 03:06 AM, Benoit Cousson wrote:
+ more DT maintainers folks
Hi all,
I know this is mostly boring user space code, but I was expecting a
little bit of comments about at least the bindings syntax:-(
I'd like to know if this is the right direction and if it worth pursuing
in that direction.
The idea was to have at least some base for further discussion during
ARM KS 2013.
I feel alone :-(
If you have any comment, go ahead!
Thanks for taking this on!
This is interesting approach using the dts syntax,
Well, this was discussed a little bit on the list, and it has the big
advantage of re-using the parser already included in DTC for free.
In term or readability, it avoids to re-defining a brand new syntax for
people who are already familiar with the DTS one.
but I worry that the
validation will only be as good as the schema written and the review of
the schema.
Well, sure, but unfortunately, that will always be the case :-(
The bindings definition being quite open, there is no easy way to ensure
proper schema / bindings without careful review of the schema. There is no
such thing as a free beer... Unfortunately :-)
I think the schema needs to define the binding rather than
define the checks. Then the schema can feed the validation checks.
This format does not seem to me as easily being able to generate
documentation from the schema which I believe is one of the goals.
Indeed, but I think is it easy to generate any kind of readable format for
the documentation purpose if needed from the actual format.
Otherwise, we should consider a schema format based on kerneldoc type of
syntax to improve readability. I'm just afraid it will become harder then to
define complex schema.
BTW, what kind of documentation are you expecting here? Is is a text that
can be added on top of each schema?
I would expect the schema to replace
Documentation/devicetree/bindings/* over time. I think the thing that
needs to be worked out here is how to add free form multi-line text.
I'm not convinced that's a realistic goal. As I see it, the
fundamental difference between a binding document and a formal schema
is that a binding defines both the syntax required of a node, and its
semantics, whereas a schema defines only syntax - the semantics still
need to be defined somewhere.
Mmm, I do not really understand what is missing in the following schema
in term of semantic compared to the original interrupt-controller bindings?
interrupt-controller {
description = Identifies the node as an interrupt controller;
is-required;
type = bool;
};
#interrupt-cells {
description = Specifies the number of cells needed to encode an
interrupt source. The type shall be a u32
and the value shall be 1.
The cell contains the interrupt number
in the range [0-128].;
is-required;
type = integer;
value = 1;
};
As soon as you have a description you can keep the content of the
original binding. But on top of that, you do have the schema validation
information.
I'm not sure to understand your point. Could you elaborate?
Regards,
Benoit
--
To unsubscribe from this list: send the line unsubscribe linux-omap in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC 00/15] Device Tree schemas and validation
Hi Stephen,
On 02/10/2013 00:22, Stephen Warren wrote:
On 09/24/2013 10:52 AM, Benoit Cousson wrote:
Hi All,
Following the discussion that happened during LCE-2013 and the email
thread started by Tomasz few months ago [1], here is a first attempt
to introduce:
- a schema language to define the bindings accurately
- DTS validation during device tree compilation in DTC itself
Sorry, this is probably going to sound a bit negative. Hopefully you
find it constructive though.
Well, I hope so too, let's see at the end of the email :-)
The syntax for a schema is the same as the one for dts. This choice has
been made to simplify its development, to maximize the code reuse and
finally because the format is human-readable.
I'm not convinced that's a good decision.
Me neither :-), but I gave the current rational...
DT is a language for representing data.
The validation checks described by schemas are rules, or code, and not
static data.
I will not be that strict with what DTS is supposed to do. In that
aspect DTS is just a way to represent information in a structured
hierarchical way.
It is for my point of view no different than XML. I know that everybody
hate XML, including me, but that shows at least what is doable with such
language. The fact that we like it or not is a different topic.
So, while I'm sure it's possible to shoe-horn at least some reasonable
subset of DT validation into DT syntax itself, I feel it's unlikely to
yield something that's scalable enough.
I don't think we have any limit with such representation. My concern
will be more the readability.
To be honest, a language choice is by nature completely subjective, and
nobody will have the same taste. So we can spend weeks arguing about
that :-)
For example, it's easy to specify that a property must be 2 cells long.
What if it could be any multiple of two? That's a lot of numbers to
explicitly enumerate as data. Sure, you can then invent syntax to
represent that specific rule (parameterized by 2), but what about the
next similar-but-different rule? The only approach I can think of to
that is to allow the schema to contain arbitrary expressions, which
would likely need to morph into arbitary statements not just
expressions. Once you're there, I think the schema would be better
represented as a programming language rather than as a data structure
that could have code hooked into it.
Sure, but how many complex cases like that do we have? I guess, we can
handle all the use-cases required by Rob with the current syntax.
Let's assume we cover 99% of the use-cases with such language, do we
really want to have a super complex language just for the corner cases?
Potentially, writing a C extension to DTC is still possible for that
specific case.
Not ideal, I do agree, but we have to be pragmatic as well.
We really need to understand how scalable we have to be before deciding
that the current representation is not good enough.
How to:
* Associate a schema to one or several nodes
As said earlier a schema can be used to validate one or several nodes
from a dts. To do this the compatible properties from the nodes which
should be validated must be present in the schema.
timer1: timer@4a318000 {
compatible = ti,omap3430-timer;
...
To write a schema which will validate OMAP Timers like the one above,
one may write the following schema:
/dts-v1/;
/ {
compatible = ti,omap[0-9]+-timer;
What about DT nodes that don't have a compatible value? We certainly
have some of those already like /memory and /chosen. We should be able
to validate their schema too. This probably doesn't invalidate being
able to look things up by compatible value though; it just means we need
some additional mechanisms too.
Yes, that's a good point and easy to add as well.
* Define constraints on properties
To define constraints on a property one has to create a node in a schema
which has as name the name of the property that one want to validate.
To specify constraints on the property ti,hwmods of OMAP Timers one
can write this schema:
/dts-v1/;
/ {
compatible = ti,omap[0-9]+-timer;
ti,hwmods {
...
};
compatible and ti,hwmods are both properties in the DT file. However, in
the schema above, one appears as a property, and one as a node. I don't
like that inconsistency. It'd be better if compatible was a node too.
That's already possible, you can check the timer.schema. The point is to
simplify the representation for simple case and use a attribute instead
of a node. But that will make 2 different representation for the same
case, which might not be that good.
If one want to use a regular as property name one can write this schema:
/dts-v1/;
/ {
compatible = abc;
def {
name = def[0-9];
Isn't it valid to have a
Re: [RFC 00/15] Device Tree schemas and validation
Hi David,
On 02/10/2013 16:29, David Gibson wrote:
On Tue, Oct 01, 2013 at 04:22:24PM -0600, Stephen Warren wrote:
On 09/24/2013 10:52 AM, Benoit Cousson wrote:
Hi All,
Following the discussion that happened during LCE-2013 and the email
thread started by Tomasz few months ago [1], here is a first attempt
to introduce:
- a schema language to define the bindings accurately
- DTS validation during device tree compilation in DTC itself
Sorry, this is probably going to sound a bit negative. Hopefully you
find it constructive though.
The syntax for a schema is the same as the one for dts. This choice has
been made to simplify its development, to maximize the code reuse and
finally because the format is human-readable.
I'm not convinced that's a good decision.
DT is a language for representing data.
The validation checks described by schemas are rules, or code, and not
static data.
So, while I'm sure it's possible to shoe-horn at least some reasonable
subset of DT validation into DT syntax itself, I feel it's unlikely to
yield something that's scalable enough.
I tend to agree.
For example, it's easy to specify that a property must be 2 cells long.
What if it could be any multiple of two? That's a lot of numbers to
explicitly enumerate as data. Sure, you can then invent syntax to
represent that specific rule (parameterized by 2), but what about the
next similar-but-different rule? The only approach I can think of to
that is to allow the schema to contain arbitrary expressions, which
would likely need to morph into arbitary statements not just
expressions. Once you're there, I think the schema would be better
represented as a programming language rather than as a data structure
that could have code hooked into it.
How to:
* Associate a schema to one or several nodes
As said earlier a schema can be used to validate one or several nodes
from a dts. To do this the compatible properties from the nodes which
should be validated must be present in the schema.
timer1: timer@4a318000 {
compatible = ti,omap3430-timer;
...
To write a schema which will validate OMAP Timers like the one above,
one may write the following schema:
/dts-v1/;
/ {
compatible = ti,omap[0-9]+-timer;
What about DT nodes that don't have a compatible value? We certainly
have some of those already like /memory and /chosen. We should be able
to validate their schema too. This probably doesn't invalidate being
able to look things up by compatible value though; it just means we need
some additional mechanisms too.
More to the point, what about the properties of a node whose format is
defined not by this node's binding but by some other nodes binding.
e.g. the exact format of reg and ranges is at least partially
determined by the parent bus's binding, and interrupts is defined
partially by the interrupt parent's binding. gpio properties are
defined by a combination of a global binding and the gpio parent,
IIRC.
Yeah, that's a general concern that Stephen raised several time as well.
We need to figure out some way to handle that.
* Define constraints on properties
To define constraints on a property one has to create a node in a schema
which has as name the name of the property that one want to validate.
To specify constraints on the property ti,hwmods of OMAP Timers one
can write this schema:
/dts-v1/;
/ {
compatible = ti,omap[0-9]+-timer;
ti,hwmods {
...
};
compatible and ti,hwmods are both properties in the DT file. However, in
the schema above, one appears as a property, and one as a node. I don't
like that inconsistency. It'd be better if compatible was a node too.
Essentially what's going on here is that to describe the constraint on
a property, a node with corresponding name is defined to encode the
parameters of that constraint. It kind of works, but it's forced. It
also hits problems since nodes and properties are technically in
different namespaces, although they rarely collide in real cases.
OK, so would you suggest keeping mapping between node / attribute in DTS
and in the schema?
If one want to use a regular as property name one can write this schema:
/dts-v1/;
/ {
compatible = abc;
def {
name = def[0-9];
Isn't it valid to have a property named name within the node itself?
How do you differentiate between specifying the node name and the name
property?
Or to look at it another way, how do you differentiate between nodes
representing encoded constraints for a property, and nodes
representing nodes directly.
What if the node name needs more validation than just a regex. For
example, suppose we want to validate the
unit-name-must-match-reg-address rule. We need to write some complex
expression using data extracted from reg to calculate the unit address.
Equally, the node name
Re: [RFC 00/15] Device Tree schemas and validation
On Tue, Oct 01, 2013 at 10:17:53AM -0500, Jon Loeliger wrote: Hi Rob, On 01/10/2013 15:17, Rob Herring wrote: On 10/01/2013 03:06 AM, Benoit Cousson wrote: + more DT maintainers folks Hi all, I know this is mostly boring user space code, but I was expecting a little bit of comments about at least the bindings syntax:-( I'd like to know if this is the right direction and if it worth pursuing in that direction. The idea was to have at least some base for further discussion during ARM KS 2013. I feel alone :-( If you have any comment, go ahead! Benoit, Sorry, I meant to ask earlier but forgot. Shouldn't this development be based on the upstream DTC repository and not the in-kernel copy of the DTC? Absolutely. Please work against upstream dtc. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgpvQSOHC9Pwv.pgp Description: PGP signature
Re: [RFC 00/15] Device Tree schemas and validation
Hi Jon, On 01/10/2013 17:17, Jon Loeliger wrote: Hi Rob, On 01/10/2013 15:17, Rob Herring wrote: On 10/01/2013 03:06 AM, Benoit Cousson wrote: + more DT maintainers folks Hi all, I know this is mostly boring user space code, but I was expecting a little bit of comments about at least the bindings syntax:-( I'd like to know if this is the right direction and if it worth pursuing in that direction. The idea was to have at least some base for further discussion during ARM KS 2013. I feel alone :-( If you have any comment, go ahead! Benoit, Sorry, I meant to ask earlier but forgot. Shouldn't this development be based on the upstream DTC repository and not the in-kernel copy of the DTC? Eventually, yes, but here the main point is to discuss the schema that will be used to defined bindings. In that case, the DTC patches code are mostly a proof of concept using the Linux kernel as example. Regards, Benoit -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC 00/15] Device Tree schemas and validation
Benoit, Sorry, I meant to ask earlier but forgot. Shouldn't this development be based on the upstream DTC repository and not the in-kernel copy of the DTC? Eventually, yes, We should *start* there, though. but here the main point is to discuss the schema that will be used to defined bindings. In that case, the DTC patches code are mostly a proof of concept Sure; all that is fine. using the Linux kernel as example. Example?(*) We shouldn't churn the kernel. jdl (*) In the sense that *this* Universe is just one example from all known Universes? :-) -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC 00/15] Device Tree schemas and validation
On Tue, Oct 01, 2013 at 03:54:20PM -0500, Rob Herring wrote: On Tue, Oct 1, 2013 at 10:06 AM, Benoit Cousson bcous...@baylibre.com wrote: Hi Rob, On 01/10/2013 15:17, Rob Herring wrote: On 10/01/2013 03:06 AM, Benoit Cousson wrote: + more DT maintainers folks Hi all, I know this is mostly boring user space code, but I was expecting a little bit of comments about at least the bindings syntax:-( I'd like to know if this is the right direction and if it worth pursuing in that direction. The idea was to have at least some base for further discussion during ARM KS 2013. I feel alone :-( If you have any comment, go ahead! Thanks for taking this on! This is interesting approach using the dts syntax, Well, this was discussed a little bit on the list, and it has the big advantage of re-using the parser already included in DTC for free. In term or readability, it avoids to re-defining a brand new syntax for people who are already familiar with the DTS one. but I worry that the validation will only be as good as the schema written and the review of the schema. Well, sure, but unfortunately, that will always be the case :-( The bindings definition being quite open, there is no easy way to ensure proper schema / bindings without careful review of the schema. There is no such thing as a free beer... Unfortunately :-) I think the schema needs to define the binding rather than define the checks. Then the schema can feed the validation checks. This format does not seem to me as easily being able to generate documentation from the schema which I believe is one of the goals. Indeed, but I think is it easy to generate any kind of readable format for the documentation purpose if needed from the actual format. Otherwise, we should consider a schema format based on kerneldoc type of syntax to improve readability. I'm just afraid it will become harder then to define complex schema. BTW, what kind of documentation are you expecting here? Is is a text that can be added on top of each schema? I would expect the schema to replace Documentation/devicetree/bindings/* over time. I think the thing that needs to be worked out here is how to add free form multi-line text. I'm not convinced that's a realistic goal. As I see it, the fundamental difference between a binding document and a formal schema is that a binding defines both the syntax required of a node, and its semantics, whereas a schema defines only syntax - the semantics still need to be defined somewhere. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgp0yD2z4Q1JA.pgp Description: PGP signature
Re: [RFC 00/15] Device Tree schemas and validation
On Tue, Oct 01, 2013 at 08:17:42AM -0500, Rob Herring wrote: On 10/01/2013 03:06 AM, Benoit Cousson wrote: + more DT maintainers folks Hi all, I know this is mostly boring user space code, but I was expecting a little bit of comments about at least the bindings syntax:-( I'd like to know if this is the right direction and if it worth pursuing in that direction. The idea was to have at least some base for further discussion during ARM KS 2013. I feel alone :-( If you have any comment, go ahead! Thanks for taking this on! This is interesting approach using the dts syntax, but I worry that the validation will only be as good as the schema written and the review of the schema. I think the schema needs to define the binding rather than define the checks. Then the schema can feed the validation checks. This format does not seem to me as easily being able to generate documentation from the schema which I believe is one of the goals. I for one don't care to review the documentation and the schema for every binding. Hrm. I'm less optimistic about entirely replacing human-readable bindings with machine-readable schemas. But I do think the schema language needs to be substantially more flexible than the draft presented here. While I think a schema syntax which mirrors dts syntax makes a lot of sense, actually defining schemas as device trees doesn't seem quite right to me. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgps4GNtO_6_u.pgp Description: PGP signature
Re: [RFC 00/15] Device Tree schemas and validation
On Tue, Oct 01, 2013 at 04:22:24PM -0600, Stephen Warren wrote:
On 09/24/2013 10:52 AM, Benoit Cousson wrote:
Hi All,
Following the discussion that happened during LCE-2013 and the email
thread started by Tomasz few months ago [1], here is a first attempt
to introduce:
- a schema language to define the bindings accurately
- DTS validation during device tree compilation in DTC itself
Sorry, this is probably going to sound a bit negative. Hopefully you
find it constructive though.
The syntax for a schema is the same as the one for dts. This choice has
been made to simplify its development, to maximize the code reuse and
finally because the format is human-readable.
I'm not convinced that's a good decision.
DT is a language for representing data.
The validation checks described by schemas are rules, or code, and not
static data.
So, while I'm sure it's possible to shoe-horn at least some reasonable
subset of DT validation into DT syntax itself, I feel it's unlikely to
yield something that's scalable enough.
I tend to agree.
For example, it's easy to specify that a property must be 2 cells long.
What if it could be any multiple of two? That's a lot of numbers to
explicitly enumerate as data. Sure, you can then invent syntax to
represent that specific rule (parameterized by 2), but what about the
next similar-but-different rule? The only approach I can think of to
that is to allow the schema to contain arbitrary expressions, which
would likely need to morph into arbitary statements not just
expressions. Once you're there, I think the schema would be better
represented as a programming language rather than as a data structure
that could have code hooked into it.
How to:
* Associate a schema to one or several nodes
As said earlier a schema can be used to validate one or several nodes
from a dts. To do this the compatible properties from the nodes which
should be validated must be present in the schema.
timer1: timer@4a318000 {
compatible = ti,omap3430-timer;
...
To write a schema which will validate OMAP Timers like the one above,
one may write the following schema:
/dts-v1/;
/ {
compatible = ti,omap[0-9]+-timer;
What about DT nodes that don't have a compatible value? We certainly
have some of those already like /memory and /chosen. We should be able
to validate their schema too. This probably doesn't invalidate being
able to look things up by compatible value though; it just means we need
some additional mechanisms too.
More to the point, what about the properties of a node whose format is
defined not by this node's binding but by some other nodes binding.
e.g. the exact format of reg and ranges is at least partially
determined by the parent bus's binding, and interrupts is defined
partially by the interrupt parent's binding. gpio properties are
defined by a combination of a global binding and the gpio parent,
IIRC.
* Define constraints on properties
To define constraints on a property one has to create a node in a schema
which has as name the name of the property that one want to validate.
To specify constraints on the property ti,hwmods of OMAP Timers one
can write this schema:
/dts-v1/;
/ {
compatible = ti,omap[0-9]+-timer;
ti,hwmods {
...
};
compatible and ti,hwmods are both properties in the DT file. However, in
the schema above, one appears as a property, and one as a node. I don't
like that inconsistency. It'd be better if compatible was a node too.
Essentially what's going on here is that to describe the constraint on
a property, a node with corresponding name is defined to encode the
parameters of that constraint. It kind of works, but it's forced. It
also hits problems since nodes and properties are technically in
different namespaces, although they rarely collide in real cases.
If one want to use a regular as property name one can write this schema:
/dts-v1/;
/ {
compatible = abc;
def {
name = def[0-9];
Isn't it valid to have a property named name within the node itself?
How do you differentiate between specifying the node name and the name
property?
Or to look at it another way, how do you differentiate between nodes
representing encoded constraints for a property, and nodes
representing nodes directly.
What if the node name needs more validation than just a regex. For
example, suppose we want to validate the
unit-name-must-match-reg-address rule. We need to write some complex
expression using data extracted from reg to calculate the unit address.
Equally, the node name perhaps has to exist in some global list of
acceptable node names. It would be extremely tricky if not impossible to
do that with a regex.
...
};
};
Above one can see
Re: [RFC 00/15] Device Tree schemas and validation
On Wed, Oct 02, 2013 at 11:54:50PM +1000, David Gibson wrote: On Tue, Oct 01, 2013 at 03:54:20PM -0500, Rob Herring wrote: I would expect the schema to replace Documentation/devicetree/bindings/* over time. I think the thing that needs to be worked out here is how to add free form multi-line text. I'm not convinced that's a realistic goal. As I see it, the fundamental difference between a binding document and a formal schema is that a binding defines both the syntax required of a node, and its semantics, whereas a schema defines only syntax - the semantics still need to be defined somewhere. So long as the schema lets you include free form text to define the semantics I'm not sure there's an incompatibility there - the same document can cover both. signature.asc Description: Digital signature
Re: [RFC 00/15] Device Tree schemas and validation
On Wed, Oct 02, 2013 at 07:08:41PM +0100, Mark Brown wrote: On Wed, Oct 02, 2013 at 11:54:50PM +1000, David Gibson wrote: On Tue, Oct 01, 2013 at 03:54:20PM -0500, Rob Herring wrote: I would expect the schema to replace Documentation/devicetree/bindings/* over time. I think the thing that needs to be worked out here is how to add free form multi-line text. I'm not convinced that's a realistic goal. As I see it, the fundamental difference between a binding document and a formal schema is that a binding defines both the syntax required of a node, and its semantics, whereas a schema defines only syntax - the semantics still need to be defined somewhere. So long as the schema lets you include free form text to define the semantics I'm not sure there's an incompatibility there - the same document can cover both. True, there's no reason the machine-readable schema and human-readable documentation can't be contained in the same file. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson pgpv8VA08IhkS.pgp Description: PGP signature
Re: [RFC 00/15] Device Tree schemas and validation
+ more DT maintainers folks
Hi all,
I know this is mostly boring user space code, but I was expecting a
little bit of comments about at least the bindings syntax:-(
I'd like to know if this is the right direction and if it worth pursuing
in that direction.
The idea was to have at least some base for further discussion during
ARM KS 2013.
I feel alone :-(
If you have any comment, go ahead!
Thanks,
Benoit
On 24/09/2013 18:52, Benoit Cousson wrote:
Hi All,
Following the discussion that happened during LCE-2013 and the email
thread started by Tomasz few months ago [1], here is a first attempt
to introduce:
- a schema language to define the bindings accurately
- DTS validation during device tree compilation in DTC itself
[1] http://www.spinics.net/lists/arm-kernel/msg262224.html
=== What it does? ===
For now device-tree bindings are defined in a not standardized
text-based format and thus any one can write the documentation for a
binding as he wish. In addition to this there is no automated way to
check if a dts is conform to the available bindings.
The goal of this series of patch is to fix this situation by adding a
well defined way to write bindings in a human-readable format. These
bindings will be written through files called schemas.
=== What is a schema? ===
A schema is a file describing the constraints that one or several nodes
of a dts must conform to. Schemas must use the file extension .schema.
A schema can check that:
- A node has a given property
- An array has a valid size
- A node contains the required children.
- A property has the correct type.
- A property has the correct value.
A schema can as well recursively check the constraints for parent nodes.
The syntax for a schema is the same as the one for dts. This choice has
been made to simplify its development, to maximize the code reuse and
finally because the format is human-readable.
=== How to defined a schema? ===
A binding directory has been added at the root of repository. Users can
add schema files anywhere in it but a nice way would be to keep the same
structure as the binding directory in the documentation.
To demonstrate how to write a schema and its capability I will use the
OMAP DTS schemas when applicable.
How to:
* Associate a schema to one or several nodes
As said earlier a schema can be used to validate one or several nodes
from a dts. To do this the compatible properties from the nodes which
should be validated must be present in the schema.
timer1: timer@4a318000 {
compatible = ti,omap3430-timer;
reg = 0x4a318000 0x80;
interrupts = 0x0 0x25 0x4;
ti,hwmods = timer1;
ti,timer-alwon;
};
To write a schema which will validate OMAP Timers like the one above,
one may write the following schema:
/dts-v1/;
/ {
compatible = ti,omap[0-9]+-timer;
...
};
The schema above will be used to validate every node in a dts which has
a compatible matching the following regular expression:
ti,omap[0-9]+-timer.
It is possible to specify multiple compatible inside a schema using this
syntax:
compatible = ti,omap[0-9]+-timer, ti,am[0-9]+-timer;
This time the schema will be application for both OMAP Timers and AM
Timers.
* Define constraints on properties
To define constraints on a property one has to create a node in a schema
which has as name the name of the property that one want to validate.
To specify constraints on the property ti,hwmods of OMAP Timers one
can write this schema:
/dts-v1/;
/ {
compatible = ti,omap[0-9]+-timer;
ti,hwmods {
...
};
};
If one want to use a regular as property name one can write this schema:
/dts-v1/;
/ {
compatible = abc;
def {
name = def[0-9];
...
};
};
Above one can see that the name property override the node name.
* Require the presence of a property
One can require the presence of a property by using the is-required
constraint.
/dts-v1/;
/ {
compatible = ti,omap[0-9]+-timer;
ti,hwmods {
is-required;
};
};
The ti,hwmods property above is set as required and its presence will
be checked in every OMAP timer.
* Require the presence of a property inside a node or inside one of its
parents
Sometimes a property required is not directly present inside a node but
is present in one of its parents. To check this, one can use
can-be-inherited in addition to is-required.
twl {
interrupt-controller;
rtc {
compatible = ti,twl4030-rtc;
interrupts = 0xc;
};
}
In the case of the rtc above the interrupt-controller is not present,
Re: [RFC 00/15] Device Tree schemas and validation
On 10/01/2013 03:06 AM, Benoit Cousson wrote:
+ more DT maintainers folks
Hi all,
I know this is mostly boring user space code, but I was expecting a
little bit of comments about at least the bindings syntax:-(
I'd like to know if this is the right direction and if it worth pursuing
in that direction.
The idea was to have at least some base for further discussion during
ARM KS 2013.
I feel alone :-(
If you have any comment, go ahead!
Thanks for taking this on!
This is interesting approach using the dts syntax, but I worry that the
validation will only be as good as the schema written and the review of
the schema. I think the schema needs to define the binding rather than
define the checks. Then the schema can feed the validation checks. This
format does not seem to me as easily being able to generate
documentation from the schema which I believe is one of the goals. I for
one don't care to review the documentation and the schema for every binding.
I would like to ensure we can start with the basics and have some
generic rules. Some examples:
- flag compatible strings in dts files that are not documented
- check required properties
- check properties against cell sizes
- a node with reg property must have a unit address
- flag properties with _
- check properties are the correct data type
Rob
Thanks,
Benoit
On 24/09/2013 18:52, Benoit Cousson wrote:
Hi All,
Following the discussion that happened during LCE-2013 and the email
thread started by Tomasz few months ago [1], here is a first attempt
to introduce:
- a schema language to define the bindings accurately
- DTS validation during device tree compilation in DTC itself
[1] http://www.spinics.net/lists/arm-kernel/msg262224.html
=== What it does? ===
For now device-tree bindings are defined in a not standardized
text-based format and thus any one can write the documentation for a
binding as he wish. In addition to this there is no automated way to
check if a dts is conform to the available bindings.
The goal of this series of patch is to fix this situation by adding a
well defined way to write bindings in a human-readable format. These
bindings will be written through files called schemas.
=== What is a schema? ===
A schema is a file describing the constraints that one or several nodes
of a dts must conform to. Schemas must use the file extension .schema.
A schema can check that:
- A node has a given property
- An array has a valid size
- A node contains the required children.
- A property has the correct type.
- A property has the correct value.
A schema can as well recursively check the constraints for parent nodes.
The syntax for a schema is the same as the one for dts. This choice has
been made to simplify its development, to maximize the code reuse and
finally because the format is human-readable.
=== How to defined a schema? ===
A binding directory has been added at the root of repository. Users can
add schema files anywhere in it but a nice way would be to keep the same
structure as the binding directory in the documentation.
To demonstrate how to write a schema and its capability I will use the
OMAP DTS schemas when applicable.
How to:
* Associate a schema to one or several nodes
As said earlier a schema can be used to validate one or several nodes
from a dts. To do this the compatible properties from the nodes which
should be validated must be present in the schema.
timer1: timer@4a318000 {
compatible = ti,omap3430-timer;
reg = 0x4a318000 0x80;
interrupts = 0x0 0x25 0x4;
ti,hwmods = timer1;
ti,timer-alwon;
};
To write a schema which will validate OMAP Timers like the one above,
one may write the following schema:
/dts-v1/;
/ {
compatible = ti,omap[0-9]+-timer;
...
};
The schema above will be used to validate every node in a dts which has
a compatible matching the following regular expression:
ti,omap[0-9]+-timer.
It is possible to specify multiple compatible inside a schema using this
syntax:
compatible = ti,omap[0-9]+-timer, ti,am[0-9]+-timer;
This time the schema will be application for both OMAP Timers and AM
Timers.
* Define constraints on properties
To define constraints on a property one has to create a node in a schema
which has as name the name of the property that one want to validate.
To specify constraints on the property ti,hwmods of OMAP Timers one
can write this schema:
/dts-v1/;
/ {
compatible = ti,omap[0-9]+-timer;
ti,hwmods {
...
};
};
If one want to use a regular as property name one can write this schema:
/dts-v1/;
/ {
compatible = abc;
def {
name = def[0-9];
...
};
};
Above one can see that the name property override the node name.
* Require the presence of a property
Re: [RFC 00/15] Device Tree schemas and validation
Hi Rob,
On 01/10/2013 15:17, Rob Herring wrote:
On 10/01/2013 03:06 AM, Benoit Cousson wrote:
+ more DT maintainers folks
Hi all,
I know this is mostly boring user space code, but I was expecting a
little bit of comments about at least the bindings syntax:-(
I'd like to know if this is the right direction and if it worth pursuing
in that direction.
The idea was to have at least some base for further discussion during
ARM KS 2013.
I feel alone :-(
If you have any comment, go ahead!
Thanks for taking this on!
This is interesting approach using the dts syntax,
Well, this was discussed a little bit on the list, and it has the big
advantage of re-using the parser already included in DTC for free.
In term or readability, it avoids to re-defining a brand new syntax for
people who are already familiar with the DTS one.
but I worry that the
validation will only be as good as the schema written and the review of
the schema.
Well, sure, but unfortunately, that will always be the case :-(
The bindings definition being quite open, there is no easy way to ensure
proper schema / bindings without careful review of the schema. There is
no such thing as a free beer... Unfortunately :-)
I think the schema needs to define the binding rather than
define the checks. Then the schema can feed the validation checks.
This format does not seem to me as easily being able to generate
documentation from the schema which I believe is one of the goals.
Indeed, but I think is it easy to generate any kind of readable format
for the documentation purpose if needed from the actual format.
Otherwise, we should consider a schema format based on kerneldoc type of
syntax to improve readability. I'm just afraid it will become harder
then to define complex schema.
BTW, what kind of documentation are you expecting here? Is is a text
that can be added on top of each schema?
I for
one don't care to review the documentation and the schema for every binding.
I would like to ensure we can start with the basics and have some
generic rules. Some examples:
- flag compatible strings in dts files that are not documented
This is almost done with the last patch of the series. Nodes are already
checked, properties are missing.
- check required properties
Done as well.
- check properties against cell sizes
Yeah, that one is still to be done.
- a node with reg property must have a unit address
Easy to add.
- flag properties with _
Easy to add as well.
- check properties are the correct data type
Already done for a couple of data type.
Thanks for your comments. Being the very first one, you might get a free
beer... meaning there might be such thing as a free beer :-)
Thanks,
Benoit
Thanks,
Benoit
On 24/09/2013 18:52, Benoit Cousson wrote:
Hi All,
Following the discussion that happened during LCE-2013 and the email
thread started by Tomasz few months ago [1], here is a first attempt
to introduce:
- a schema language to define the bindings accurately
- DTS validation during device tree compilation in DTC itself
[1] http://www.spinics.net/lists/arm-kernel/msg262224.html
=== What it does? ===
For now device-tree bindings are defined in a not standardized
text-based format and thus any one can write the documentation for a
binding as he wish. In addition to this there is no automated way to
check if a dts is conform to the available bindings.
The goal of this series of patch is to fix this situation by adding a
well defined way to write bindings in a human-readable format. These
bindings will be written through files called schemas.
=== What is a schema? ===
A schema is a file describing the constraints that one or several nodes
of a dts must conform to. Schemas must use the file extension .schema.
A schema can check that:
- A node has a given property
- An array has a valid size
- A node contains the required children.
- A property has the correct type.
- A property has the correct value.
A schema can as well recursively check the constraints for parent nodes.
The syntax for a schema is the same as the one for dts. This choice has
been made to simplify its development, to maximize the code reuse and
finally because the format is human-readable.
=== How to defined a schema? ===
A binding directory has been added at the root of repository. Users can
add schema files anywhere in it but a nice way would be to keep the same
structure as the binding directory in the documentation.
To demonstrate how to write a schema and its capability I will use the
OMAP DTS schemas when applicable.
How to:
* Associate a schema to one or several nodes
As said earlier a schema can be used to validate one or several nodes
from a dts. To do this the compatible properties from the nodes which
should be validated must be present in the schema.
timer1: timer@4a318000 {
compatible = ti,omap3430-timer;
reg = 0x4a318000 0x80;
interrupts = 0x0
Re: [RFC 00/15] Device Tree schemas and validation
Hi Rob, On 01/10/2013 15:17, Rob Herring wrote: On 10/01/2013 03:06 AM, Benoit Cousson wrote: + more DT maintainers folks Hi all, I know this is mostly boring user space code, but I was expecting a little bit of comments about at least the bindings syntax:-( I'd like to know if this is the right direction and if it worth pursuing in that direction. The idea was to have at least some base for further discussion during ARM KS 2013. I feel alone :-( If you have any comment, go ahead! Benoit, Sorry, I meant to ask earlier but forgot. Shouldn't this development be based on the upstream DTC repository and not the in-kernel copy of the DTC? Thanks, jdl -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC 00/15] Device Tree schemas and validation
On Tue, Oct 1, 2013 at 10:06 AM, Benoit Cousson bcous...@baylibre.com wrote: Hi Rob, On 01/10/2013 15:17, Rob Herring wrote: On 10/01/2013 03:06 AM, Benoit Cousson wrote: + more DT maintainers folks Hi all, I know this is mostly boring user space code, but I was expecting a little bit of comments about at least the bindings syntax:-( I'd like to know if this is the right direction and if it worth pursuing in that direction. The idea was to have at least some base for further discussion during ARM KS 2013. I feel alone :-( If you have any comment, go ahead! Thanks for taking this on! This is interesting approach using the dts syntax, Well, this was discussed a little bit on the list, and it has the big advantage of re-using the parser already included in DTC for free. In term or readability, it avoids to re-defining a brand new syntax for people who are already familiar with the DTS one. but I worry that the validation will only be as good as the schema written and the review of the schema. Well, sure, but unfortunately, that will always be the case :-( The bindings definition being quite open, there is no easy way to ensure proper schema / bindings without careful review of the schema. There is no such thing as a free beer... Unfortunately :-) I think the schema needs to define the binding rather than define the checks. Then the schema can feed the validation checks. This format does not seem to me as easily being able to generate documentation from the schema which I believe is one of the goals. Indeed, but I think is it easy to generate any kind of readable format for the documentation purpose if needed from the actual format. Otherwise, we should consider a schema format based on kerneldoc type of syntax to improve readability. I'm just afraid it will become harder then to define complex schema. BTW, what kind of documentation are you expecting here? Is is a text that can be added on top of each schema? I would expect the schema to replace Documentation/devicetree/bindings/* over time. I think the thing that needs to be worked out here is how to add free form multi-line text. I for one don't care to review the documentation and the schema for every binding. I would like to ensure we can start with the basics and have some generic rules. Some examples: - flag compatible strings in dts files that are not documented This is almost done with the last patch of the series. Nodes are already checked, properties are missing. - check required properties Done as well. - check properties against cell sizes Yeah, that one is still to be done. - a node with reg property must have a unit address Easy to add. - flag properties with _ Easy to add as well. - check properties are the correct data type Already done for a couple of data type. Good to know. It wasn't really clear what common items are checked. An example schema for clocks or gpio (or any other common) binding would be nice. Do you have a run with all the warnings found? Thanks for your comments. Being the very first one, you might get a free beer... meaning there might be such thing as a free beer :-) In that case, it all looks perfect. :) Rob -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC 00/15] Device Tree schemas and validation
On 09/24/2013 10:52 AM, Benoit Cousson wrote:
Hi All,
Following the discussion that happened during LCE-2013 and the email
thread started by Tomasz few months ago [1], here is a first attempt
to introduce:
- a schema language to define the bindings accurately
- DTS validation during device tree compilation in DTC itself
Sorry, this is probably going to sound a bit negative. Hopefully you
find it constructive though.
The syntax for a schema is the same as the one for dts. This choice has
been made to simplify its development, to maximize the code reuse and
finally because the format is human-readable.
I'm not convinced that's a good decision.
DT is a language for representing data.
The validation checks described by schemas are rules, or code, and not
static data.
So, while I'm sure it's possible to shoe-horn at least some reasonable
subset of DT validation into DT syntax itself, I feel it's unlikely to
yield something that's scalable enough.
For example, it's easy to specify that a property must be 2 cells long.
What if it could be any multiple of two? That's a lot of numbers to
explicitly enumerate as data. Sure, you can then invent syntax to
represent that specific rule (parameterized by 2), but what about the
next similar-but-different rule? The only approach I can think of to
that is to allow the schema to contain arbitrary expressions, which
would likely need to morph into arbitary statements not just
expressions. Once you're there, I think the schema would be better
represented as a programming language rather than as a data structure
that could have code hooked into it.
How to:
* Associate a schema to one or several nodes
As said earlier a schema can be used to validate one or several nodes
from a dts. To do this the compatible properties from the nodes which
should be validated must be present in the schema.
timer1: timer@4a318000 {
compatible = ti,omap3430-timer;
...
To write a schema which will validate OMAP Timers like the one above,
one may write the following schema:
/dts-v1/;
/ {
compatible = ti,omap[0-9]+-timer;
What about DT nodes that don't have a compatible value? We certainly
have some of those already like /memory and /chosen. We should be able
to validate their schema too. This probably doesn't invalidate being
able to look things up by compatible value though; it just means we need
some additional mechanisms too.
* Define constraints on properties
To define constraints on a property one has to create a node in a schema
which has as name the name of the property that one want to validate.
To specify constraints on the property ti,hwmods of OMAP Timers one
can write this schema:
/dts-v1/;
/ {
compatible = ti,omap[0-9]+-timer;
ti,hwmods {
...
};
compatible and ti,hwmods are both properties in the DT file. However, in
the schema above, one appears as a property, and one as a node. I don't
like that inconsistency. It'd be better if compatible was a node too.
If one want to use a regular as property name one can write this schema:
/dts-v1/;
/ {
compatible = abc;
def {
name = def[0-9];
Isn't it valid to have a property named name within the node itself?
How do you differentiate between specifying the node name and the name
property?
What if the node name needs more validation than just a regex. For
example, suppose we want to validate the
unit-name-must-match-reg-address rule. We need to write some complex
expression using data extracted from reg to calculate the unit address.
Equally, the node name perhaps has to exist in some global list of
acceptable node names. It would be extremely tricky if not impossible to
do that with a regex.
...
};
};
Above one can see that the name property override the node name.
Override implies that dtc would change the node name during compilation.
I think s/override/validate/ or s/override/overrides the validation
rules for/?
* Require the presence of a property inside a node or inside one of its
parents
...
/dts-v1/;
/ {
compatible = ti,twl[0-9]+-rtc;
interrupt-controller {
is-required;
can-be-inherited;
interrupt-controller isn't a good example here, since it isn't a
property that would typically be inherited. Why not use interrupt-parent
instead?
One can check if 'node' has the following subnode 'subnode1', 'subnode2',
and 'abc' with the schema below:
/dts-v1/;
/ {
compatible = comp;
children = abc, subnode[0-9];
};
How is the schema for each sub-node specified?
What if some nodes are optional and some required? The conditions where
a sub-node is required might be complex, and I think we'd always want to
be able to represent them in whatever schema language we chose.
The most obvious way would be to
