I'll try again with an example: - An SBOM for Windows 10 is a Collection that could have millions of elements, yes? The serialized file containing the values of those elements could be megabytes. - An SBOM for My App is a Collection with a few elements. My App runs on / depends on Windows.
*When I serialize the SBOM for My App, how big is the file? Megabytes or kilobytes?* That is my definition of "depends on". If Microsoft serializes and signs the file containing the Windows SBOM and its millions of elements, the chain of integrity is broken if the MyApp SBOM file has a copy of Windows element values instead of references. The difference between a logical model and a data model is that the logical model doesn't care how relationships are implemented, they just exist. A data model *defines* how relationships are implemented - as either nested values or a map/array of independent values. My definition of "depends on" is: *the value (and hash) of every element is independent of the value (and hash) of every other element. * Elements cannot be nested, an SBOM (Collection) element must have an array of IRIs, not a map/array of values. That requirement exists at the data level because a pure logical model doesn't care. But to the extent that data shapes are hybridized into the SPDX model, it must also require that independence. Note that the full SpdxFile is not an array, it is an object with namespace, namespaceMap, defaults, elementValues, and references to other SpdxFiles. But the elementValues property is an array because the values aren't nested. Regards, David On Tue, Jul 19, 2022 at 12:47 AM William Bartholomew (CELA) via lists.spdx.org <[email protected]> wrote: > CIL > > ------------------------------ > *From:* [email protected] <[email protected]> on behalf of > David Kemp via lists.spdx.org <[email protected]> > *Sent:* Monday, July 18, 2022 6:18 PM > *To:* SPDX-list <[email protected]> > *Subject:* [EXTERNAL] Re: [spdx-tech] V3 serialization > > One principle is that the goal of serialization is to put Elements into > physical format, NOT to create new elements that didn't exist prior to > serialization. If you have 6 elements going into serialization, you should > have 6 elements coming out, not 7. > > *[William] *Agreed, does my example violate that? It would be difficult > for a serialization to "generate" elements because of the id and other > required properties so I had not considered this a possibility. > > The second principle is that logical elements should be independent: the > value of one element does not depend on the value of any other element. > > *[William] *I think it depends on your definition of "depends on" (pun > intended). Elements may have properties that are references to other > elements and serializers may choose to use that information for more > compact serialization but since this would get unwound on deserialization > that's immaterial. > > I believe that those two principles are worth adopting as design > requirements. > > It is ugly to put something into serialization and get something else back > out, > > *[William] *Agreed, though a lot of serializers/deserializers end up > making minor changes as a result of normalization and other processes. Not > ideal but that's an implementation detail within each > serializer/deserializer. > > and it's really ugly to stuff one element's value inside another > > *[William] *I don't agree with this, at least for "collection" elements. > Also, the serialization model for collection elements could support either > element references or the element itself so if you think it's ugly then you > would have the option of not doing nesting. > > not least because you can wind up with infinite recursion with documents > inside documents inside documents inside documents > > *[William] *This is avoidable and using references instead of nesting > doesn't prevent this problem. In fact, if you only use nesting then it's > impossible to have infinite recursion, it's only when you use references > that becomes possible. > > Even two levels of element nesting makes things quite difficult to > disentangle. > > *[William] *I don't agree, for collections the nesting makes it obvious > which collection an element is part of without having to follow the id > references. Since the serialization model could support either approach I > don't see this being a blocker. > > The fundamental principle is that a file containing data is not an > element. A Transfer Unit is defined by a data schema, just like the > content of any XML file or JSON file or ASN.1 file. If the logical model > has a Document element that describes an X.509 certificate, that element > has interesting facts about the certificate but does not define its > content. It is essential to remember the difference between the bytes in a > file and the properties of a File or Document element - the difference > between a thing and metadata about that thing. > > *[William] *We've had this discussion a number of times, the Collection > element (and its subclasses) aren't metadata about collections, document, > SBOM, etc. they are the collection, document, SBOM, etc. There is no > "physical" thing outside of the SPDX document that is the collection, > document, SBOM, etc., they only exist in the SPDX graph. You could take > that SBOM, serialize it to disk, and then have a File element that talks > about the physical serialization of the SBOM, but that's different to the > SBOM SPDX element. > > * defaults: > I created a separate defaults property to hold the five defaultable > properties in order to distinguish them from non-defaultable properties. > Gary and I like the idea, but I'm not wedded to it. The transfer unit > schema could have "defaultCreatedBy", "defaultCreated", etc properties at > the top level, to highlight that they are defaults, unlike name, > description, comments, etc. Whatever the mechanism, there must be a way to > ensure that "name" doesn't take an inappropriate default value if it isn't > populated, while the default for "profiles" is appropriate. > > *[William] *I'm struggling with multiple properties that have the same > definition having different names and different locations on the objects, > it feels like a lot to explain. We could flag certain properties as > inheritable in the schema, and this only applies to collection elements so > I think the scope is quite narrow. > > * array vs map > I used map as a conversation starter, because it fits the "unique" > semantics of element ids, and because mapping types are ubiquitous now, > XML schema had it in 2005 > https://www.w3.org/2005/07/xml-schema-patterns.html#Maps > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.w3.org%2F2005%2F07%2Fxml-schema-patterns.html%23Maps&data=05%7C01%7Cwillbar%40microsoft.com%7C5fdb31fe5d124147f1d808da69249c20%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637937903545815080%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=EQ3nPm2ddYPE8xvFM0KqQAlHYZwbK%2BoIxh9K8hTz9r4%3D&reserved=0>, > and it's a built-in part of JSON. > > *[William] *While it is built-in to XML and JSON my experience is that > it's not been supported well by schema languages and > serializers/deserializers. I know I've had situations where I had to > duplicate the id property in the class to ensure that other things work > correctly (and to maintain the independence of the class). Also, in most > object oriented languages there is not a way to get the key from the object > so you end up having to track the key independently of the object which is > a pain. > > JSON-LD even treats ID differently from other properties by giving it a > reserved @ID type, and SQL databases have primary keys with the special > characteristic that they uniquely identify the record rather than being > just another column. Autogenerated ids are often hidden because they are > ubiquitous. > > *[William] *In both JSON-LD and SQL the properties are still normal > properties, in JSON-LD it's still a property on the object it just has a > special name, in SQL it's still a column in the table it just has special > metadata attached to it. Even autogenerated ids are typically normal > columns they're just system generated and you can't change their definition. > > And finally, you introduced Map to the logical model for Extensions. If > it's OK for extensions, it's OK for Elements :-). > > *[William] *Not the same 🙂. The map for extensions is a map of > "extension type" to value, not of "id" to value. It is a consequence of us > deciding that each type can only be assigned once that it can also be used > as an id, but it is primarily a type, not an id. If we changed that design > decision it would no longer function as an id. > > Seriously though, I'm not wedded to Map. Treating Id as any other > property but having some prose saying that it can be used as a primary key > / unique identifier is OK, it's just kind of loose given that references > from foreign to primary keys is a universal concept. > > *[William] *In SQL Server (others are similar) a foreign key takes the > form FOREIGN KEY (ChildCol1, ChildCol2) REFERENCES parentTable (ParentCol1, > ParentCol2, ...), they're still just columns, nothing magical about them, > not even their names. > > * type property > Since JSON does not have types it's good practice to ensure that "type: > identity" cannot collide with a property named "identity". At the core > profile all type and property names are defined and don't collide, but if > "type" goes away we'll need to ensure that properties defined in any > profile cannot collide with types defined in any profile. Again JSON-LD > treats @type as a reserved property: > https://w3c.github.io/json-ld-syntax/#typed-values > <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fjson-ld-syntax%2F%23typed-values&data=05%7C01%7Cwillbar%40microsoft.com%7C5fdb31fe5d124147f1d808da69249c20%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637937903545815080%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=CKnF7%2BIEKZf8stmRlX21mxCvsHPWJi1OOT7zbGdsNQ4%3D&reserved=0> > . > > *[William] *Agreed, and type isn't in the logical model, a JSON-LD > serializer would use @type, an XML one would use XML namespaces and element > names, a ProtoBuf one would use message types. Since my examples were > "plain" JSON which does not have a built-in way of declaring types I used a > "plain" property to capture the type, I agree that the name of this > property should avoid potential conflict (e.g. by prefixing with an _). > > * document root > A transfer unit file is not an Element and not a logical type or a class. > The bytes in SPDX documents are not defined by the logical model, they just > have to be able to be de-serialized into element instances. > > *[William] *Same disagreement as above. > > Data schemas (for JSON, XML, ASN.1, ...) explicitly do not define classes, > they define only data types. > > *[William] *I'm not sure what definition of "class" you're using here, > but the boxes on the diagram could be represent in an OO language as > classes or interfaces, for our purposes I don't think the distinction > between class and data type is meaningful. > > Regards, > Dave > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#4690): https://lists.spdx.org/g/Spdx-tech/message/4690 Mute This Topic: https://lists.spdx.org/mt/92468742/21656 Group Owner: [email protected] Unsubscribe: https://lists.spdx.org/g/Spdx-tech/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
