Hello! > On Apr 21, 2017, at 3:16 AM, Tony Atkins <[email protected]> wrote: > > Hi, Steve: > > Happy to take over GPII-111, as that seems very much like what I've planned > to focus on for the next month or so. >
Awesome, thanks much! > I guess my point is: Breaking these up so they aren't just one file per OS > sounds great, but I would recommend against requiring a 5-10 line file for > every single setting. And while these are in files now, I hope we can > remember with any API we encounter that the JSON data is what's important, > and whatever physical storage it sits in is mostly accidental complexity. > > I disagree somewhat with the considered choice of using a GitHub repo as > being "mostly accidental complexity". Although as you say we have the > freedom to change back-ends in the future, I am proposing a filesystem and a > github-based workflow as an initial choice because it has "intentional > simplicity" versus building our own back end, mechanisms for managing the > data, workflow, workflow tools, et cetera. > You're right, having a clear way for organizing the data in some type of tree system and using the github workflow is essential complexity for our organization structure. If I remember right, what spurred me to think this at all was during the APCP meeting when we were discussing that part of the reason they may have to be in separate files was a limitation of a 3rd party validation framework, or whether it was worth our time to script it's API. That is largely what I considered to be the accidental complexity, that we were forced into some layout and storage situation by the validator requiring that referenced schemas absolutely had to be in some sort of posix file. In the back of my head though, I still imagine that the essential complexity you're working on is for organizing all this data and developing a standard way for people to contribute to it, and should successfully translate to another storage methodology that could represent blocks of texts in a tree structure and have some sort of versioning. Be that couch documents, or whatever. At the current state in time, the most robust and nicest set of tooling for this just happens to be files+git+github. > Setting that aside, the key distinction here is not "one schema per setting". > It's "one definition per setting". Individual schemas will represent what > is valid for a particular use case, and will reuse one or more definitions > <https://spacetelescope.github.io/understanding-json-schema/structuring.html#reuse>. > As long as we distinctly define each setting in a way that can be referred > to externally, the grouping is up to us. The key concerns to me are: > Supporting reuse. > Keeping the files small to support contribution and review. > So, with that in mind, here is a proposal. > "common" settings definitions that occur across platforms would be in a > central file. > "platform-unique" settings definitions would be in a file per OS. > "solution-unique" settings definitions could be in one file per solution (or > family of related solutions that use the same conventions). This seems super-reasonable, and something I might have to think about some more too. I had never thought too much about #2, ( I can guess what they might be, but am having a hard time pulling out examples right now). With the actual ability to inhereit settings from grades, it is nice to think about things that are essentially the same in the linux.json and win32.json file being merged into one sort of entry. Like the chrome and easy123 type ones. > This level of organization means that most contributors and reviewers would > be working with a "solution-unique" definitions file that is relatively > small. As we do with code reviews, part of a good PR would involve > identifying more widely useful common patterns that need to be pulled upward > to a new "platform-unique" or "common" settings. The larger the definitions > block, the less frequently it would be modified. > > The above just refers to definitions to be used in individual schemas. I am > still reviewing prior work before I start sketching out actual schemas, but > can start showing you what I mean using your existing work. Let's start with > the first entry in your data > <https://github.com/sgithens/gpii-devpmt/blob/master/src/js/commonTerms.json5#L3>. > Right now it looks like: > > { > "http://registry.gpii.net/common/fontSize > <http://registry.gpii.net/common/fontSize>": { > name: "Font Size", > description: "", > schema: { > type: "integer" > } > }, > > Although you would likely mature this approach once the JSON UI Schema spec > comes into being, for now I would suggest using the metadata supported by the > standard > <https://spacetelescope.github.io/understanding-json-schema/reference/generic.html#metadata>, > namely the "title", and "description" fields. This allows you to collapse > down your proto-schema to look more like: > > "http://registry.gpii.net/common/fontSize > <http://registry.gpii.net/common/fontSize>": { > title: "Font Size", > description: "The font size to use in a particular context.", > type: "integer" > } > Interesting, putting those into the actual JSON schema. I do wonder though if we won't need to keep the schema as an embedded resource, as we might have items that legit don't belong in it? I had pulled 'name' and 'description' from Javi's document: https://wiki.gpii.net/w/Personal_Control_Panel_API#Settings.27_Metadata starting out just because it was the info I needed to render the DevPTT. Looking at that full list (and maybe other random things we need), it will be interesting to see whether it could *all* just go in the schema document. It probably depends on much extra information we might want for each setting. At the very least it makes sense to put a title and description in the actual JSON schema, but there still may be other non JSON Schema information that warrants the level up still... I just wonder if things could get wordy to a point that they shouldn't be in there, paragraph documentation with like links to API's from the vendors website and stuff. (still on a persetting basis) > This now represents a single field in a larger object, with the key > "http://registry.gpii.net/common/fontSize > <http://registry.gpii.net/common/fontSize>". To make your schema reusable, > you would move the definition into a definitions block. It doesn't > necessarily have to live in a separate file, but just to illustrate the > beginnings of the structure I mentioned above, let's assume I have a > "common.json" definitions file that looks like: > > { > "$schema": "http://json-schema.org/schema# > <http://json-schema.org/schema#>" > id: "common.json", > definitions: { > "fontSize": { > id: "fontSize", > title: "Font Size", > description: "The font size to use in a particular > context.", > type: "integer" > } > } > } > > The actual schema used for validation could now look more like: > > { > "$schema": "http://json-schema.org/schema# > <http://json-schema.org/schema#>" > id: "devpmt.json", > properties: { > "http://registry.gpii.net/common/fontSize > <http://registry.gpii.net/common/fontSize>": { $ref: "common.json#fontSize" } > } > } That does look super cool, and I like how it could either be in or not in a file, depending on how complex it might, and again, looking at these libraries, it doesn't look like it should be too hard for us to ever process them from nodes if need be. Part of the 'separate file' worry I have was to make sure we don't have a situation where, like for JAWS you'd have to make like: jaws-solution.json jaws-setting-Options.PrimarySynthesizer.json jaws-setting-jaws-ENU-Global.Rate.json jaws-setting-ENU-JAWSCursor.Rate.json jaws-setting-ENU-Keyboard.Rate.json jaws-setting-ENU-MenuAndDialog.Rate.json ...this goes on for like a ream of paper ALSO, here is what I'm hoping we can do. As you've noticed, and lot of the stuff I'm stubbing in is just like { type: "string" }, or { type: "integer" }. I realize that eventually some of these should be more detailed, but sometimes you're really just stubbing in like 100 values that are all of like 2 or 3 types. In that case I was hoping I could go: > { > "http://registry.gpii.net/common/fontSize > <http://registry.gpii.net/common/fontSize>": { > name: "Font Size", > description: "", > schema: { $ref: "stringSchema" } > }, That does put us back to our previous larger structure... I don't know maybe they'd have to be parsed and generated on the fly. It would be nice to have just those 5 or 8 base schemas you could just also use when you're crazy fast stubbing out something like JAWS that has a ton of settings that are for all practical purposes the same. I might have to think more about this use case, but I feel like I want to do somethign close to it. > > If this seems reasonable enough to flesh out a bit further, I would be happy > to take a few minutes and prepare a PR against the devpmt, demonstrating this > with all of your examples. My idea is that that would be something tangible > (and tested) to review in our next meeting, and that I would start work on a > fuller set of definitions in the live registries based on our discussion. > > Anyway, please review and comment. > Yeah, I think this would be a great set of stuff for the hackathon as well. Talking about schemas and validation, I feel like some people have started certain ones, but I would really love to get a set of JSON schemas in for our most important formats that people edit... NP Sets, Solutions, probably other things. I want the DevPTT as well, to be able to validate an existing or new NP Set against all the schema's we're making above, especially as the first step in avoiding settings injection issues down the road. Cheers to the max, Steve > Cheers, > > > Tony > > On Fri, Apr 21, 2017 at 6:06 AM, Steven Githens <[email protected] > <mailto:[email protected]>> wrote: > Hi Tony! > > This all sounds cool... and I might just assign over GPII-111 to you, if you > think it's more or less the actual work you're doing right now. You can take > a look, and if you think it is, feel free to take it. > > Just one comment below. > >> On Apr 12, 2017, at 5:43 AM, Tony Atkins <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi, All: >> >> As we have long discussed, currently the solutions and settings used within >> the GPII are stored in massive JSON files in the "universal" repository. I >> have been tasked with helping move us towards the kind of granularity, >> inheritance, and testability we discussed in Toronto. I have been sketching >> out initial documentation and a loading/validation harness >> <https://github.com/the-t-in-rtf/gpii-live-registries>, and wanted to >> summarize for wider discussion. >> >> First, as discussed in Toronto, the idea is that the "live" registries would >> be a separate repo that contains the data that currently lives in universal, >> more finely broken down. Changes to the data would be submitted as pull >> requests against this repo. The platform-specific repos would use a >> versioned release of the "live" data (more on that in a bit). >> >> Each solution and setting would be a distinct grade, saved to a single >> JSON(5) file. We would use the effective path and filename to create an >> implicit and unique grade name for each options file. This accomplishes two >> things: > > > I think having a default convention for the repo of having each solution in > it's own file is probably good, but I would hope that each setting for that > solution wouldn't have to be in it's own file... maybe it could be if you > want, but hopefully optional. Having barely survived that decade in the > early 2000's when J2EE was cool and having to put every single public java > class has left some anxiety in my stomach thinking about this. > > Mostly, I want to make sure that we future proof ourselves, and have this > validation and solutions registry tooling work well in any situation where > you have some JSON data that makes up a schema. Regardless of whether it's > in a file, a couch document, a node in another JSON document, or being > dynamically and temporarily stored in the local storage of an awesome web > based authoring tool, which is most likely just over the horizon for us. > > Out of a paranoia [1], I did start reading through the spec and looking at > some of the validation libraries, and everything seems like it should be Ok. > And even though I don't have the schema directive in yet (although it's only > 1 five minute vim macro away ;) ), they do actually seem to validate fine. > From what I mentioned on the APCP call, I am actually hoping that we can > start with the metadata I've created for the Generic Preferences and filling > in some of the apps settings metadata (JAWS mostly), since it is like a good > half days worth of typing. I'm happy to spend the 10 minutes with a vim > macro to make them look however we need them, and split them up in to files. > > I guess my point is: Breaking these up so they aren't just one file per OS > sounds great, but I would recommend against requiring a 5-10 line file for > every single setting. And while these are in files now, I hope we can > remember with any API we encounter that the JSON data is what's important, > and whatever physical storage it sits in is mostly accidental complexity. > > Cheers, > Steve > > [1] and left over stress/nightmares of when I was still working on J2EE > projects. >> We will have an easier time detecting namespace collisions with this model. >> We can detect the existence of and perform standard tests against each grade >> in isolation (see below). >> So, what do I mean by "grades" in this context? Basically, anything you can >> do in an options block without writing code can be stored in one of these >> JSON(5) files. Settings and solutions derive from concrete gpii.setting and >> gpii.solution grades. Abstract grades are also possible, such as platform >> and platform version mix-ins. >> >> A new loader would scan through an "options file hierarchy" and associate >> each block of options with its namespace, as though the user had called >> fluid.defaults(namespace, options). Once all grades have their defaults >> defined, we can search for any grades that extend gpii.solution or >> gpii.setting, and do things like: >> Confirm that each component can be safely instantiated. >> Confirm that the component satisfies the contract defined for the base >> grade, for example, that it provides an "isInstalled" invoker. >> For "abstract" grades, we would not attempt to instantiate them, only to >> confirm that each is extended by at least one "concrete" grade that has been >> tested. >> Platform specific tests would take place within the platform-specific repos, >> which would test their version of the "live" data, for example calling each >> solution's "isInstalled" method to confirm that nothing breaks. As with any >> versioned dependency change, we would submit a PR against a platform repo >> and confirm that the new version of the "live" data does not break anything >> before merging and releasing a new version of the platform repo. >> >> So, that's the proposed workflow and test harness, which are independent of >> the data format. Please comment. Once we have even "lazy consensus" >> agreement on that, we will immediately need to move forward with discussions >> about how we represent each solution/setting and the relationships between >> settings. >> >> Cheers, >> >> >> Tony >> >> _______________________________________________ >> Architecture mailing list >> [email protected] <mailto:[email protected]> >> http://lists.gpii.net/mailman/listinfo/architecture >> <http://lists.gpii.net/mailman/listinfo/architecture> > >
_______________________________________________ Architecture mailing list [email protected] http://lists.gpii.net/mailman/listinfo/architecture
