Great conversation! I'm still catching up on Spec, but it seems like there's a correlation here to type systems. Type systems can introduce a certain kind of technical debt - type debt? It seems that leaving maps open here is also attempting to avoid that sort of type debt - parochial maps that don't play nice with others. And when we constrain maps in that closed way, aren't we creating some new subtype of a map, with fundamentally different semantics? If you are going to fully close a map, you might as well use a deftype and make a custom object and not call it a map, right?
On Wednesday, November 15, 2017 at 11:34:04 AM UTC-5, Eric Normand wrote: > > Wow, this has been a really great discussion. I have only played with spec > a little, not used it in production. Reading this and participating has > really helped me clarify a lot of things. Thanks! > > In particular, I've been thinking a lot about the three things you > mentioned, Didier: safety, compatibility, and correctness. I think it's > great that you pulled them apart. I've written a number of public-facing > APIs in my day, and what I always wound up doing was being really strict > with the clients because it actually helps correctness. It helps give > really fast feedback to third parties writing client code because they get > error messages about everything that's wrong with their request. Anything > that's silently accepted could mask a bigger problem and, especially with > third-party clients, you want to get those problems out in the open as soon > as possible. > > But the more I look at the behavior of s/keys, the more I'm starting to > appreciate it. It's not trying to be strict to educate clients. It's for > asking "does this have the minimum I need to handle this map?" And this is > very much how idiomatic Clojure is written. We ignore keys we don't care > about and pass them along. Or we select only the keys we care about and > operate on those. I wonder why we don't want to do this with our APIs. It's > so comfortable and practical internally. > > Now, the educational value of strictness (as developers write clients for > our API) is important. But it doesn't seem to be in the purview of spec. > I've been thinking about a few guidelines, because I haven't seen them > written down: > > 1. Use namespaced keywords with globally unique namespaces that you own. > 2. Don't "invent" keys in a namespace you don't own. > 3. Validate the keys you care about, to ensure that you have the minimum > you need to continue (using a s/keys spec) with appropriate types. > 4. Handle extra keys you can't use in the appropriate way (ignore them and > pass them on OR select only the ones you want). > > With these, you actually do get a lot of the safety and correctness you're > talking about, but it's more of a systemic safety (unexpected input won't > break the system) and safety across changes (growth). > > There are still two problems that this won't solve. The first is the OP's > original problem, which was typos in keys allowing values with bad types to > get through a s/valid? check. That's serious but could easily be checked > with Stuart's code, using what Spec provides. So I think that's a real > solution provided by Spec. You could check in a test, for instance. > > The second problem is that education problem: you want clients to know > when they send typos, not just silently ignore them. I think this is > important. We've already got s/valid?, which asks "can I handle this?". > Maybe you could implement something called strict? that takes a spec and a > value and makes sure there aren't any extra keys. It doesn't seem to belong > in the spec. To me, it should go in a function which you could deploy when > you need it. > > > Rock on! > Eric > > On Tuesday, November 14, 2017 at 8:04:34 PM UTC-6, Didier wrote: >> >> | I think you're assuming you're validating API endpoints where client >> and server are tightly coupled. I can imagine a microservices system where >> some services put maps on a queue and others consume them. The consumers >> should allow keys they don't understand. They and also need to validate >> that the keys they need are there and of the right shape. >> >> That use case sounds more like the exception then the rule to me. I still >> think its valid, and for that I recognise there should be a way to have a >> spec which allows any key and requires specific ones. >> >> Now that I've listened to everyone, I see that there might be many things >> needed by different people, and its not clear if all should be provided by >> spec directly. >> >> Mostly I can see wanting: >> A. To spec a map in which you expect the keys to have a registered spec. >> B. To spec a map in which you don't expect the keys to have a >> registered spec. >> 1. To spec a map in which the keys should be closed to only the req and >> opt keys that are defined. >> 2. To spec a map in which the keys should be open to what is defined, >> but where certain keys are required and others reserved as optional. >> >> Where we want to be able to mix and match A,B with 1,2. >> >> I see a safe default being A,1. Though I'd be okay with A,2. I think B is >> the bigger issue not to have as default, and is less intuitive. It's prone >> to typos and forgetting to add the spec or require the proper namespace, >> etc. >> >> My conclusion: I think s/keys is a funny part of spec, because compared >> to all other provided speccing tool, this one is really just syntax sugar >> for convenience. Yet it makes sense, because it is just so damn convenient. >> But for a lot of people like me, its convenient in a surprising way, which >> leans towards compatibility instead of correctness and security. And most >> baffling, trying to spec a map in a correct, secure and convenient way is >> not possible, at least not with the same convenience. So I'd say if there >> was just also convenient s/closed-keys and s/specified-keys and >> s/specified-closed-keys everyone would be happy, and it would also be more >> explicit what distinction exist between all 4. I'd probably always reach >> first for s/specified-closed-keys, and relax if I need too only, but all >> least everyone would have what they want and could chose for themselves. >> >> -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
