Re: [core.spec] Stricter map validations?

Thanks Juan! This works in Clojure, but it doesn't work for me in Clojurescript due to the use of "resolve" " Assert failed: Argument to resolve must be a quoted symbol" This is where I got stuck too. It looks like spec does have access to the predicate at runtime (after all, it can figure

Re: [core.spec] Stricter map validations?

I added some stuff on https://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9 that makes your example work I think. We can

Re: [core.spec] Stricter map validations?

I don't know if this is a fruitful avenue to pursue, but the Eastwood lint tool [1] uses tools.reader to read Clojure code, and the tools.analyzer library to analyze it, while expanding all macros. If someone can figure out a way to scan through the Abstract Syntax Tree output of tools.analyzer

Re: [core.spec] Stricter map validations?

I ran into a case today where I mistyped the keyword in a `keys` spec and was surprised that validation was not catching invalid data. For my purposes, it would be sufficient to have a automated test that looks at my specs and identifies typos in the spirit of

Re: [core.spec] Stricter map validations?

Hello everyone, On Thursday, 16 November 2017 23:29:56 UTC+1, John Newman wrote: > > [...] when we constrain maps in that closed way, aren't we creating some > new subtype of a map, with fundamentally different semantics? If you are > going to fully close a map, you might as well use a deftype

Re: [core.spec] Stricter map validations?

Great conversation! I'm still catching up on Spec, but it seems like there's a correlation here to type systems. Type systems can introduce a certain kind of technical debt - type debt? It seems that leaving maps open here is also attempting to avoid that sort of type debt - parochial maps

RE: [core.spec] Stricter map validations?

m <clojure@googlegroups.com> on behalf of Didier <didi...@gmail.com> Sent: Tuesday, November 14, 2017 11:21:04 AM To: Clojure Subject: Re: [core.spec] Stricter map validations? Eric does raise an interesting question tho I think so too. I'm still finding it hard to come up with a single ex

Re: [core.spec] Stricter map validations?

Wow, this has been a really great discussion. I have only played with spec a little, not used it in production. Reading this and participating has really helped me clarify a lot of things. Thanks! In particular, I've been thinking a lot about the three things you mentioned, Didier: safety,

Re: [core.spec] Stricter map validations?

Saw this thread on the REPL, interesting discussion. On Tuesday, October 3, 2017 at 10:57:34 AM UTC-7, Alex Miller wrote: > > > It's not about easier, it's about possible. Open grows, closed breaks. > > I agree with the broad brush, but it's important to listen to this usability feedback

Re: [core.spec] Stricter map validations?

| I think you're assuming you're validating API endpoints where client and server are tightly coupled. I can imagine a microservices system where some services put maps on a queue and others consume them. The consumers should allow keys they don't understand. They and also need to validate

Re: [core.spec] Stricter map validations?

an empty string and it broke that client) >>> >> >>> >>> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN >>> An Architect's View -- http://corfield.org/ >>> >>> "If you're not annoying somebody, you're not really alive." &

Re: [core.spec] Stricter map validations?

rom:* clo...@googlegroups.com <clo...@googlegroups.com> on behalf of >> Seth Verrinder <set...@gmail.com> >> *Sent:* Tuesday, November 14, 2017 8:45:30 AM >> *To:* Clojure >> *Subject:* Re: [core.spec] Stricter map validations? >> >> I took part of the goal to

Re: [core.spec] Stricter map validations?

t) >>> >>> >>> >>> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN >>> An Architect's View -- http://corfield.org/ >>> >>> "If you're not annoying somebody, you're not really alive." >>> -- Margaret Atwood >>>

Re: [core.spec] Stricter map validations?

-------- >> *From:* clo...@googlegroups.com <clo...@googlegroups.com> on behalf of >> Seth Verrinder <set...@gmail.com> >> *Sent:* Tuesday, November 14, 2017 8:45:30 AM >> *To:* Clojure >> *Subject:* Re: [core.spec] Stricter map validations? >&

Re: [core.spec] Stricter map validations?

et Atwood > > > -- > *From:* clo...@googlegroups.com <clo...@googlegroups.com > > on behalf of Seth Verrinder <set...@gmail.com > > > *Sent:* Tuesday, November 14, 2017 8:45:30 AM > *To:* Clojure > *Subject:* Re: [core.spec] Stricter map vali

Re: [core.spec] Stricter map validations?

m:* clojure@googlegroups.com <clojure@googlegroups.com> on behalf of > Seth Verrinder <set...@gmail.com> > *Sent:* Tuesday, November 14, 2017 8:45:30 AM > *To:* Clojure > *Subject:* Re: [core.spec] Stricter map validations? > > I took part of the goal to be that specs th

RE: [core.spec] Stricter map validations?

alive." -- Margaret Atwood From: clojure@googlegroups.com <clojure@googlegroups.com> on behalf of Seth Verrinder <set...@gmail.com> Sent: Tuesday, November 14, 2017 8:45:30 AM To: Clojure Subject: Re: [core.spec] Stricter map validations? I

Re: [core.spec] Stricter map validations?

Oh, I see! That makes sense. Thanks! On Tuesday, November 14, 2017 at 10:45:30 AM UTC-6, Seth Verrinder wrote: > > I took part of the goal to be that specs themselves would remain > compatible, so an old set of specs wouldn't start failing on data that > conforms to a new but compatible set of

Re: [core.spec] Stricter map validations?

I took part of the goal to be that specs themselves would remain compatible, so an old set of specs wouldn't start failing on data that conforms to a new but compatible set of specs. That sort of compatibility isn't possible when you go from disallowing something to allowing it. On Tuesday,

Re: [core.spec] Stricter map validations?

Hey everybody! I'm chiming in after seeing this linked to in The Repl (https://therepl.net/). On Alex's suggestion, I rewatched Spec-ulation last night. The parts about negation and evolution are towards the end. I was struck (once again) by how clearly he picked apart changes. Relaxing a

Re: [core.spec] Stricter map validations?

I could reference symbols that are not declared, but I prefer >>>>>> the >>>>>> compiler errors before going live. >>>>>> >>>>>> On Saturday, October 7, 2017 at 12:01:34 AM UTC+2, Sean Corfield >>>>>> w

Re: [core.spec] Stricter map validations?

oach, I wouldn't be too upset >>>>>> if >>>>>> I lost that ability and it started throwing an error. I mean it throws >>>>>> an >>>>>> error if I go to generate it anyway. >>>>>> >>>>>> >>

Re: [core.spec] Stricter map validations?

actually does >>>>>> happily define s/keys specs without correspondingly speccing the leaves >>>>>> as >>>>>> an "incrementally lock down/validate" approach, I wouldn't be too upset >>>>>> if >>>>>> I lost that ability and it started throwing an error. I

Re: [core.spec] Stricter map validations?

d up!** >>>>> >>>>> >>>>> >>>>> I don’t want to have to write (s/def ::some-key any?) all over the >>>>> place as I’m developing specs, just to satisfy an overly eager checker >>>>> (in >>

Re: [core.spec] Stricter map validations?

ey spec might not even show >>>> up until much further down the line. >>>> >>>> >>>> >>>> To me, this default behavior of silently not checking the _*value*_ >>>> associated with a _*key*_ is in keeping with the design pr

Re: [core.spec] Stricter map validations?

> associated with a _*key*_ is in keeping with the design principles of >>> spec which focus on maps being based on a *key set*, while offering >>> functions to allow you to optionally check values. >>> >>> >>> >>> Sean Corfield -- (970) FOR-SEAN --

Re: [core.spec] Stricter map validations?

) FOR-SEAN -- (904) 302-SEAN >> An Architect's View -- http://corfield.org/ >> >> "If you're not annoying somebody, you're not really alive." >> -- Margaret Atwood >> >> >> -- >> *From:* clo...@googlegroups.com <cl

Re: [core.spec] Stricter map validations?

- Margaret Atwood > > > -- > *From:* clo...@googlegroups.com <clo...@googlegroups.com > > on behalf of Beau Fabry <imf...@gmail.com > > *Sent:* Friday, October 6, 2017 9:10:36 AM > *To:* Clojure > *Subject:* Re: [core.spec]

Re: [core.spec] Stricter map validations?

Atwood > > > -- > *From:* clo...@googlegroups.com <clo...@googlegroups.com > > on behalf of Beau Fabry <imf...@gmail.com > > *Sent:* Friday, October 6, 2017 9:10:36 AM > *To:* Clojure > *Subject:* Re: [core.spec] Stricter map validat

RE: [core.spec] Stricter map validations?

egroups.com <clojure@googlegroups.com> on behalf of Beau Fabry <imf...@gmail.com> Sent: Friday, October 6, 2017 9:10:36 AM To: Clojure Subject: Re: [core.spec] Stricter map validations? A use case that comes to mind is a system/library that specifies the structure of some inputs/outputs, but

Re: [core.spec] Stricter map validations?

A use case that comes to mind is a system/library that specifies the structure of some inputs/outputs, but lets users/consumers (optionally) specify further validation of the leaves. I suppose that would be possible with (s/def ::foo any?) but you'd have to be a bit more careful about load

Re: [core.spec] Stricter map validations?

Thanks, Beau. I am still interested why this default behavior has been chosen. It doesn't seem like a reasonable trade-off at this point. It enables me to say: "The map must have this key", without specifying how the data mapped to it will look like. If I ever wanted to do that, I could as

Re: [core.spec] Stricter map validations?

Leon, perhaps you could add this code to your test suite? boot.user=> (let [kws (atom #{})] #_=> (clojure.walk/postwalk (fn [x] (when (qualified-keyword? x) (swap! kws conj x)) x) (map s/form (vals (s/registry (clojure.set/difference @kws (set (keys (s/registry #_=> )

Re: [core.spec] Stricter map validations?

Open maps/specs are fine. s/keys supporting unregistered specs are not. At least to me. I just fixed two more bugs in production that were would not have happened. What are the supposed benefits of this feature? I can only infer "being able to require keys without their spec being known"

Re: [core.spec] Stricter map validations?

Seems like that's the reasonable place to check it, otherwise you're forced into an ordering for your specs and cannot write recursive strict map specs. On Wednesday, October 4, 2017 at 8:59:59 AM UTC-7, Yuri Govorushchenko wrote: > > Thanks. This approach is also different from the macro

Re: [core.spec] Stricter map validations?

Thanks. This approach is also different from the macro because it will check specs existence at the validation time, not at the s/def call. On Wednesday, October 4, 2017 at 4:18:16 PM UTC+3, Moritz Ulrich wrote: > > Yuri Govorushchenko writes: > > > Thank you the pointers!

Re: [core.spec] Stricter map validations?

Yuri Govorushchenko writes: > Thank you the pointers! So far I ended up with writing a small `map` macro > which is similar to `s/keys` but checks that keys are already in the > registry: > https://gist.github.com/metametadata/5f600e20e0e9b0ce6bce146c6db429e2 Note that

Re: [core.spec] Stricter map validations?

While it doesn't use spec, you can do this using the validate-map-keys helper function in the Tupelo library . It verify that a map does not contain any keys other than those you specify. The unit test shows it in action: (ns tst.demo.core (:use demo.core

Re: [core.spec] Stricter map validations?

I'm loving Spec as an FYI, and I'll use it even if it stays as is, even in its alpha state it is a great tool. I also highly value the Clojure core value to avoid breakage, as the cost to enterprise of every migration is very high. I'm just wanting to give more data points to the Core team

Re: [core.spec] Stricter map validations?

Open Specs seem like a good idea. In real life, we need to close our Data at system borders. With Java, there is Jackson, which fails by default on extra keys. The Schema, the models are closed by default. JSON Schemas can be closed. Saying "use normal clojure" on top of Spec mean

Re: [core.spec] Stricter map validations?

On Tuesday, October 3, 2017 at 12:25:40 PM UTC-5, Didier wrote: > > | Spec-tools (https://github.com/metosin/spec-tools) has some tools for > this: the spec visitor (walking over all core specs, e.g. to collect all > registered specs) and map-conformers: fail-on-extra-keys and >

Re: [core.spec] Stricter map validations?

On Tuesday, October 3, 2017 at 11:48:04 AM UTC-5, Yuri Govorushchenko wrote: > > 3) I agree that my argumentation was vague but the sentiment still holds > true: I'd like to easily validate keyword args with the same level of > confidence as I can validate positional args. > Nothing is

Re: [core.spec] Stricter map validations?

I don't disagree that there might be a security use-case, and regardless of how realistic it is businesses need people to tick boxes (that doesn't seem like something core should worry about?), but what is the actual security risk of an unspec'd map value, or things being referenced and unused?

Re: [core.spec] Stricter map validations?

| Spec-tools (https://github.com/metosin/spec-tools) has some tools for this: the spec visitor (walking over all core specs, e.g. to collect all registered specs) and map-conformers: fail-on-extra-keys and strip-extra-keys. I understand the core team wanting to take a minimal approach to spec,

Re: [core.spec] Stricter map validations?

| I would have had to re-spec all the intermediate functions I'm not able to follow how that is? You would have only needed this if you are somewhere validating or have setup test.check to run. In both cases, if you wanted to validate, why don't you also want to validate this new field? Or

Re: [core.spec] Stricter map validations?

3) I agree that my argumentation was vague but the sentiment still holds true: I'd like to easily validate keyword args with the same level of confidence as I can validate positional args. 1,2) > Because all map keys are independent and don't rely on each other for ordering. > You can add

Re: [core.spec] Stricter map validations?

Thank you, it looks handy! On Tuesday, October 3, 2017 at 6:59:27 PM UTC+3, Tommi Reiman wrote: > > Hi. > > Spec-tools (https://github.com/metosin/spec-tools) has some tools for > this: the spec visitor (walking over all core specs, e.g. to collect all > registered specs) and map-conformers:

Re: [core.spec] Stricter map validations?

Hi. Spec-tools (https://github.com/metosin/spec-tools) has some tools for this: the spec visitor (walking over all core specs, e.g. to collect all registered specs) and map-conformers: fail-on-extra-keys and strip-extra-keys. Here's an example to strip away extra keys: (require

Re: [core.spec] Stricter map validations?

On Tuesday, October 3, 2017 at 8:10:30 AM UTC-5, Yuri Govorushchenko wrote: > > 1) About `s/keys` silently ignoring missing value specs. My question was: > "Is there any way to ensure that the keys I used in `s/keys` have the > associated specs defined?." > > Specs can be defined or added

Re: [core.spec] Stricter map validations?

1) About `s/keys` silently ignoring missing value specs. My question was: "Is there any way to ensure that the keys I used in `s/keys` have the associated specs defined?." Specs can be defined or added later, so there is no valid way to do this. OK, so requiring that values are spec-ed can't

Re: [core.spec] Stricter map validations?

My critique is not towards closed keysets but about being able to (s/keys :req [::some-kw]) without having defined the spec ::some-kw, but e.g. ::soem-kw instead. This can trip you up badly, and not at compile time. I'd be surprised if this never happened to you. Regarding open maps and

Re: [core.spec] Stricter map validations?

My critique is not towards closed keysets but about being able to (s/keys :req [::some-kw]) without having defined the spec ::some-kw, but e.g. ::soem-kw instead. This can trip you up badly, and not at compile time. I'd be surprised if this never happened to you. Regarding open maps and

Re: [core.spec] Stricter map validations?

On Tue, Oct 3, 2017 at 2:55 AM, Peter Hull wrote: > On puzzler's database example, I would have thought that restricting the > keys that go into the DB should not be the job of spec (since functions may > not be instrumented anyway), but the job of the 'core logic'. Maybe

Re: [core.spec] Stricter map validations?

I have found, as an application evolves, some extra data is required which comes from elsewhere. For example, I have an application where data is read from a network socket, transformed and stored. For auditing, I later decided it was useful to capture the remote address from the socket right

Re: [core.spec] Stricter map validations?

Yesterday, I was checking a map of info submitted via web before putting its contents into a database. To prevent people from spamming the database, it's necessary to make sure there aren't additional keys thrown into the map. It would be nice to have a *convenient *way to express this in spec,

Re: [core.spec] Stricter map validations?

| we have experienced on virtually every consulting project we'd done including spec at Cognitect I'm sure this is sometimes true, but I can't think of how that would happen. Could you detail it a little? For me, whenever I needed to add more keys, it was simple to evolve the spec with the

Re: [core.spec] Stricter map validations?

On Monday, October 2, 2017 at 12:30:57 PM UTC-5, Leon Grapenthin wrote: > > I second this from my experience, using spec quite extensively since its > release. > > We already had some invalid data passing silently because of this. It can > easily happen if you have a typo in the spec. > > Also

Re: [core.spec] Stricter map validations?

On Monday, October 2, 2017 at 10:37:31 AM UTC-5, Yuri Govorushchenko wrote: > > Hi! > > I have some noobie questions for which I couldn't google the compelling > answers. > > 1) Is there any way to ensure that the keys I used in `s/keys` have the > associated specs defined? > Specs can be

Re: [core.spec] Stricter map validations?

Thank you the pointers! So far I ended up with writing a small `map` macro which is similar to `s/keys` but checks that keys are already in the registry: https://gist.github.com/metametadata/5f600e20e0e9b0ce6bce146c6db429e2 On Monday, October 2, 2017 at 10:03:57 PM UTC+3, Beau Fabry wrote: > >

Re: [core.spec] Stricter map validations?

I vote for strict validation also. It's a security threat on most applications not to strictly validate inputs and outputs. I would rather Clojure defaults to be secure, and people needing to implement their own less strict validation consciously. I'm afraid a lot of people will rely on spec

Re: [core.spec] Stricter map validations?

fwiw, I vote for leaving it. It's extra flexibility and is afaict a fairly easy error to catch. Here's a function that I think should give you every qualified keyword that is used in a spec but that does not have a spec defined for it: boot.user=> (let [kws (atom #{})] #_=>

Re: [core.spec] Stricter map validations?

I second this from my experience, using spec quite extensively since its release. We already had some invalid data passing silently because of this. It can easily happen if you have a typo in the spec. Also we never experienced benefits from being able to not spec keys required in s/keys. It

Re: [core.spec] Stricter map validations?

> 1) Is there any way to ensure that the keys I used in `s/keys` have the associated specs defined? I think you should be able to do this by writing a function that uses s/registry s/form and s/get-spec On Monday, October 2, 2017 at 8:37:31 AM UTC-7, Yuri Govorushchenko wrote: > > Hi! > > I