Hey there! I'm going to start with your ending, then go back up to the top.
> Finally, there are enough breadcrumbs in your replies to make me guess > (without being afraid of an unintended reveal ;)) that your identity is John > Viega, correct? It is very nice to see an established industry player > approach Nim and share their perspective. I particularly enjoyed reading a > recent write up of yours: > <https://blog.crashoverride.com/a-personal-history-of-the-appsec-industry> Yes, that's me. Thanks very much for your kind words. And just to be clear, while I was happy to be anonymous, I definitely don't mind, or else I wouldn't have dipped into any of the details of my personal experience. > > multiple people who understand generics and interfaces in other languages, > > but don't find the docs on concepts sufficient. >> >> First, people coming at from languages like Go who are expecting more of an >> 'interface replacement' don't understand that they're not concrete types. >> They want to declare arrays of their concepts and then mix adding in ints, >> objects and strings to some seq[] and don't understand why they can't have a >> seq of their concept. > > Not the first time that I have seen this confusion about concepts. The > (experimental) manual defines them as user-defined type classes, and the > manual warns about type classes being compile time only concepts. Probably > the docs on concepts could link to type classes and might give an additional > warning. The main problem though is that concepts in Nim are very little used > in the wild so you will find in general very few documents around that talk > about them and very few codebases that use them. They recently went through a > refactoring and I would say that they actually might have a revival. I guess > the still lingering question is whether or not they advantages they provide > over generics is enough. Also, being very little used they seems to still > have quite some bugs. I was trying the code coming from this nice comparison > with generics (forum thread) and bizarrely enough it does not compile > (playground) Personally, I'm a fan of concepts, and think they could be valuable to others. When I needed them, it was definitely the place where I got bogged down the longest. I'd stare at all the available docs, try what I thought should work, get an error, stare at the docs some more, then try it another way. I eventually started looking at code, but ElegantBeef provided the illumination I needed. But, while I might have hit some bugs during that process, the only thing I was doing wrong when I was finally set straight was that I was expecting UFCS to bind the same way with generics as without. That one is on me, because I had read that a few times and was originally taking note about it, but I lost the thread. But, then again, the compiler didn't help me pick the thread back up. I'd get the inscrutable "Could not Instantiate T" with _no_ other context to indicate WHY instantiation wasn't possible. Since then, I've had 0 problems making use of concepts. But, two things here: 1. It definitely would be good to have a how-to (and/or a tutorial) for concepts that absolutely hammers home some of these "gotchas". This is one thing that I definitely _do_ intend to write, assuming I find time while it's still fresh enough in my mind. 2. I've seen some mention that compiler error messages have gotten a lot better recently. I'd say that, while the above error was the LEAST enlightening one, I _frequently_ get errors that were not as easy to understand as I would have liked. Two examples of this. One, often when there are type mismatches between A and B, it is not clear from the message which type is the "expected" type. For the average developer to know the expected type when they first arrive, they'd have to know, (for instance), if you "expect" the formals to be the source of truth, or if the actuals are, because that's the part of the parse tree you're on, and what you've been inferring types for. I would have much rather the compiler said something that clearly identified what type Nim things each construct is. For actual proc calls, this wasn't often a problem since I think it does so this (and then shows me every possible match). But either there are plenty of other cases where this isn't as clear, or at least it's not clear enough when there's only one matching procedure. Because I KNOW that I've had some cognitive burden multiple times, trying to figure out which type is which (esp when it's something like an operation on two objects where both of the types are being inferred). This has never slowed me down for long, but it still happens sometimes, and was bad when I started. Second, I _frequently_ would get "invalid indentation" errors at the end of a line. Once I realized that it meant I did something wrong on the line leading up to it, it was not a problem, but before I internalized that, it was often a head scratcher, where I'd keep rewriting the same thing until it ended up working. Perhaps one of those HOWTO guides should be a recipe on what to look out for, based on the compiler error message? That's one of those things where, having done a lot of compiler work, I know how hard it can be to avoid some cases where there are head scratchers, and how hard it can be to actually make further improvements. So docs might actually be helpful here. > from what I can tell the best replacement for go's interface is > <https://github.com/yglukhov/iface> > >> I've seen similar problems with, for example, the OpenSSL integration. >> Having written a book on OpenSSL myself, I know that library sucks big time, >> and their documentation, while extensive, is still garbage. But beyond the >> actual wrapping of a socket, the nim-level documentation of the API isn't >> really there, and it probably should be, because a bunch of people come in >> wanting to use cryptography in some way; I just got done fielding one issue >> in the chat room, where the person was using an INCREDIBLY crappy 3rd party >> library to do basic AES encryption that should be available any time -d:ssl >> is on. Unfortunately, these are the kinds of things that will cause even >> GOOD experienced developers to bounce. > > I would say the reference library for cryptographic algorithms in general and > AES in particular is <https://github.com/cheatfate/nimcrypto> Since it is > part of status and has undergone what seems to be a rather extensive > cryptographic audit (which had in scope that repo, see e.g. this aes issue, > now fixed) hopefully it is not too crappy :) I am pretty opinionated on cryptography as you might imagine. I think that even excellent cryptographers have gotten things wrong in this domain, and it's much better to give people, to use an analogy, pre-fab'd houses instead of a tool-box and other raw materials. That is, crypto libraries are doing the world a disservice when they're not use-case based (generally, based on standards well vetted by cryptographers), and are more tool-box based. I specifically helped out someone a few days ago who was using nimcrypto, and I was really disappointed how far on the 'toolbox' side of the equation it is. For instance, bcmode.nim lets you choose modes that no cryptographer would ever use for any reason anymore (particularly, CFB, OFB). And, while GCM mode is there (which is good, since it's ubiquitous as the default ciphersuite in TLS and elsewhere), it does NOT actually conform to the GCM spec. Specifically, in GCM, encryption produces a single ciphertext that's supposed to be treated as a single unit, especially when passed to the decryption operation. Decryption is NOT supposed to succeed unless the integrity check passes. Yet, in nimcrypto, it's easy to just call the API in such a way that you never handle the tag... it will happily decrypt a message without any integrity check. That's easy as pie to do, and indeed one thing I saw with that person I was helping. By the way, this reminds me, that Nim with 2.0 should do the one thing I liked best about Rust... make the "standard" library random() call a secure random call. In my early career I found it pretty to break tons of practical systems for using a non-cryptographic generator, exploiting such things in online gambling sites, slot machines, games, ... Why let people shoot themselves in the foot? I can argue this one at great length, but suffice to say, you can always have an "insecure" interface somewhere for anyone who thinks they know better, but the average developer shouldn't have to be an expert on security, they should be able to just call "random()" and not worry. If I knew people were receptive to this one, I might actually go out of my way to do a PR. Happy to take it to another thread or offline if people are interested in discussing. Coming back to what I'd call the "developer onboarding" experience, though, I think I'd ask you all to reconsider the trend of slimming down the libraries that ship with Nim. In the early days of Python, the standard libraries were VERY extensive for a programming language in its day, and that's part of why people found it very easy to get things done in it. The language syntax is what comes to mind for many, but having watched it happen, I'd say the libraries played a much bigger part. Some of those libraries weren't awesome, sure, but I believe it was, by a long amount of time, the first language where you could get a web server or SMTP server in a line of code, etc. I do see the value in keeping a tight language core, with a "slim" standard library for environments where that is important. But, slim is often not important. So I'd suggest have essentially 3 rings of libraries, the minimal core, the stable, mature, widely useful core, and then as many decently written modules as possible that provide interesting, orthogonal capabilities. Here, too, have a team explicitly responsible for grooming the libraries, but have it be both official and, VERY importantly, (unlike fusion is today), shipped with the language. For instance! You mentioned iFace as a good option for interfaces. It gets suggested frequently in the chat room. One person recently said that it looked unmaintained, and didn't want to take the risk. I said I'd looked at it enough to say it does the job it's supposed to do, and looks well written. But Nim is changing pretty fast for a programming language, and if I want _my_ code to not have to go through major rewrites in the future, I don't want dependencies down the road that aren't going to change with the language. Having the broader Nim project take ownership of such things, gives people confidence that they can use it. And, I love the idea of a standard library of "use them if you want them" language features like interfaces. I shouldn't have to look at 8 different pattern matching libraries, unless I don't like the one in the standard library. There's a lot of cool software in Nim already, and centralizing it will help adoption. Leaving people on their own to figure out which solution seems like it might work, and which ones are likely to be broken in a year, figuring out if I need to do it myself... that's all a lot of friction that deep standard libraries can fix. Another example of that one; I recently had need to use object storage. There's one fairly complete Nim library that covers a lot of AWS's services (I didn't look for other cloud providers). But, unfortunately, it makes it VERY hard to figure out how to use it. Then there are a few really basic ones that only do object storage, and aren't always even well written. I took the easy path for now, but I was thinking, there are good C libraries here; NIM would do well to have a strong, well-documented library, even if it wraps under the hood. Especially if it can abstract across multiple providers reasonably well. Even if someone were to do that, and it were to become fairly popular, if it's not part of something "official", you're going to leave engineers doing a lot of legwork on these kinds of decisions. > > I've got several people at work who, on paper should like the language, and > > are smart, senior engineers, who love the concepts. But they have been > > turned off by the docs, and are not willing to put in the effort to switch > > from Go or what have you. > > this is definitely sad to hear. from what I can tell, go seems very strong in > network security area and nim might not be that enticing to them until it > grows and settles better for that niche. I don't think it's particularly security-industry related feedback. I've heard it from a couple of friends who I've gushed about the language to, who aren't in security. Though, I guess to the discussion above, the crypto in Nim isn't easy well-documented (and nimcrypto is certainly not thorough either). But nobody on my team was looking at any of the crypto bits. > > For instance, the first time I had to use os.getAppFileName(), the fact > > that the documentation was right under os.getAppDir() let me to assume that > > os.getAppFileName() would be just the file name, not the full path (i.e., > > that I would need to do joinPath() on the two. It wasn't a lot of friction > > to find out I was wrong and fix it, but those are pretty basic semantics > > that should be in the docs that aren't. Enough little paper cuts like that > > will drive away many people from the target audience. > > I agree there are a number of paper cuts like these occasionally to be found > in the docs. They are not too many nor not too bad and I would say these are > the type of things that individual contributors could improve on without too > much guidance. You're right, I think the reference docs are solid reference docs. There aren't too many paper cuts there, but some of the things we discuss above (including compiler messages, some gaps in more tutorial/howto style content, and the ability to easily find either the docs or libraries that are most appropriate) are probably more important. > instead I would like to remind myself and others on one of the great efforts > that I think contributed a lot to improve the documentation, at the same time > onboarding new people: an activity to improve stdlib docs led by @miran in > 2019 <https://github.com/nim-lang/Nim/issues/10330> > > still thankful about that, not sure if it would be worth the effort of doing > again something similar but it is a great example of past work in > documentation. Yes, perhaps I didn't state it clearly enough, but I found the reference docs to be quite good, and when I went beyond that to the tutorials and so on that do exist, they were usually good as well. It's nice to see, and do appreciate all the hard work that's gone into everything. > > For instance, one member of my team tried to get the VSCode integration > > working, installed one of the major lldb integrations, and yet for whatever > > reason, none of his set breakpoints would ever fire. He couldn't find any > > FAQ or README to help quickly, and that was the final straw, he went back > > to Go. Maybe he was just doing something wrong, but he knows how to use a > > debugger w/ VSCode (whereas I'm just a unix grey beard using emacs as my > > IDE). > > I would say this is another known pain points of nim. Debugger is not a > widespread or straightforward practice. Another good place for some "official" docs... a HOW-TO for setting up debuggers on various platforms (not just on the command line, but specifically in VSCode, by far the most popular IDE today), and a tutorial on debugging, because a LOT of people coming to Nim (esp. those coming from Python who never discovered pdb) would love to learn if it could be made accessable enough. > > All in all, two months in, Nim's my favorite language of all time (and over > > the last 25 years I've taught both programming languages and compilers > > plenty of times). I do feel like it should be a lot more popular than it is. >> >> If the ecosystem were bigger, maybe the docs would be sufficient. But I >> believe the ecosystem can't grow fast enough for that, unless the docs are >> even better than they already are, by a lot. > > this is very nice to hear and I think we all share the fact that we feel like > it should be a lot more popular than it is. > > indeed this leads to what was indeed the main topic of the thread: how can we > drive Nim's adoption? and I gather a partial answer you give in this thread > is: > > the topic of how do we make nim successful is also a topic that is > occasionally discussed here in the forum (that might excuse some of the > rudeness of other commenters ;)), but it sure is not a topic that is going to > go out of fashion too soon (alas!). it is a very broad topic. I am still not > convinced by this discussion that reference docs are the most important thing > to focus on, but I have always believed that more general documents (as > @mratsim reminds us: how to, tutorials, explanation) should definitely be a > priority! (my number one priority is community building) That all makes sense! I was never trying to say the docs are the big problem, just a) point out that it's probably still part of the issue, and b) understand more deeply what can be done to drive more adoption. > In my spare limited spare time, I have started nimib project with the > explicit goal of producing more of this kind of nim documentation (I think > last year I have produced over 20 single documents using this on various nim > topics, in nblog and elsewhere - unfortunately discoverability is rather bad > yet). Incidentally there is also a ongoing very welcome effort to translating > the wiki to nimib(ook), which might help with wiki. Thank you as well for your hard work :) > Apart from this type of documentation, I think a very welcome type of nim > related content from experienced newcomers such as yourself is some kind of > formal or informal essay on why they find nim great (given their background). > There has been a recent example shared in the forum which made some waves > also on HN and given your background I think it an extended essay would be > very much welcome (here on forum but possibly also as a guest blogpost). Well, I personally will try to find time to do such things. I can't make any guarantees, but I'm far more excited about Nim in 2023 than I was about Python in 1994 (and I was pretty excited about it), so I do want to help if I can! I should say thanks again, too. I really appreciate you putting so much effort in your thoughtful response. John
