Thanks, thoughts below. > trying to be synthetic, but I have enjoyed the exchange, there is material to > untangle in multiple threads here. :) > on concepts: looking forward to your > write up! > on compiler errors: yep, they can get bad, with some experience > they become better to navigate, still could definitely be an area of > improvement. an howto guide on compiler errors should be much welcome. > on > nimcrypto: ok, now if I can see where you come from and the gist of the > criticism. maybe some of your remarks could end up in issue in the repo. how > to use crypto is definitely more difficult and require special expertise than > how to implement crypto. It's also _incredibly_ easy to do things that make > crypto code susceptible to side-channel attacks. When algorithms don't run in > constant time, you can bet there's a failure mode that's significant. I > really don't love OpenSSL, but I'd prefer to better document and expose their > stuff, with well-thought out APIs than to re-invent the wheel, unless there's > going to be a VERY big investment. > > on making random a secure call: I guess one of the advantages of current call > to random is that is fast and a secure version would necessarily be slower. > current one is based on some variations of xoroshiro that aims for speed and > statistical quality and still would be the one to use for example in monte > carlo simulation in a scientific context. whether the default is the fast or > the secure one I guess it could be up to debate (and not sure I am sold on > this) and an alterantive could be provided: secure_random or fast_random. > pretty sure that for nim 2.0 the ship is already sailed though (it is frozen > and only bugfix mode).
It's easy to provide BOTH kinds of APIs (and Nim, in fact, already does). But, the question is, what's the right UX. Obviously, if you make the default that people reach for insecure by default, most people are going to do that. It's not actually too terribly much of an improvement to say, "secure by default, but if you want fast, do call XXX". The psychology of the average person is such that they're going to underestimate the security risk, but to overestimate their needs for performance. And, by the way, using the OS for random numbers is actually pretty fast, far faster than most people expect. There are a few reasons for it: 1. The bulk of the work is in entropy collection and maintenance, and your OS is always doing that anyway. 2. OSes can easily buffer output here, so that you may not be paying much of anything other than the system call. 3. AES-based symmetric crypto on most platforms of the last 15 years, including some pretty cheap ones, will be able to do all the modestly expensive work in hardware. 4. Even in software, AES is more than fast enough for almost every practical use case, even if they have need for MASSIVE amounts of random data. One of the many reasons people underestimate the security risk-- they assume the existing algorithms are good enough. All non-cryptographic algorithms are highly structured, mathematically, which is a particular problem for random numbers. Performance wise, I've got an underpowered AMD Linux box sitting around (a NUC-like form factor), and I just asked it for 100Mb of data via dd, so we can get a sense of the raw cost of the cryptography (that's mostly all we're really going to have to pay): 262144+0 records in 262144+0 records out 134217728 bytes (134 MB, 128 MiB) copied, 0.868457 s, 155 MB/s [viega@triforce sami]$ Now assume a statistical PRNG is as fast as me just reading from /dev/zero. Probably it's not quite as fast, because, while you might not have to pay for system calls, there's still some math in there: [viega@triforce sami]$ dd if=/dev/zero of=randtest count=262144 262144+0 records in 262144+0 records out 134217728 bytes (134 MB, 128 MiB) copied, 0.376467 s, 357 MB/s [viega@triforce sami]$ It's only about twice as fast. Frankly, the performance difference is such a non-issue that I would nuke existing statistical PRNGs all together. If you want something with a solid PROOF that the output is indistinguishable from random, yet you want repeatability, then the library should have a cryptographic pseudo-random number generator, where you can save and load state, if you're a statistician. > developer onboarding/slimming stdlib: I also would tend to prefer a fat > stdlib instead of a slim one but I think the hard argument there is that > there is not enough manpower currently and I think this is a lesson hard > learned by core devs, so it seems a fair choice for this 2.0 iteration. the > theme on trying to ship some nim distribution of libraries is something that > has come up and fusion is indeed recognized as a failed experiment > (unfortunately). centralizing stdlib like iface and others (e.g. pattern > matching) I think is not feasible until those libraries get a decent amount > of usage as nimble package (and yes, with such a young ecosystem it is hard > to avoid the risk of having to rewrite because of obsolete dependencies). I personally believe that empowering people to own things like this will help drive activity. And I do think if significant "official" library work is NOT done, it's going to be a long hard road just to improve the Nim adoption rate. I think it'd be fine to semver libraries, and organize them based on maturity, or something else like that to deal w/ the lack of maturity problem, because I think the biggest barrier is seeing the community commit to maintaining something for the long-term. Knowing that commitment is there would probably be enough to unlock more contributors. > on foundation: no progress that I know of. this is a topic that I think would > advance better with a group of people in the same room (and some prep work). > of course a new thread on that topic would not hurt. nice to hear about the > expert connections, they might turn useful! thanks for the availability! I feel like I'm too new to the community to launch or drive any discussion around the actual governance, but I'll happily pay attention and weigh in when others see fit to start threads on it :)
