I agree with a separate org. I picked here to start simply because most of the code out there that has relevant pieces probably belongs to someone on this mailing list. I think that once it is stable it may be a candidate for inclusion. But until it is stable I wouldn't consider it.
I also agree that some of the packages can possibly kept separate from each other. And there's probably some base package where we'd want to extract some code common to all crypto as you mentioned. Also, I don't want to volunteer anyone, but I think there needs to be a single spot where we try to maintain important code related to crypto. If for no other reason than the trust aspect of it. I understand the concerns about security once it gets large but that's why a group of packages under a single org makes sense. And it provides us to be a little more forward thinking in what we want the packages to look like. The best example I can think of to illustrate my thoughts here is this: *Imagine we wanted to add a check when Phoenix starts up to ensure that the certificate being passed to Cowboy hasn't expired. Or log out a message with the expiry date so that it can be tracked easily.* *That is reusable code. I can think of several other cases where people might want to check the expiry on a certificate. So then we extract it.* *The problem is that the code needed to read out that date is very similar to code needed to read out any details of the certificate. And it probably belongs with code needed to manipulate certificates instead of being in a separate package, because it would either duplicate some of that code, or depend heavily on it and would be useful inside that package.* I am willing to help maintain it. But I definitely can't do this all by myself so I'm looking for support. I also don't want hijack the code that is out there already. So I'm trying to gauge interest in this. If I was to make elixir-crypto who should be a part of it? Thanks for your feedback José. :) On Thursday, October 4, 2018 at 12:48:28 AM UTC-6, José Valim wrote: > > Over my keynote at ElixirConf we discussed when to add things to the > language, which are also in our development page > <https://elixir-lang.org/development.html>, and although crypto is super > important, having crypto functionality in Elixir would not provide anything > different than if it belongs to a package. Therefore, I don't believe it is > a good candidate for inclusion. > > I would not treat the creation of plug_crypto and pbcs as a yellow flag > but I would say quite the opposite, the community is finally starting to > extract and share their crypto code into packages, which is exactly the > direction we want to go. > > In fact, if we want to move to a shared tool, breaking the existing code > apart is a necessity, so we can preserve backwards compatibility for users > of those APIs. However, both packages are still lacking in some areas, > especially in the documentation front. pbcs focuses on algorithm names, so > developers, including myself, have no idea how to use them. That's why the > plug_crypto API focuses on use cases. At the same time, plug_crypto > focusing on use cases means someone looking for an existing algorithm will > have a hard time finding it. > > I would also like to say that there are some disadvantages to creating a > package that attempts to tackle everything related to crypto: > > * The package will provide too many functionality and become too large, > which increases its footprint and makes it harder to maintain > * The APIs will likely grow in size in terms of options, which also > makes it hard to maintain and may become itself a security issue > > Maybe the best approach is to create an elixir-crypto organization and > have multiple packages in there. They can depend on a master package with > basic functionality (like secure_compare) and have everything else built on > top using shared conventions. I am also certain that this should not belong > to the elixir-lang organization. If our answer to every problem is "let's > put it to the Elixir team to maintain", then we are in deep problem. I do > understand you were not necessarily asking the Elixir team to maintain it > but, if it is in the elixir-lang organization, then it is OUR > responsibility. > > Thanks for the proposal, Mark! > > *José Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > > On Thu, Oct 4, 2018 at 4:35 AM Mark Madsen <m...@idyll.io <javascript:>> > wrote: > >> Why not just do this in Erlang? Because it doesn't solve the problem of >> making Elixir easier for developers. If you need to use crypto in Elixir >> right now it's not easy. >> Adding full PKCS7 to OTP still isn't going to make working with it in >> Elixir easy. >> >> The plan here isn't to just wrap things. It's to make them easier. >> >> But instead of arguments lets look at the facts: >> >> - some of code I am proposing is being written, already, in lots of >> places and lots of projects. examples were given. I'm sure there's more. >> - that same code is then being extracted into packages in an attempt to >> share it. >> - by the nature of software development if a bunch of people are writing >> the same code separately it's going to be less secure than if a bunch of >> people write it together and share/review it. >> >> Now, there probably are some enhancements that do belong in OTP. And we >> should make them there. Off the top of my head I think that it's difficult >> to build a PKCS7 message in Erlang. Possibly dealing with detached >> signatures. We probably want those changes in OTP. But the way to discover >> that is to build this on top of OTP and find out what it's missing and then >> work on the pieces we need. >> >> In my opinion this package is already being written. It's just not being >> planned out so we're only getting the pieces that people are extracting out >> of their code. >> >> Instead of that approach lets plan this out, with a focus on developer >> productivity. >> >> >> On Wednesday, October 3, 2018 at 8:01:27 PM UTC-6, Allen Madsen wrote: >>> >>> A proposal like this has popped up before. Perhaps you could address >>> some of the arguments made there. The primary one being that improvements >>> to crypto should happen in OTP. >>> >>> >>> https://groups.google.com/forum/#!searchin/elixir-lang-core/crypto%7Csort:date/elixir-lang-core/J-Idvs6ije8/eC-pmklSBQAJ >>> >>> Allen Madsen >>> http://www.allenmadsen.com >>> >>> >>> On Wed, Oct 3, 2018 at 7:45 PM Mark Madsen <m...@idyll.io> wrote: >>> >>>> Crypto is super hard. And super important. >>>> >>>> Currently Elixir pushes users to call Erlang when working with crypto. >>>> This doesn't align with Elixirs goal of developer productivity. >>>> >>>> My experience on working with Erlang's crypto application is: >>>> >>>> - It is intimidating for newcomer who are just learning Elixir. >>>> - While Erlang provides documentation, none of it refers back >>>> to Public Key Cryptography Standards directly making it difficult to >>>> work >>>> with or understand in the context of other libraries and code. >>>> - Erlang provides building blocks, but they require writing a bunch >>>> of work to be useful (were starting to see this with pbcs extracted out >>>> of >>>> hex and now the work to extract plug_crypto out of plug). >>>> - the data structures, for example the data structure for a >>>> certificate, are hard to work with. >>>> >>>> This has resulted in a couple hex packages up to this date to handle >>>> PKCS#5 and PKCS#7 padding. >>>> And more recently PBCS was extracted out of Hex >>>> https://github.com/hexpm/pbcs and there's work ongoing to extract the >>>> crypto out of Plug https://github.com/elixir-plug/plug_crypto >>>> >>>> Another candidate is the certificate generation in Phoenix from >>>> https://github.com/phoenixframework/phoenix/blob/master/lib/mix/tasks/phx.gen.cert.ex >>>> >>>> - I think much of the code in there should be extracted. It shouldn't live >>>> in a mix task, but instead should come along with language. (not trying to >>>> pick on @voltone, I am very grateful for his work there, I wouldn't have >>>> done any better.) >>>> >>>> But I'm personally guilty of worse. I have a bunch of code needed to >>>> parse certificates to extract details and provisioning profiles to extract >>>> PKCS#7 signature details. >>>> >>>> Crypto is so important that I think we need a central place to interact >>>> with any of the Public Key Cryptography Standards. Ideally we could have >>>> support for: >>>> >>>> - PublicKey/PrivateKey >>>> - X509 >>>> - PKCS#5 >>>> - PKCS#7/CMS >>>> - PKCS#10 >>>> - PKCS#12 >>>> >>>> It would give us an opportunity to write documentation around the usage >>>> of the functions to help out newcomers and provide data structures that >>>> are >>>> easier to work with in Elixir. It also puts as much of this code as >>>> possible in a central place where we can get a lot of eyes >>>> >>>> I know this is going to be a lot of work. If I had to do it myself it'd >>>> probably take a couple years given the time I have to work on this kinda >>>> stuff and that assumes its my only priority (which it isn't). But I also >>>> see some of the pieces of this happening in separate places right now and >>>> if everyone contributed the work they are extracting from their codebases >>>> to one spot we'd probably be 30% or more done. >>>> >>>> Eventually I think this is important enough that it should be part of >>>> Elixir, but to iterate on it initially I think it should be done >>>> separately. >>>> >>>> So I propose that someone makes https://github.com/elixir-lang/crypto >>>> and we start planning this out there. :) >>>> >>>> Thanks >>>> >>>> Mark (@idyll) >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "elixir-lang-core" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to elixir-lang-co...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/elixir-lang-core/dcf99a35-8a2b-4022-830e-983075b486d1%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/elixir-lang-core/dcf99a35-8a2b-4022-830e-983075b486d1%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "elixir-lang-core" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to elixir-lang-co...@googlegroups.com <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elixir-lang-core/57b38fc5-1eef-4d90-a7bf-87629759138b%40googlegroups.com >> >> <https://groups.google.com/d/msgid/elixir-lang-core/57b38fc5-1eef-4d90-a7bf-87629759138b%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/ea3fe40e-1994-45ba-83bd-bac390bf77fd%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
