> On Feb. 9, 2017, 6:31 p.m., Jie Yu wrote: > > include/mesos/mesos.proto, line 1994 > > <https://reviews.apache.org/r/56052/diff/3/?file=1628247#file1628247line1994> > > > > Actually, one thought here. Given that environment varaible's source > > cannot be a byte stream. Do we want to have explicit typing: REFERENCE, > > TEXT, BYTES for secret so that the validation for environment source can be > > easier and more explicit? > > Greg Mann wrote: > When a user specifies a REFERENCE-type Secret, would they need to be able > to specify text vs. bytes in that case? If so, then we might want two > different sets of type information. I don't know if a user's module > implementation might need to know the encoding (or lack thereof) of a > REFERENCE Secret when fetching it? > > If we don't think the REFERENCE-type secret needs any encoding > information, then we could use {REFERENCE, TEXT} for the enum types in this > patch, so that we could add a BYTES type at the top level later. Another > option would be to include type information inside the Value message, but > this adds a bit more complexity to the message. > > Jie Yu wrote: > Yes, you're right that this also applies to REFERENCE type secret. What > happen in the future that we start to support REFERENCE type through modules. > What the agent should do for environment variable. Does the agent need to do > base64 encoding (as you mentioned, how does the agent know that the content > received is a string or byte stream, and how does the user know?), or leave > it as a plain string? > > Looks like k8s secret always do base64 encoding? Maybe that's how they > solve the issue? > > Let's do our due diligence to evaluate all the options thoroughly because > this is a very important API.
Yea it looks like k8s solves this by always base64-encoding the secret data. Since k8s provides its own secret-store, they can establish this constraint easily. In our case, if we want to support arbitrary back-ends, we can't necessarily make this assumption. For example, see the API section of the docs for the Hashicorp Vault PKI backend: https://www.vaultproject.io/docs/secrets/pki/. Some of these endpoints return raw binary data. If we require that a secret fetcher module be used to populate REFERENCE type secrets, then we could still require base64-encoding, and the secret fetcher could be responsible for encoding fetched secrets appropriately. Let's consider a concrete example: a cluster using a Hashicorp Vault secret store which has two back-ends loaded: 'generic' (the default) and 'pki'. Given the current `Secret` design in this patch, a fetcher module could infer the correct back-end to use via the `reference.name`. For example, the module could map these names directly onto the Vault API: a name of '/pki/ca' would grab the PKI back-end's CA, and a name of '/pki/cert/<serial#>' could grab a particular certificate from the PKI back-end. A name of '/generic/my_secret' could fetch the secret named "my_secret" in the generic back-end. In any case, if we wanted to mandate only base64-encoded secrets within Mesos, the secret fetcher could be aware of the type of data received from the back-end's API and encode it appropriately, if necessary. Or, we could allow binary secrets and include explicit type information to distinguish between text and binary. If we do this, however, it's not clear to me that the user launching a task should need to specify whether the secret is text or bytes - perhaps the secret fetcher could determine this from a particular secret name, or perhaps some secret stores may return both text and binary secrets from the same API call? I'm not sure, need to do some more research. If we want to provide a more generic HTTP-based secret-fetching model in which a user could specify an arbitrary endpoint that would return the desired secret, then it makes sense for that user to specify text vs. bytes. However, in that case it's not clear to me where the generic fetcher gets the credentials needed to fetch the secret itself. This brings me to another important issue regarding secret fetching: how does the fetcher establish that the executor or task in question is authorized to fetch the specified secret? With this proto, we're attempting to provide a first-class object permitting the secure fetching of secrets; do we also want to provide a first-class identity associated with the executor/task for the purpose of authorizing the secret-fetching? - Greg ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56052/#review164960 ----------------------------------------------------------- On Feb. 9, 2017, 6:33 a.m., Greg Mann wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56052/ > ----------------------------------------------------------- > > (Updated Feb. 9, 2017, 6:33 a.m.) > > > Review request for mesos, Adam B, Jie Yu, Kapil Arya, Jan Schlicht, and Vinod > Kone. > > > Bugs: MESOS-6996 > https://issues.apache.org/jira/browse/MESOS-6996 > > > Repository: mesos > > > Description > ------- > > This patch adds a new `Secret` protobuf message which > is designed to serve as a generic mechanism for passing > priviledged information within Mesos. > > > Diffs > ----- > > include/mesos/mesos.proto 34a288b6f5c2fca090a7aa7a61798e3255d6663a > include/mesos/v1/mesos.proto 6638111d10f4a36cdf91dfce1019871e9839c306 > > Diff: https://reviews.apache.org/r/56052/diff/ > > > Testing > ------- > > `make check` > > > Thanks, > > Greg Mann > >
