Re: [protobuf] Proposal: a mechanism to deal with sensitive/redacted fields in string output

['Adam Cozzette' via Protocol Buffers]

*From: *Zellyn Hunter 
*Date: *Mon, May 13, 2019 at 8:16 AM
*To: *Adam Cozzette
*Cc: *Josh Humphries, Protocol Buffers

On Fri, May 10, 2019 at 6:06 PM Adam Cozzette  wrote:
>
>> I asked for feedback about this proposal within Google and unfortunately
>> it sounds like there's not a lot of support for accepting this kind of
>> change. The general feedback I got was that it's best to simply avoid
>> printing out any protos at all if they might contain sensitive information.
>> This kind of feature might provide a false sense of security and encourage
>> developers to print out protos that haven't necessarily been fully
>> annotated with the sensitive field option. There was some agreement that in
>> Java it is particularly easy to print stringified protos by accident, but
>> it seems that ideally we would want to disable that behavior entirely
>> rather than redacting particular fields.
>>
>
> For what it's worth, when discussing this before, some folks on the
> Protobuf Team mentioned that the parts of Google that deal with financial
> transactions actually have something similar to our proposal. Or at least
> something that accomplishes the same goal.
>

That is true, but from what I understand that solution is a bit different.
It is built as libraries and tools on top of protobuf, so while it has the
advantage of not needing access to protobuf internals, it doesn't really
prevent accidental stringification the same way. You would have to call a
library function to sanitize a message, so this would not just happen
automatically.

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/CADqAXr6r1k%3Dr46ZS2-gGHd7TEQc1wh0OjQM8rRMMyYo3R2r8yA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Proposal: a mechanism to deal with sensitive/redacted fields in string output

[Zellyn Hunter]

On Fri, May 10, 2019 at 6:06 PM Adam Cozzette  wrote:

> I asked for feedback about this proposal within Google and unfortunately
> it sounds like there's not a lot of support for accepting this kind of
> change. The general feedback I got was that it's best to simply avoid
> printing out any protos at all if they might contain sensitive information.
> This kind of feature might provide a false sense of security and encourage
> developers to print out protos that haven't necessarily been fully
> annotated with the sensitive field option. There was some agreement that in
> Java it is particularly easy to print stringified protos by accident, but
> it seems that ideally we would want to disable that behavior entirely
> rather than redacting particular fields.
>

For what it's worth, when discussing this before, some folks on the
Protobuf Team mentioned that the parts of Google that deal with financial
transactions actually have something similar to our proposal. Or at least
something that accomplishes the same goal.


> I gather that Square is already relying on this functionality in its
> internal protobuf fork, so I would say if it helps we could probably at
> least try to refactor things to minimize the complexity of maintaining that
> behavior difference.
>
That would be super-helpful. I'll have to catch up on the current state of
protobuf library code and figure out how to allow convenient interception.

Zellyn

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/CAMQ7dq5jRg9TT0uMfiOreD0F1K4d1EXtUEqP9SkE5aC1jt9UKA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Proposal: a mechanism to deal with sensitive/redacted fields in string output

['Adam Cozzette' via Protocol Buffers]

I asked for feedback about this proposal within Google and unfortunately it
sounds like there's not a lot of support for accepting this kind of change.
The general feedback I got was that it's best to simply avoid printing out
any protos at all if they might contain sensitive information. This kind of
feature might provide a false sense of security and encourage developers to
print out protos that haven't necessarily been fully annotated with the
sensitive field option. There was some agreement that in Java it is
particularly easy to print stringified protos by accident, but it seems
that ideally we would want to disable that behavior entirely rather than
redacting particular fields.

I gather that Square is already relying on this functionality in its
internal protobuf fork, so I would say if it helps we could probably at
least try to refactor things to minimize the complexity of maintaining that
behavior difference.

*From: *Zellyn Hunter 
*Date: *Thu, Apr 25, 2019 at 7:01 AM
*To: *Josh Humphries
*Cc: *Protocol Buffers

Hey there, just checking in from Paternity Leave. Last I heard, the
> Protobuf team was not opposed to the idea, but thought it would be
> relatively invasive and thus unlikely to get accepted. I believe that it
> will hardly be invasive at all, so I think the most-likely-to-succeed
> course of action is to provide a related set of minimal patches to (a)
> protoc, (b) the C protobuf implementation, (c) the Java implementation and
> (d) the Go implementation, supporting sensitive fields.
>
> Unfortunately I just haven't found the time to do all that.
>
> Zellyn
>
>
> On Fri, Apr 19, 2019 at 5:12 PM Josh Humphries 
> wrote:
>
>> I don't think there has been any movement on this, but I'd like to ping
>> the thread again.
>>
>> I am still a strong proponent of standard metadata in the proto source
>> and descriptor to indicate sensitive things. I also think it's a truly wise
>> idea to use that information to trigger auto-redaction of sensitive data
>> when serializing message in certain contexts (such as "stringification",
>> like when emitting the data into a debug log message).
>>
>> At FullStory, we've built some interesting tools for using protobufs and
>> gRPC. Most recent is a web UI for gRPC:
>> https://github.com/fullstorydev/grpcui. I would *love* to be able to add
>> features to some of these tools that can be aware of sensitive data in
>> messages that they handle and even give the data special treatment in some
>> ways. For the web UI, there's a few interesting things that a client might
>> choose to do in the face of sensitive data. For one, if that page includes
>> an analytics/recording tool, it could make sure the elements in the DOM
>> that may contain sensitive data are excluded from analysis/recording.
>> Another: it could refuse to send an RPC over an insecure connection if any
>> of the message's fields are marked as sensitive.
>>
>> Outside of the web UI, especially in a world with GDPR, there are all
>> kinds of tools that could be built that care deeply about whether data is
>> sensitive or not. For example, static analysis/linters that make sure
>> sensitive data is treated in a suitably sensitive manner. Without a
>> standard way to denote that in the proto, open-source tools in this vein
>> become more complicated to use. If open-source tools must provide their own
>> custom option (as an example), and then someone wants to use multiple such
>> tools on their codebase, they end up having to redundantly define multiple
>> options on each sensitive field. (Admittedly, this is a manufactured
>> hypothetical.)
>>
>> Anyhow, where do things stand with this enhancement request? Is there
>> anything I can do to stoke the fire and get some attention on it?
>>
>> 
>> *Josh Humphries*
>> jh...@bluegosling.com
>>
>>
>> On Wed, Aug 22, 2018 at 10:01 AM Zellyn  wrote:
>>
>>> Apologies for the long delay, but I got radically reassigned at work, so
>>> I haven't had much time to work on this. But it keeps niggling at me,
>>> because I hate our internal protobuf forks so much.
>>>
>>> Here is the proposal: Proto Proposal: a “sensitive” field option
>>> 
>>> .
>>>
>>> Zellyn
>>>
>>> On Wednesday, February 22, 2017 at 12:10:14 PM UTC-5, Adam Cozzette
>>> wrote:

 Hi Zellyn, this sounds like a reasonable idea. As the next step could
 you perhaps write up a short proposal with more details on what exactly it
 would mean for a field to be redacted? To me it seems like the important
 thing would be to make sure it's clear how redacted fields are supposed to
 be behave in each situation (i.e. when they should be dropped or not), so
 that there's no uncertainty about when they're dropped and when they're
 preserved. (For example, we might say that they're never shown when a proto
 is implicitly stringified but maybe preserved in all other situations?) We
 might 

Re: [protobuf] Proposal: a mechanism to deal with sensitive/redacted fields in string output

['Adam Cozzette' via Protocol Buffers]

Let me bring this up again with the rest of the protobuf team and see if
there is a consensus on this.

On Thu, Apr 25, 2019 at 7:01 AM Zellyn Hunter  wrote:

> Hey there, just checking in from Paternity Leave. Last I heard, the
> Protobuf team was not opposed to the idea, but thought it would be
> relatively invasive and thus unlikely to get accepted. I believe that it
> will hardly be invasive at all, so I think the most-likely-to-succeed
> course of action is to provide a related set of minimal patches to (a)
> protoc, (b) the C protobuf implementation, (c) the Java implementation and
> (d) the Go implementation, supporting sensitive fields.
>
> Unfortunately I just haven't found the time to do all that.
>
> Zellyn
>
>
> On Fri, Apr 19, 2019 at 5:12 PM Josh Humphries 
> wrote:
>
>> I don't think there has been any movement on this, but I'd like to ping
>> the thread again.
>>
>> I am still a strong proponent of standard metadata in the proto source
>> and descriptor to indicate sensitive things. I also think it's a truly wise
>> idea to use that information to trigger auto-redaction of sensitive data
>> when serializing message in certain contexts (such as "stringification",
>> like when emitting the data into a debug log message).
>>
>> At FullStory, we've built some interesting tools for using protobufs and
>> gRPC. Most recent is a web UI for gRPC:
>> https://github.com/fullstorydev/grpcui. I would *love* to be able to add
>> features to some of these tools that can be aware of sensitive data in
>> messages that they handle and even give the data special treatment in some
>> ways. For the web UI, there's a few interesting things that a client might
>> choose to do in the face of sensitive data. For one, if that page includes
>> an analytics/recording tool, it could make sure the elements in the DOM
>> that may contain sensitive data are excluded from analysis/recording.
>> Another: it could refuse to send an RPC over an insecure connection if any
>> of the message's fields are marked as sensitive.
>>
>> Outside of the web UI, especially in a world with GDPR, there are all
>> kinds of tools that could be built that care deeply about whether data is
>> sensitive or not. For example, static analysis/linters that make sure
>> sensitive data is treated in a suitably sensitive manner. Without a
>> standard way to denote that in the proto, open-source tools in this vein
>> become more complicated to use. If open-source tools must provide their own
>> custom option (as an example), and then someone wants to use multiple such
>> tools on their codebase, they end up having to redundantly define multiple
>> options on each sensitive field. (Admittedly, this is a manufactured
>> hypothetical.)
>>
>> Anyhow, where do things stand with this enhancement request? Is there
>> anything I can do to stoke the fire and get some attention on it?
>>
>> 
>> *Josh Humphries*
>> jh...@bluegosling.com
>>
>>
>> On Wed, Aug 22, 2018 at 10:01 AM Zellyn  wrote:
>>
>>> Apologies for the long delay, but I got radically reassigned at work, so
>>> I haven't had much time to work on this. But it keeps niggling at me,
>>> because I hate our internal protobuf forks so much.
>>>
>>> Here is the proposal: Proto Proposal: a “sensitive” field option
>>> 
>>> .
>>>
>>> Zellyn
>>>
>>> On Wednesday, February 22, 2017 at 12:10:14 PM UTC-5, Adam Cozzette
>>> wrote:

 Hi Zellyn, this sounds like a reasonable idea. As the next step could
 you perhaps write up a short proposal with more details on what exactly it
 would mean for a field to be redacted? To me it seems like the important
 thing would be to make sure it's clear how redacted fields are supposed to
 be behave in each situation (i.e. when they should be dropped or not), so
 that there's no uncertainty about when they're dropped and when they're
 preserved. (For example, we might say that they're never shown when a proto
 is implicitly stringified but maybe preserved in all other situations?) We
 might also need to be careful to get this right for all languages early;
 even if there's some language where we don't care about redaction for now,
 it will be hard to change later without making a breaking change.

 On Thu, Feb 16, 2017 at 1:45 PM, zellyn via Protocol Buffers <
 prot...@googlegroups.com> wrote:

> There are many ways that protocol buffers might be stringified into
> logs, accidentally or on purpose, printed in stack traces, etc. The
> built-in behavior stringifies the entire protobuf recursively, including
> all field data.
>
> At Square, we deal with payments, and often have data of varying
> sensitivity in protobuf fields, which we'd like to be elided from
> stringified output.
>
> We use an internal fork of protoc to handle a custom field option,
> "redacted", and have also patched the 

Re: [protobuf] Proposal: a mechanism to deal with sensitive/redacted fields in string output

[Zellyn Hunter]

Hey there, just checking in from Paternity Leave. Last I heard, the
Protobuf team was not opposed to the idea, but thought it would be
relatively invasive and thus unlikely to get accepted. I believe that it
will hardly be invasive at all, so I think the most-likely-to-succeed
course of action is to provide a related set of minimal patches to (a)
protoc, (b) the C protobuf implementation, (c) the Java implementation and
(d) the Go implementation, supporting sensitive fields.

Unfortunately I just haven't found the time to do all that.

Zellyn


On Fri, Apr 19, 2019 at 5:12 PM Josh Humphries 
wrote:

> I don't think there has been any movement on this, but I'd like to ping
> the thread again.
>
> I am still a strong proponent of standard metadata in the proto source and
> descriptor to indicate sensitive things. I also think it's a truly wise
> idea to use that information to trigger auto-redaction of sensitive data
> when serializing message in certain contexts (such as "stringification",
> like when emitting the data into a debug log message).
>
> At FullStory, we've built some interesting tools for using protobufs and
> gRPC. Most recent is a web UI for gRPC:
> https://github.com/fullstorydev/grpcui. I would *love* to be able to add
> features to some of these tools that can be aware of sensitive data in
> messages that they handle and even give the data special treatment in some
> ways. For the web UI, there's a few interesting things that a client might
> choose to do in the face of sensitive data. For one, if that page includes
> an analytics/recording tool, it could make sure the elements in the DOM
> that may contain sensitive data are excluded from analysis/recording.
> Another: it could refuse to send an RPC over an insecure connection if any
> of the message's fields are marked as sensitive.
>
> Outside of the web UI, especially in a world with GDPR, there are all
> kinds of tools that could be built that care deeply about whether data is
> sensitive or not. For example, static analysis/linters that make sure
> sensitive data is treated in a suitably sensitive manner. Without a
> standard way to denote that in the proto, open-source tools in this vein
> become more complicated to use. If open-source tools must provide their own
> custom option (as an example), and then someone wants to use multiple such
> tools on their codebase, they end up having to redundantly define multiple
> options on each sensitive field. (Admittedly, this is a manufactured
> hypothetical.)
>
> Anyhow, where do things stand with this enhancement request? Is there
> anything I can do to stoke the fire and get some attention on it?
>
> 
> *Josh Humphries*
> jh...@bluegosling.com
>
>
> On Wed, Aug 22, 2018 at 10:01 AM Zellyn  wrote:
>
>> Apologies for the long delay, but I got radically reassigned at work, so
>> I haven't had much time to work on this. But it keeps niggling at me,
>> because I hate our internal protobuf forks so much.
>>
>> Here is the proposal: Proto Proposal: a “sensitive” field option
>> 
>> .
>>
>> Zellyn
>>
>> On Wednesday, February 22, 2017 at 12:10:14 PM UTC-5, Adam Cozzette wrote:
>>>
>>> Hi Zellyn, this sounds like a reasonable idea. As the next step could
>>> you perhaps write up a short proposal with more details on what exactly it
>>> would mean for a field to be redacted? To me it seems like the important
>>> thing would be to make sure it's clear how redacted fields are supposed to
>>> be behave in each situation (i.e. when they should be dropped or not), so
>>> that there's no uncertainty about when they're dropped and when they're
>>> preserved. (For example, we might say that they're never shown when a proto
>>> is implicitly stringified but maybe preserved in all other situations?) We
>>> might also need to be careful to get this right for all languages early;
>>> even if there's some language where we don't care about redaction for now,
>>> it will be hard to change later without making a breaking change.
>>>
>>> On Thu, Feb 16, 2017 at 1:45 PM, zellyn via Protocol Buffers <
>>> prot...@googlegroups.com> wrote:
>>>
 There are many ways that protocol buffers might be stringified into
 logs, accidentally or on purpose, printed in stack traces, etc. The
 built-in behavior stringifies the entire protobuf recursively, including
 all field data.

 At Square, we deal with payments, and often have data of varying
 sensitivity in protobuf fields, which we'd like to be elided from
 stringified output.

 We use an internal fork of protoc to handle a custom field option,
 "redacted", and have also patched the stringification code to print
 "[REDACTED]" for those fields. We do the same in Go, and in the C
 implementation (for Ruby).

 Last year, we chatted with the protobuf team, and they were sympathetic
 to our use case (in fact, they mentioned that the 

Re: [protobuf] Proposal: a mechanism to deal with sensitive/redacted fields in string output

[Josh Humphries]

I don't think there has been any movement on this, but I'd like to ping the
thread again.

I am still a strong proponent of standard metadata in the proto source and
descriptor to indicate sensitive things. I also think it's a truly wise
idea to use that information to trigger auto-redaction of sensitive data
when serializing message in certain contexts (such as "stringification",
like when emitting the data into a debug log message).

At FullStory, we've built some interesting tools for using protobufs and
gRPC. Most recent is a web UI for gRPC:
https://github.com/fullstorydev/grpcui. I would *love* to be able to add
features to some of these tools that can be aware of sensitive data in
messages that they handle and even give the data special treatment in some
ways. For the web UI, there's a few interesting things that a client might
choose to do in the face of sensitive data. For one, if that page includes
an analytics/recording tool, it could make sure the elements in the DOM
that may contain sensitive data are excluded from analysis/recording.
Another: it could refuse to send an RPC over an insecure connection if any
of the message's fields are marked as sensitive.

Outside of the web UI, especially in a world with GDPR, there are all kinds
of tools that could be built that care deeply about whether data is
sensitive or not. For example, static analysis/linters that make sure
sensitive data is treated in a suitably sensitive manner. Without a
standard way to denote that in the proto, open-source tools in this vein
become more complicated to use. If open-source tools must provide their own
custom option (as an example), and then someone wants to use multiple such
tools on their codebase, they end up having to redundantly define multiple
options on each sensitive field. (Admittedly, this is a manufactured
hypothetical.)

Anyhow, where do things stand with this enhancement request? Is there
anything I can do to stoke the fire and get some attention on it?


*Josh Humphries*
jh...@bluegosling.com


On Wed, Aug 22, 2018 at 10:01 AM Zellyn  wrote:

> Apologies for the long delay, but I got radically reassigned at work, so I
> haven't had much time to work on this. But it keeps niggling at me, because
> I hate our internal protobuf forks so much.
>
> Here is the proposal: Proto Proposal: a “sensitive” field option
> 
> .
>
> Zellyn
>
> On Wednesday, February 22, 2017 at 12:10:14 PM UTC-5, Adam Cozzette wrote:
>>
>> Hi Zellyn, this sounds like a reasonable idea. As the next step could you
>> perhaps write up a short proposal with more details on what exactly it
>> would mean for a field to be redacted? To me it seems like the important
>> thing would be to make sure it's clear how redacted fields are supposed to
>> be behave in each situation (i.e. when they should be dropped or not), so
>> that there's no uncertainty about when they're dropped and when they're
>> preserved. (For example, we might say that they're never shown when a proto
>> is implicitly stringified but maybe preserved in all other situations?) We
>> might also need to be careful to get this right for all languages early;
>> even if there's some language where we don't care about redaction for now,
>> it will be hard to change later without making a breaking change.
>>
>> On Thu, Feb 16, 2017 at 1:45 PM, zellyn via Protocol Buffers <
>> prot...@googlegroups.com> wrote:
>>
>>> There are many ways that protocol buffers might be stringified into
>>> logs, accidentally or on purpose, printed in stack traces, etc. The
>>> built-in behavior stringifies the entire protobuf recursively, including
>>> all field data.
>>>
>>> At Square, we deal with payments, and often have data of varying
>>> sensitivity in protobuf fields, which we'd like to be elided from
>>> stringified output.
>>>
>>> We use an internal fork of protoc to handle a custom field option,
>>> "redacted", and have also patched the stringification code to print
>>> "[REDACTED]" for those fields. We do the same in Go, and in the C
>>> implementation (for Ruby).
>>>
>>> Last year, we chatted with the protobuf team, and they were sympathetic
>>> to our use case (in fact, they mentioned that the part of Google that deals
>>> with payments has something similar internally: I think that's where the
>>> "sensitive" name came from). I'd like to get that discussion rolling again.
>>>
>>> We'd like to see one of the following happen (in decreasing order of
>>> awesomeness for us):
>>>
>>>- upstreaming of the "redacted" field option, and modification of
>>>the runtimes to elide redacted fields when stringifying
>>>- introduction of generic interception points to selectively
>>>override default stringification behavior in Java, Go, and Ruby (at 
>>> least).
>>>- addition of a "SerializeToString" or equivalent method, and
>>>removal of default full-stringification behavior of the toString 

Re: [protobuf] Proposal: a mechanism to deal with sensitive/redacted fields in string output

[Zellyn]

Apologies for the long delay, but I got radically reassigned at work, so I 
haven't had much time to work on this. But it keeps niggling at me, because 
I hate our internal protobuf forks so much.

Here is the proposal: Proto Proposal: a “sensitive” field option 

.

Zellyn

On Wednesday, February 22, 2017 at 12:10:14 PM UTC-5, Adam Cozzette wrote:
>
> Hi Zellyn, this sounds like a reasonable idea. As the next step could you 
> perhaps write up a short proposal with more details on what exactly it 
> would mean for a field to be redacted? To me it seems like the important 
> thing would be to make sure it's clear how redacted fields are supposed to 
> be behave in each situation (i.e. when they should be dropped or not), so 
> that there's no uncertainty about when they're dropped and when they're 
> preserved. (For example, we might say that they're never shown when a proto 
> is implicitly stringified but maybe preserved in all other situations?) We 
> might also need to be careful to get this right for all languages early; 
> even if there's some language where we don't care about redaction for now, 
> it will be hard to change later without making a breaking change.
>
> On Thu, Feb 16, 2017 at 1:45 PM, zellyn via Protocol Buffers <
> prot...@googlegroups.com > wrote:
>
>> There are many ways that protocol buffers might be stringified into logs, 
>> accidentally or on purpose, printed in stack traces, etc. The built-in 
>> behavior stringifies the entire protobuf recursively, including all field 
>> data.
>>
>> At Square, we deal with payments, and often have data of varying 
>> sensitivity in protobuf fields, which we'd like to be elided from 
>> stringified output.
>>
>> We use an internal fork of protoc to handle a custom field option, 
>> "redacted", and have also patched the stringification code to print 
>> "[REDACTED]" for those fields. We do the same in Go, and in the C 
>> implementation (for Ruby).
>>
>> Last year, we chatted with the protobuf team, and they were sympathetic 
>> to our use case (in fact, they mentioned that the part of Google that deals 
>> with payments has something similar internally: I think that's where the 
>> "sensitive" name came from). I'd like to get that discussion rolling again.
>>
>> We'd like to see one of the following happen (in decreasing order of 
>> awesomeness for us):
>>
>>- upstreaming of the "redacted" field option, and modification of the 
>>runtimes to elide redacted fields when stringifying
>>- introduction of generic interception points to selectively override 
>>default stringification behavior in Java, Go, and Ruby (at least).
>>- addition of a "SerializeToString" or equivalent method, and removal 
>>of default full-stringification behavior of the toString (Java), String 
>>(Go), etc. - that way you only serialize on purpose
>>   - many tests rely on string comparison, even though nobody is 
>>   supposed to rely on it being stable - perhaps the default behavior 
>> could 
>>   compute a hash?
>>
>> Josh Humpries (who now works at Fullstory) created a proposal 
>>  a while back, but it 
>> didn't go anywhere. I reached out to the protobuf team, and Damien Neil 
>> suggested that this group was the appropriate place to propose such changes.
>>
>> Bikeshed away!
>>
>> Zellyn
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to protobuf+u...@googlegroups.com .
>> To post to this group, send email to prot...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

Re: [protobuf] Proposal: a mechanism to deal with sensitive/redacted fields in string output

['Adam Cozzette' via Protocol Buffers]

Hi Zellyn, this sounds like a reasonable idea. As the next step could you
perhaps write up a short proposal with more details on what exactly it
would mean for a field to be redacted? To me it seems like the important
thing would be to make sure it's clear how redacted fields are supposed to
be behave in each situation (i.e. when they should be dropped or not), so
that there's no uncertainty about when they're dropped and when they're
preserved. (For example, we might say that they're never shown when a proto
is implicitly stringified but maybe preserved in all other situations?) We
might also need to be careful to get this right for all languages early;
even if there's some language where we don't care about redaction for now,
it will be hard to change later without making a breaking change.

On Thu, Feb 16, 2017 at 1:45 PM, zellyn via Protocol Buffers <
protobuf@googlegroups.com> wrote:

> There are many ways that protocol buffers might be stringified into logs,
> accidentally or on purpose, printed in stack traces, etc. The built-in
> behavior stringifies the entire protobuf recursively, including all field
> data.
>
> At Square, we deal with payments, and often have data of varying
> sensitivity in protobuf fields, which we'd like to be elided from
> stringified output.
>
> We use an internal fork of protoc to handle a custom field option,
> "redacted", and have also patched the stringification code to print
> "[REDACTED]" for those fields. We do the same in Go, and in the C
> implementation (for Ruby).
>
> Last year, we chatted with the protobuf team, and they were sympathetic to
> our use case (in fact, they mentioned that the part of Google that deals
> with payments has something similar internally: I think that's where the
> "sensitive" name came from). I'd like to get that discussion rolling again.
>
> We'd like to see one of the following happen (in decreasing order of
> awesomeness for us):
>
>- upstreaming of the "redacted" field option, and modification of the
>runtimes to elide redacted fields when stringifying
>- introduction of generic interception points to selectively override
>default stringification behavior in Java, Go, and Ruby (at least).
>- addition of a "SerializeToString" or equivalent method, and removal
>of default full-stringification behavior of the toString (Java), String
>(Go), etc. - that way you only serialize on purpose
>   - many tests rely on string comparison, even though nobody is
>   supposed to rely on it being stable - perhaps the default behavior could
>   compute a hash?
>
> Josh Humpries (who now works at Fullstory) created a proposal
>  a while back, but it
> didn't go anywhere. I reached out to the protobuf team, and Damien Neil
> suggested that this group was the appropriate place to propose such changes.
>
> Bikeshed away!
>
> Zellyn
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.