Re: [go-nuts] The docs for secretbox seem very wrong

[Ian Lance Taylor]

On Mon, Oct 9, 2023, 1:59 PM Dean Schulze  wrote:

> That is mixing concerns (low cohesion), though.  At the very least they
> need to explain that in the docs.  Otherwise that first parameter makes no
> sense.
>

This is a common design pattern in Go.  We shouldn't expect that every
function that uses it has to explain the idea.  We should figure out a way
for people to discover and understand it.

Ian

On Monday, October 9, 2023 at 2:51:11 PM UTC-6 David Anderson wrote:
>
>> The first parameter lets you avoid an allocation. If you're constructing
>> a packet that consists of ciphertext surrounded by some framing, you can
>> make([]byte, 0, lengthOfPacket) to preallocate the necessary amount of
>> storage upfront. Then, calls to secretbox don't allocate.
>>
>> If you don't need that behavior, you can pass in a nil as the first
>> parameter to get a fresh slice allocated for you by the append(), but in
>> hot codepaths involving cryptography, being able to preallocate and reuse
>> storage (e.g. via sync.Pool) can give you significant reduction in GC
>> pressure.
>>
>> - Dave
>>
>> On Mon, Oct 9, 2023, at 08:43, Dean Schulze wrote:
>>
>> So why even bother with the first parameter then?  That looks like a
>> badly designed method signature.
>>
>> On Monday, October 9, 2023 at 8:12:33 AM UTC-6 Axel Wagner wrote:
>>
>> I get the impression that the thing you are missing is that appending to
>> a slice does not modify it. That is, `append(s, x...)` modifies neither the
>> length, nor the content of `s`, you have to type `x = append(s, x...)`.
>> `Seal` (and `Open`) work exactly the same. They append to the `out`
>> parameter - and to enable you to get at the slice, they return it.
>>
>> The docs could be slightly more clear, by explicitly stating that they
>> return the appended-to slice.0
>>
>>
>> On Mon, Oct 9, 2023 at 3:46 PM Dean Schulze  wrote:
>>
>> If the docs are correct, how do you append to nil?  That's what the docs
>> say.  Take a look at them.
>>
>> Your code example shows the first parameter to Seal() can be nil.  So
>> what does that parameter do?  How do you append to it?
>> On Sunday, October 8, 2023 at 11:19:13 PM UTC-6 Axel Wagner wrote:
>>
>> For what it's worth, here is an example that demonstrates a typical
>> encryption/decryption roundtrip, perhaps more clearly:
>> https://go.dev/play/p/ZZry8IgTJQ_-
>> The `out` parameter can be used to make this more efficient by using
>> pre-allocated buffers (depending on use case) and there are cases where you
>> don't have to send the nonce, because you can derive them from common data,
>> which is why both of these parameters are there. But in the most common
>> usage, you'd do what this example does.
>>
>> The example code from the docs tries to be a little bit more efficient
>> and packs the `Box` struct into a single byte, perhaps at the cost of
>> understandability.
>>
>> On Mon, Oct 9, 2023 at 7:06 AM Axel Wagner 
>> wrote:
>>
>> oh I forgot to emphasize: I don't believe the output is *really*
>> ``. That is, I don't believe you can really treat
>> the first N bytes as the encrypted text and decrypt it (say, if you didn't
>> care about the authentication). It's just that you ultimately need to add
>> 16 bytes of extra information to carry that authentication tag, which is
>> why the box needs to be 16 bytes longer than the message. In reality, the
>> two are probably cleverly mixed - I'm not a cryptographer.
>> I just wanted to demonstrate where all the information ultimately goes.
>>
>> On Mon, Oct 9, 2023 at 7:03 AM Axel Wagner 
>> wrote:
>>
>> I don't really understand your issue. You call
>> encrypted := secretbox.Seal(nonce[:], []byte(s), &nonce, &secretKey)
>> That means you pass `nonce[:]` as the `out` argument, `s` as the
>> `message` argument, and the nonce and key and assign the result to
>> `encrypted`.
>> According to the docs of `secretbox`, `Seal` will `append` the encrypted
>> message to `nonce[:]` and that encrypted message will be 16 bytes longer
>> than the message, which is 11 bytes long. Appending 37 (16+11) bytes to a
>> 24 byte nonce gives you 51 bytes, which is what you observe as the length
>> of `encrypted`.
>> The length of `nonce` doesn't change (it's an array, after all) - but
>> passing `append` to a slice does not change the length of the slice, it
>> just returns a new slice, so that seems expected.
>>
>> So, from what I can tell, the code does exactly what the docs say it
>> should do.
>>
>> > In their example code the out parameter is nil.  So what does it do?
>>
>> Appending to `nil` allocates a new slice. The point of still accepting an
>> `out` parameter is that you can potentially prevent an allocation by
>> passing in a slice with 0 length and extra capacity (which can then be
>> allocated on the stack, or which is from a `sync.Pool`). If you don't need
>> that, passing in `nil` seems fine.
>>
>> > The second argument is encrypted[len(nonce):] which includes the
>> Overhead at the start of the []byte. App

Re: [go-nuts] The docs for secretbox seem very wrong

[Dean Schulze]

That is mixing concerns (low cohesion), though.  At the very least they 
need to explain that in the docs.  Otherwise that first parameter makes no 
sense.

On Monday, October 9, 2023 at 2:51:11 PM UTC-6 David Anderson wrote:

> The first parameter lets you avoid an allocation. If you're constructing a 
> packet that consists of ciphertext surrounded by some framing, you can 
> make([]byte, 0, lengthOfPacket) to preallocate the necessary amount of 
> storage upfront. Then, calls to secretbox don't allocate.
>
> If you don't need that behavior, you can pass in a nil as the first 
> parameter to get a fresh slice allocated for you by the append(), but in 
> hot codepaths involving cryptography, being able to preallocate and reuse 
> storage (e.g. via sync.Pool) can give you significant reduction in GC 
> pressure.
>
> - Dave
>
> On Mon, Oct 9, 2023, at 08:43, Dean Schulze wrote:
>
> So why even bother with the first parameter then?  That looks like a badly 
> designed method signature.
>
> On Monday, October 9, 2023 at 8:12:33 AM UTC-6 Axel Wagner wrote:
>
> I get the impression that the thing you are missing is that appending to a 
> slice does not modify it. That is, `append(s, x...)` modifies neither the 
> length, nor the content of `s`, you have to type `x = append(s, x...)`.
> `Seal` (and `Open`) work exactly the same. They append to the `out` 
> parameter - and to enable you to get at the slice, they return it.
>
> The docs could be slightly more clear, by explicitly stating that they 
> return the appended-to slice.0
>
>
> On Mon, Oct 9, 2023 at 3:46 PM Dean Schulze  wrote:
>
> If the docs are correct, how do you append to nil?  That's what the docs 
> say.  Take a look at them.
>
> Your code example shows the first parameter to Seal() can be nil.  So what 
> does that parameter do?  How do you append to it?
> On Sunday, October 8, 2023 at 11:19:13 PM UTC-6 Axel Wagner wrote:
>
> For what it's worth, here is an example that demonstrates a typical 
> encryption/decryption roundtrip, perhaps more clearly:
> https://go.dev/play/p/ZZry8IgTJQ_-
> The `out` parameter can be used to make this more efficient by using 
> pre-allocated buffers (depending on use case) and there are cases where you 
> don't have to send the nonce, because you can derive them from common data, 
> which is why both of these parameters are there. But in the most common 
> usage, you'd do what this example does.
>
> The example code from the docs tries to be a little bit more efficient and 
> packs the `Box` struct into a single byte, perhaps at the cost of 
> understandability.
>
> On Mon, Oct 9, 2023 at 7:06 AM Axel Wagner  
> wrote:
>
> oh I forgot to emphasize: I don't believe the output is *really* 
> ``. That is, I don't believe you can really treat 
> the first N bytes as the encrypted text and decrypt it (say, if you didn't 
> care about the authentication). It's just that you ultimately need to add 
> 16 bytes of extra information to carry that authentication tag, which is 
> why the box needs to be 16 bytes longer than the message. In reality, the 
> two are probably cleverly mixed - I'm not a cryptographer.
> I just wanted to demonstrate where all the information ultimately goes.
>
> On Mon, Oct 9, 2023 at 7:03 AM Axel Wagner  
> wrote:
>
> I don't really understand your issue. You call
> encrypted := secretbox.Seal(nonce[:], []byte(s), &nonce, &secretKey)
> That means you pass `nonce[:]` as the `out` argument, `s` as the `message` 
> argument, and the nonce and key and assign the result to `encrypted`.
> According to the docs of `secretbox`, `Seal` will `append` the encrypted 
> message to `nonce[:]` and that encrypted message will be 16 bytes longer 
> than the message, which is 11 bytes long. Appending 37 (16+11) bytes to a 
> 24 byte nonce gives you 51 bytes, which is what you observe as the length 
> of `encrypted`.
> The length of `nonce` doesn't change (it's an array, after all) - but 
> passing `append` to a slice does not change the length of the slice, it 
> just returns a new slice, so that seems expected.
>
> So, from what I can tell, the code does exactly what the docs say it 
> should do.
>
> > In their example code the out parameter is nil.  So what does it do?
>
> Appending to `nil` allocates a new slice. The point of still accepting an 
> `out` parameter is that you can potentially prevent an allocation by 
> passing in a slice with 0 length and extra capacity (which can then be 
> allocated on the stack, or which is from a `sync.Pool`). If you don't need 
> that, passing in `nil` seems fine.
>
> > The second argument is encrypted[len(nonce):] which includes the 
> Overhead at the start of the []byte. Apparently that Overhead is important.
>
> Yes, the Overhead is important. It is used to authenticate the message. 
> You can imagine the process of `Seal` as "encrypt the message and attach a 
> hash". The hash is the Overhead. The process also needs a random `nonce`, 
> that both the sender and the receive

Re: [go-nuts] The docs for secretbox seem very wrong

[Dean Schulze]

So why even bother with the first parameter then?  That looks like a badly 
designed method signature.

On Monday, October 9, 2023 at 8:12:33 AM UTC-6 Axel Wagner wrote:

> I get the impression that the thing you are missing is that appending to a 
> slice does not modify it. That is, `append(s, x...)` modifies neither the 
> length, nor the content of `s`, you have to type `x = append(s, x...)`.
> `Seal` (and `Open`) work exactly the same. They append to the `out` 
> parameter - and to enable you to get at the slice, they return it.
> The docs could be slightly more clear, by explicitly stating that they 
> return the appended-to slice.0
>
> On Mon, Oct 9, 2023 at 3:46 PM Dean Schulze  wrote:
>
>> If the docs are correct, how do you append to nil?  That's what the docs 
>> say.  Take a look at them.
>>
>> Your code example shows the first parameter to Seal() can be nil.  So 
>> what does that parameter do?  How do you append to it?
>>
>> On Sunday, October 8, 2023 at 11:19:13 PM UTC-6 Axel Wagner wrote:
>>
>>> For what it's worth, here is an example that demonstrates a typical 
>>> encryption/decryption roundtrip, perhaps more clearly:
>>> https://go.dev/play/p/ZZry8IgTJQ_-
>>> The `out` parameter can be used to make this more efficient by using 
>>> pre-allocated buffers (depending on use case) and there are cases where you 
>>> don't have to send the nonce, because you can derive them from common data, 
>>> which is why both of these parameters are there. But in the most common 
>>> usage, you'd do what this example does.
>>>
>>> The example code from the docs tries to be a little bit more efficient 
>>> and packs the `Box` struct into a single byte, perhaps at the cost of 
>>> understandability.
>>>
>>> On Mon, Oct 9, 2023 at 7:06 AM Axel Wagner  
>>> wrote:
>>>
 oh I forgot to emphasize: I don't believe the output is *really* 
 ``. That is, I don't believe you can really treat 
 the first N bytes as the encrypted text and decrypt it (say, if you didn't 
 care about the authentication). It's just that you ultimately need to add 
 16 bytes of extra information to carry that authentication tag, which is 
 why the box needs to be 16 bytes longer than the message. In reality, the 
 two are probably cleverly mixed - I'm not a cryptographer.
 I just wanted to demonstrate where all the information ultimately goes.

 On Mon, Oct 9, 2023 at 7:03 AM Axel Wagner  
 wrote:

> I don't really understand your issue. You call
> encrypted := secretbox.Seal(nonce[:], []byte(s), &nonce, &secretKey)
> That means you pass `nonce[:]` as the `out` argument, `s` as the 
> `message` argument, and the nonce and key and assign the result to 
> `encrypted`.
> According to the docs of `secretbox`, `Seal` will `append` the 
> encrypted message to `nonce[:]` and that encrypted message will be 16 
> bytes 
> longer than the message, which is 11 bytes long. Appending 37 (16+11) 
> bytes 
> to a 24 byte nonce gives you 51 bytes, which is what you observe as the 
> length of `encrypted`.
> The length of `nonce` doesn't change (it's an array, after all) - but 
> passing `append` to a slice does not change the length of the slice, it 
> just returns a new slice, so that seems expected.
>
> So, from what I can tell, the code does exactly what the docs say it 
> should do.
>
> > In their example code the out parameter is nil.  So what does it do?
>
> Appending to `nil` allocates a new slice. The point of still accepting 
> an `out` parameter is that you can potentially prevent an allocation by 
> passing in a slice with 0 length and extra capacity (which can then be 
> allocated on the stack, or which is from a `sync.Pool`). If you don't 
> need 
> that, passing in `nil` seems fine.
>
> > The second argument is encrypted[len(nonce):] which includes the 
> Overhead at the start of the []byte. Apparently that Overhead is 
> important.
>
> Yes, the Overhead is important. It is used to authenticate the 
> message. You can imagine the process of `Seal` as "encrypt the message 
> and 
> attach a hash". The hash is the Overhead. The process also needs a random 
> `nonce`, that both the sender and the receiver need to know. That's why 
> the 
> example code sends it along with the message (it doesn't have to be 
> secret). So that `Seal` call does, effectively (again, for illustrative 
> purposes):
> encrypted := append(append(nonce, ), )
> As `nonce` is an array, this allocates a new backing array for the 
> returned slice, which ends up filled with
> 
>
> The `Open` call then retrieves the `nonce` from the first 24 bytes (by 
> copying it into `decryptNonce`) and passes the `` 
> slice as the `box` argument. Which decrypts the message, authenticates 
> the 
> hash and appends the decrypted message t

Re: [go-nuts] The docs for secretbox seem very wrong

['Axel Wagner' via golang-nuts]

I get the impression that the thing you are missing is that appending to a
slice does not modify it. That is, `append(s, x...)` modifies neither the
length, nor the content of `s`, you have to type `x = append(s, x...)`.
`Seal` (and `Open`) work exactly the same. They append to the `out`
parameter - and to enable you to get at the slice, they return it.
The docs could be slightly more clear, by explicitly stating that they
return the appended-to slice.0

On Mon, Oct 9, 2023 at 3:46 PM Dean Schulze 
wrote:

> If the docs are correct, how do you append to nil?  That's what the docs
> say.  Take a look at them.
>
> Your code example shows the first parameter to Seal() can be nil.  So what
> does that parameter do?  How do you append to it?
>
> On Sunday, October 8, 2023 at 11:19:13 PM UTC-6 Axel Wagner wrote:
>
>> For what it's worth, here is an example that demonstrates a typical
>> encryption/decryption roundtrip, perhaps more clearly:
>> https://go.dev/play/p/ZZry8IgTJQ_-
>> The `out` parameter can be used to make this more efficient by using
>> pre-allocated buffers (depending on use case) and there are cases where you
>> don't have to send the nonce, because you can derive them from common data,
>> which is why both of these parameters are there. But in the most common
>> usage, you'd do what this example does.
>>
>> The example code from the docs tries to be a little bit more efficient
>> and packs the `Box` struct into a single byte, perhaps at the cost of
>> understandability.
>>
>> On Mon, Oct 9, 2023 at 7:06 AM Axel Wagner 
>> wrote:
>>
>>> oh I forgot to emphasize: I don't believe the output is *really*
>>> ``. That is, I don't believe you can really treat
>>> the first N bytes as the encrypted text and decrypt it (say, if you didn't
>>> care about the authentication). It's just that you ultimately need to add
>>> 16 bytes of extra information to carry that authentication tag, which is
>>> why the box needs to be 16 bytes longer than the message. In reality, the
>>> two are probably cleverly mixed - I'm not a cryptographer.
>>> I just wanted to demonstrate where all the information ultimately goes.
>>>
>>> On Mon, Oct 9, 2023 at 7:03 AM Axel Wagner 
>>> wrote:
>>>
 I don't really understand your issue. You call
 encrypted := secretbox.Seal(nonce[:], []byte(s), &nonce, &secretKey)
 That means you pass `nonce[:]` as the `out` argument, `s` as the
 `message` argument, and the nonce and key and assign the result to
 `encrypted`.
 According to the docs of `secretbox`, `Seal` will `append` the
 encrypted message to `nonce[:]` and that encrypted message will be 16 bytes
 longer than the message, which is 11 bytes long. Appending 37 (16+11) bytes
 to a 24 byte nonce gives you 51 bytes, which is what you observe as the
 length of `encrypted`.
 The length of `nonce` doesn't change (it's an array, after all) - but
 passing `append` to a slice does not change the length of the slice, it
 just returns a new slice, so that seems expected.

 So, from what I can tell, the code does exactly what the docs say it
 should do.

 > In their example code the out parameter is nil.  So what does it do?

 Appending to `nil` allocates a new slice. The point of still accepting
 an `out` parameter is that you can potentially prevent an allocation by
 passing in a slice with 0 length and extra capacity (which can then be
 allocated on the stack, or which is from a `sync.Pool`). If you don't need
 that, passing in `nil` seems fine.

 > The second argument is encrypted[len(nonce):] which includes the
 Overhead at the start of the []byte. Apparently that Overhead is important.

 Yes, the Overhead is important. It is used to authenticate the message.
 You can imagine the process of `Seal` as "encrypt the message and attach a
 hash". The hash is the Overhead. The process also needs a random `nonce`,
 that both the sender and the receiver need to know. That's why the example
 code sends it along with the message (it doesn't have to be secret). So
 that `Seal` call does, effectively (again, for illustrative purposes):
 encrypted := append(append(nonce, ), )
 As `nonce` is an array, this allocates a new backing array for the
 returned slice, which ends up filled with
 

 The `Open` call then retrieves the `nonce` from the first 24 bytes (by
 copying it into `decryptNonce`) and passes the ``
 slice as the `box` argument. Which decrypts the message, authenticates the
 hash and appends the decrypted message to `out` (which is `nil` in the
 example code).

 So, the docs are correct. And it seems to me, the code works as
 expected. I'm not sure where the misunderstanding is.

>>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send 

Re: [go-nuts] The docs for secretbox seem very wrong

[Jan Mercl]

On Mon, Oct 9, 2023 at 3:46 PM Dean Schulze  wrote:

> If the docs are correct, how do you append to nil?

https://go.dev/play/p/WY0Bycj-_Tn

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA40n-WtCRNSp8eLQxDfhjhKBhR4UPdciNPNaS9Oj7v1EdBD-Q%40mail.gmail.com.

Re: [go-nuts] The docs for secretbox seem very wrong

[Bruno Albuquerque]

Now including the list and hopefully with less typos. :)

append returns a new slice. Appending to nil just means that you are
guaranteed that the returned slice will be allocated inside the append
function. The same happens if you try to append to a slice that does not
have enough capacity to hold the appended items (i.e. append will create a
new larger one, copy the existing data, add the data to be appended and
return the new slice).

-Bruno

On Mon, Oct 9, 2023 at 9:46 AM Dean Schulze 
wrote:

> If the docs are correct, how do you append to nil?  That's what the docs
> say.  Take a look at them.
>
> Your code example shows the first parameter to Seal() can be nil.  So what
> does that parameter do?  How do you append to it?
>
> On Sunday, October 8, 2023 at 11:19:13 PM UTC-6 Axel Wagner wrote:
>
>> For what it's worth, here is an example that demonstrates a typical
>> encryption/decryption roundtrip, perhaps more clearly:
>> https://go.dev/play/p/ZZry8IgTJQ_-
>> The `out` parameter can be used to make this more efficient by using
>> pre-allocated buffers (depending on use case) and there are cases where you
>> don't have to send the nonce, because you can derive them from common data,
>> which is why both of these parameters are there. But in the most common
>> usage, you'd do what this example does.
>>
>> The example code from the docs tries to be a little bit more efficient
>> and packs the `Box` struct into a single byte, perhaps at the cost of
>> understandability.
>>
>> On Mon, Oct 9, 2023 at 7:06 AM Axel Wagner 
>> wrote:
>>
>>> oh I forgot to emphasize: I don't believe the output is *really*
>>> ``. That is, I don't believe you can really treat
>>> the first N bytes as the encrypted text and decrypt it (say, if you didn't
>>> care about the authentication). It's just that you ultimately need to add
>>> 16 bytes of extra information to carry that authentication tag, which is
>>> why the box needs to be 16 bytes longer than the message. In reality, the
>>> two are probably cleverly mixed - I'm not a cryptographer.
>>> I just wanted to demonstrate where all the information ultimately goes.
>>>
>>> On Mon, Oct 9, 2023 at 7:03 AM Axel Wagner 
>>> wrote:
>>>
 I don't really understand your issue. You call
 encrypted := secretbox.Seal(nonce[:], []byte(s), &nonce, &secretKey)
 That means you pass `nonce[:]` as the `out` argument, `s` as the
 `message` argument, and the nonce and key and assign the result to
 `encrypted`.
 According to the docs of `secretbox`, `Seal` will `append` the
 encrypted message to `nonce[:]` and that encrypted message will be 16 bytes
 longer than the message, which is 11 bytes long. Appending 37 (16+11) bytes
 to a 24 byte nonce gives you 51 bytes, which is what you observe as the
 length of `encrypted`.
 The length of `nonce` doesn't change (it's an array, after all) - but
 passing `append` to a slice does not change the length of the slice, it
 just returns a new slice, so that seems expected.

 So, from what I can tell, the code does exactly what the docs say it
 should do.

 > In their example code the out parameter is nil.  So what does it do?

 Appending to `nil` allocates a new slice. The point of still accepting
 an `out` parameter is that you can potentially prevent an allocation by
 passing in a slice with 0 length and extra capacity (which can then be
 allocated on the stack, or which is from a `sync.Pool`). If you don't need
 that, passing in `nil` seems fine.

 > The second argument is encrypted[len(nonce):] which includes the
 Overhead at the start of the []byte. Apparently that Overhead is important.

 Yes, the Overhead is important. It is used to authenticate the message.
 You can imagine the process of `Seal` as "encrypt the message and attach a
 hash". The hash is the Overhead. The process also needs a random `nonce`,
 that both the sender and the receiver need to know. That's why the example
 code sends it along with the message (it doesn't have to be secret). So
 that `Seal` call does, effectively (again, for illustrative purposes):
 encrypted := append(append(nonce, ), )
 As `nonce` is an array, this allocates a new backing array for the
 returned slice, which ends up filled with
 

 The `Open` call then retrieves the `nonce` from the first 24 bytes (by
 copying it into `decryptNonce`) and passes the ``
 slice as the `box` argument. Which decrypts the message, authenticates the
 hash and appends the decrypted message to `out` (which is `nil` in the
 example code).

 So, the docs are correct. And it seems to me, the code works as
 expected. I'm not sure where the misunderstanding is.

>>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> emai

Re: [go-nuts] The docs for secretbox seem very wrong

[Dean Schulze]

If the docs are correct, how do you append to nil?  That's what the docs 
say.  Take a look at them.

Your code example shows the first parameter to Seal() can be nil.  So what 
does that parameter do?  How do you append to it?

On Sunday, October 8, 2023 at 11:19:13 PM UTC-6 Axel Wagner wrote:

> For what it's worth, here is an example that demonstrates a typical 
> encryption/decryption roundtrip, perhaps more clearly:
> https://go.dev/play/p/ZZry8IgTJQ_-
> The `out` parameter can be used to make this more efficient by using 
> pre-allocated buffers (depending on use case) and there are cases where you 
> don't have to send the nonce, because you can derive them from common data, 
> which is why both of these parameters are there. But in the most common 
> usage, you'd do what this example does.
>
> The example code from the docs tries to be a little bit more efficient and 
> packs the `Box` struct into a single byte, perhaps at the cost of 
> understandability.
>
> On Mon, Oct 9, 2023 at 7:06 AM Axel Wagner  
> wrote:
>
>> oh I forgot to emphasize: I don't believe the output is *really* 
>> ``. That is, I don't believe you can really treat 
>> the first N bytes as the encrypted text and decrypt it (say, if you didn't 
>> care about the authentication). It's just that you ultimately need to add 
>> 16 bytes of extra information to carry that authentication tag, which is 
>> why the box needs to be 16 bytes longer than the message. In reality, the 
>> two are probably cleverly mixed - I'm not a cryptographer.
>> I just wanted to demonstrate where all the information ultimately goes.
>>
>> On Mon, Oct 9, 2023 at 7:03 AM Axel Wagner  
>> wrote:
>>
>>> I don't really understand your issue. You call
>>> encrypted := secretbox.Seal(nonce[:], []byte(s), &nonce, &secretKey)
>>> That means you pass `nonce[:]` as the `out` argument, `s` as the 
>>> `message` argument, and the nonce and key and assign the result to 
>>> `encrypted`.
>>> According to the docs of `secretbox`, `Seal` will `append` the encrypted 
>>> message to `nonce[:]` and that encrypted message will be 16 bytes longer 
>>> than the message, which is 11 bytes long. Appending 37 (16+11) bytes to a 
>>> 24 byte nonce gives you 51 bytes, which is what you observe as the length 
>>> of `encrypted`.
>>> The length of `nonce` doesn't change (it's an array, after all) - but 
>>> passing `append` to a slice does not change the length of the slice, it 
>>> just returns a new slice, so that seems expected.
>>>
>>> So, from what I can tell, the code does exactly what the docs say it 
>>> should do.
>>>
>>> > In their example code the out parameter is nil.  So what does it do?
>>>
>>> Appending to `nil` allocates a new slice. The point of still accepting 
>>> an `out` parameter is that you can potentially prevent an allocation by 
>>> passing in a slice with 0 length and extra capacity (which can then be 
>>> allocated on the stack, or which is from a `sync.Pool`). If you don't need 
>>> that, passing in `nil` seems fine.
>>>
>>> > The second argument is encrypted[len(nonce):] which includes the 
>>> Overhead at the start of the []byte. Apparently that Overhead is important.
>>>
>>> Yes, the Overhead is important. It is used to authenticate the message. 
>>> You can imagine the process of `Seal` as "encrypt the message and attach a 
>>> hash". The hash is the Overhead. The process also needs a random `nonce`, 
>>> that both the sender and the receiver need to know. That's why the example 
>>> code sends it along with the message (it doesn't have to be secret). So 
>>> that `Seal` call does, effectively (again, for illustrative purposes):
>>> encrypted := append(append(nonce, ), )
>>> As `nonce` is an array, this allocates a new backing array for the 
>>> returned slice, which ends up filled with
>>> 
>>>
>>> The `Open` call then retrieves the `nonce` from the first 24 bytes (by 
>>> copying it into `decryptNonce`) and passes the `` 
>>> slice as the `box` argument. Which decrypts the message, authenticates the 
>>> hash and appends the decrypted message to `out` (which is `nil` in the 
>>> example code).
>>>
>>> So, the docs are correct. And it seems to me, the code works as 
>>> expected. I'm not sure where the misunderstanding is.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3d7d75e5-d79a-4f81-978d-cb6feb7b12efn%40googlegroups.com.

Re: [go-nuts] The docs for secretbox seem very wrong

['Axel Wagner' via golang-nuts]

For what it's worth, here is an example that demonstrates a typical
encryption/decryption roundtrip, perhaps more clearly:
https://go.dev/play/p/ZZry8IgTJQ_-
The `out` parameter can be used to make this more efficient by using
pre-allocated buffers (depending on use case) and there are cases where you
don't have to send the nonce, because you can derive them from common data,
which is why both of these parameters are there. But in the most common
usage, you'd do what this example does.

The example code from the docs tries to be a little bit more efficient and
packs the `Box` struct into a single byte, perhaps at the cost of
understandability.

On Mon, Oct 9, 2023 at 7:06 AM Axel Wagner 
wrote:

> oh I forgot to emphasize: I don't believe the output is *really*
> ``. That is, I don't believe you can really treat
> the first N bytes as the encrypted text and decrypt it (say, if you didn't
> care about the authentication). It's just that you ultimately need to add
> 16 bytes of extra information to carry that authentication tag, which is
> why the box needs to be 16 bytes longer than the message. In reality, the
> two are probably cleverly mixed - I'm not a cryptographer.
> I just wanted to demonstrate where all the information ultimately goes.
>
> On Mon, Oct 9, 2023 at 7:03 AM Axel Wagner 
> wrote:
>
>> I don't really understand your issue. You call
>> encrypted := secretbox.Seal(nonce[:], []byte(s), &nonce, &secretKey)
>> That means you pass `nonce[:]` as the `out` argument, `s` as the
>> `message` argument, and the nonce and key and assign the result to
>> `encrypted`.
>> According to the docs of `secretbox`, `Seal` will `append` the encrypted
>> message to `nonce[:]` and that encrypted message will be 16 bytes longer
>> than the message, which is 11 bytes long. Appending 37 (16+11) bytes to a
>> 24 byte nonce gives you 51 bytes, which is what you observe as the length
>> of `encrypted`.
>> The length of `nonce` doesn't change (it's an array, after all) - but
>> passing `append` to a slice does not change the length of the slice, it
>> just returns a new slice, so that seems expected.
>>
>> So, from what I can tell, the code does exactly what the docs say it
>> should do.
>>
>> > In their example code the out parameter is nil.  So what does it do?
>>
>> Appending to `nil` allocates a new slice. The point of still accepting an
>> `out` parameter is that you can potentially prevent an allocation by
>> passing in a slice with 0 length and extra capacity (which can then be
>> allocated on the stack, or which is from a `sync.Pool`). If you don't need
>> that, passing in `nil` seems fine.
>>
>> > The second argument is encrypted[len(nonce):] which includes the
>> Overhead at the start of the []byte. Apparently that Overhead is important.
>>
>> Yes, the Overhead is important. It is used to authenticate the message.
>> You can imagine the process of `Seal` as "encrypt the message and attach a
>> hash". The hash is the Overhead. The process also needs a random `nonce`,
>> that both the sender and the receiver need to know. That's why the example
>> code sends it along with the message (it doesn't have to be secret). So
>> that `Seal` call does, effectively (again, for illustrative purposes):
>> encrypted := append(append(nonce, ), )
>> As `nonce` is an array, this allocates a new backing array for the
>> returned slice, which ends up filled with
>> 
>>
>> The `Open` call then retrieves the `nonce` from the first 24 bytes (by
>> copying it into `decryptNonce`) and passes the ``
>> slice as the `box` argument. Which decrypts the message, authenticates the
>> hash and appends the decrypted message to `out` (which is `nil` in the
>> example code).
>>
>> So, the docs are correct. And it seems to me, the code works as expected.
>> I'm not sure where the misunderstanding is.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAEkBMfG5VU40UqAU%2B-PYT3Jn63g3nY92mof0o%2BrcRCLHA72iYQ%40mail.gmail.com.

Re: [go-nuts] The docs for secretbox seem very wrong

['Axel Wagner' via golang-nuts]

oh I forgot to emphasize: I don't believe the output is *really*
``. That is, I don't believe you can really treat
the first N bytes as the encrypted text and decrypt it (say, if you didn't
care about the authentication). It's just that you ultimately need to add
16 bytes of extra information to carry that authentication tag, which is
why the box needs to be 16 bytes longer than the message. In reality, the
two are probably cleverly mixed - I'm not a cryptographer.
I just wanted to demonstrate where all the information ultimately goes.

On Mon, Oct 9, 2023 at 7:03 AM Axel Wagner 
wrote:

> I don't really understand your issue. You call
> encrypted := secretbox.Seal(nonce[:], []byte(s), &nonce, &secretKey)
> That means you pass `nonce[:]` as the `out` argument, `s` as the `message`
> argument, and the nonce and key and assign the result to `encrypted`.
> According to the docs of `secretbox`, `Seal` will `append` the encrypted
> message to `nonce[:]` and that encrypted message will be 16 bytes longer
> than the message, which is 11 bytes long. Appending 37 (16+11) bytes to a
> 24 byte nonce gives you 51 bytes, which is what you observe as the length
> of `encrypted`.
> The length of `nonce` doesn't change (it's an array, after all) - but
> passing `append` to a slice does not change the length of the slice, it
> just returns a new slice, so that seems expected.
>
> So, from what I can tell, the code does exactly what the docs say it
> should do.
>
> > In their example code the out parameter is nil.  So what does it do?
>
> Appending to `nil` allocates a new slice. The point of still accepting an
> `out` parameter is that you can potentially prevent an allocation by
> passing in a slice with 0 length and extra capacity (which can then be
> allocated on the stack, or which is from a `sync.Pool`). If you don't need
> that, passing in `nil` seems fine.
>
> > The second argument is encrypted[len(nonce):] which includes the
> Overhead at the start of the []byte. Apparently that Overhead is important.
>
> Yes, the Overhead is important. It is used to authenticate the message.
> You can imagine the process of `Seal` as "encrypt the message and attach a
> hash". The hash is the Overhead. The process also needs a random `nonce`,
> that both the sender and the receiver need to know. That's why the example
> code sends it along with the message (it doesn't have to be secret). So
> that `Seal` call does, effectively (again, for illustrative purposes):
> encrypted := append(append(nonce, ), )
> As `nonce` is an array, this allocates a new backing array for the
> returned slice, which ends up filled with
> 
>
> The `Open` call then retrieves the `nonce` from the first 24 bytes (by
> copying it into `decryptNonce`) and passes the ``
> slice as the `box` argument. Which decrypts the message, authenticates the
> hash and appends the decrypted message to `out` (which is `nil` in the
> example code).
>
> So, the docs are correct. And it seems to me, the code works as expected.
> I'm not sure where the misunderstanding is.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAEkBMfFyqCjRXbtr1ciDV3qjeDw3Ae%2BGm5VRRZSzrbUsqeKAKA%40mail.gmail.com.

Re: [go-nuts] The docs for secretbox seem very wrong

['Axel Wagner' via golang-nuts]

I don't really understand your issue. You call
encrypted := secretbox.Seal(nonce[:], []byte(s), &nonce, &secretKey)
That means you pass `nonce[:]` as the `out` argument, `s` as the `message`
argument, and the nonce and key and assign the result to `encrypted`.
According to the docs of `secretbox`, `Seal` will `append` the encrypted
message to `nonce[:]` and that encrypted message will be 16 bytes longer
than the message, which is 11 bytes long. Appending 37 (16+11) bytes to a
24 byte nonce gives you 51 bytes, which is what you observe as the length
of `encrypted`.
The length of `nonce` doesn't change (it's an array, after all) - but
passing `append` to a slice does not change the length of the slice, it
just returns a new slice, so that seems expected.

So, from what I can tell, the code does exactly what the docs say it should
do.

> In their example code the out parameter is nil.  So what does it do?

Appending to `nil` allocates a new slice. The point of still accepting an
`out` parameter is that you can potentially prevent an allocation by
passing in a slice with 0 length and extra capacity (which can then be
allocated on the stack, or which is from a `sync.Pool`). If you don't need
that, passing in `nil` seems fine.

> The second argument is encrypted[len(nonce):] which includes the Overhead
at the start of the []byte. Apparently that Overhead is important.

Yes, the Overhead is important. It is used to authenticate the message. You
can imagine the process of `Seal` as "encrypt the message and attach a
hash". The hash is the Overhead. The process also needs a random `nonce`,
that both the sender and the receiver need to know. That's why the example
code sends it along with the message (it doesn't have to be secret). So
that `Seal` call does, effectively (again, for illustrative purposes):
encrypted := append(append(nonce, ), )
As `nonce` is an array, this allocates a new backing array for the returned
slice, which ends up filled with


The `Open` call then retrieves the `nonce` from the first 24 bytes (by
copying it into `decryptNonce`) and passes the ``
slice as the `box` argument. Which decrypts the message, authenticates the
hash and appends the decrypted message to `out` (which is `nil` in the
example code).

So, the docs are correct. And it seems to me, the code works as expected.
I'm not sure where the misunderstanding is.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAEkBMfEy_%3Dd5PpOwn7xSUWnZU3mqva8_%2Bf%3D%2BAcMMisegaLqdxA%40mail.gmail.com.

[go-nuts] The docs for secretbox seem very wrong

[Dean Schulze]

The docs for secretbox.Seal 
 say:

func Seal(out, message []byte, nonce *[24]byte, key *[32]byte) []byte

Seal appends an encrypted and authenticated copy of message to out, which 
must not overlap message. The key and nonce pair must be unique for each 
distinct message and the output will be Overhead bytes longer than message.

Using their example code below I find that 

the first argument out is unchanged
the return value is len(nonce) + Overhead longer than the input message, 
not Overhead longer

It's not clear what the first argument to Seal does.  Nothing is appended 
to it and it is not changed, but it has to be there.  Does it have to be 
the nonce[:]?


Likewise the docs for Open say

func Open(out, box []byte, nonce *[24]byte, key *[32]byte) ([]byte, bool)

Open authenticates and decrypts a box produced by Seal and appends the 
message to out, which must not overlap box. The output will be Overhead 
bytes smaller than box.

In their example code the out parameter is nil.  So what does it do?  The 
second argument is encrypted[len(nonce):] which includes the Overhead at 
the start of the []byte.  Apparently that Overhead is important.

The docs seem wildly wrong.

Here's code based on their example:

func main() {

secretKeyBytes, err := 
hex.DecodeString("6368616e676520746869732070617373776f726420746f206120736563726574")
if err != nil {
panic(err)
}

var secretKey [32]byte
copy(secretKey[:], secretKeyBytes)

var nonce [24]byte
if _, err := io.ReadFull(rand.Reader, nonce[:]); err != nil {
panic(err)
}

nonceOrig := nonce
fmt.Printf("len(nonce):  %v\n", len(nonce))
s := "hello world"
encrypted := secretbox.Seal(nonce[:], []byte(s), &nonce, &secretKey)
fmt.Printf("len(nonce):  %v\n", len(nonce))
fmt.Printf("len(s): %v\n", len(s))
fmt.Printf("len(encrypted): %v\n", len(encrypted))
if !reflect.DeepEqual(nonceOrig, nonce) {
fmt.Println("nonce changed")
}

var decryptNonce [24]byte
copy(decryptNonce[:], encrypted[:24])

if !reflect.DeepEqual(decryptNonce, nonce) {
fmt.Println("decryptNonce, nonce differ")
}

decrypted, ok := secretbox.Open(nil, encrypted[24:], &decryptNonce, 
&secretKey)
if !ok {
panic("decryption error")
}

fmt.Println(string(decrypted))

encrypted2 := secretbox.Seal([]byte{}, []byte(s), &nonce, &secretKey)
copy(decryptNonce[:], encrypted2[:24])
decrypted2, ok := secretbox.Open(nil, encrypted2[24:], &decryptNonce, 
&secretKey)
if !ok {
fmt.Printf("decryption error 2: %v\n", decrypted2)
//panic("decryption error 2")
}
fmt.Printf("decrypted2: %v\n", decrypted2)

encrypted3 := secretbox.Seal(nil, []byte(s), &nonce, &secretKey)
decrypted3, ok := secretbox.Open(nil, encrypted3[24:], &nonce, &secretKey)
if !ok {
fmt.Printf("decryption error 3: %v\n", decrypted3)
}
fmt.Printf("decrypted3: %v\n", decrypted3)
}


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5d31760e-59ec-4c4e-af11-c4826ae5560cn%40googlegroups.com.