Re: [go-nuts] Json decode : Monitoring errors properly

[thomas]

I got it.
In fact, if the buffer is bad, it is not erased, and Buffered() returns it 
(and not the following).
That was my misunderstanding.

Thank you all !

On Thursday, 27 December 2018 00:31:12 UTC+1, Kevin Conway wrote:
>
> > I believe https://golang.org/pkg/encoding/json/#Decoder.Buffered was 
> added for this purpose.
>
> I just caught on that this method is exactly what you showed in the 
> original message. I guess my input can be reduced to "I think that's your 
> only option when using the decoder".
>
> I do think you've correctly identified they the buffer isn't guaranteed to 
> contain the whole object since the decoder has early exit error conditions. 
> I'm not sure how you'd change that without rewriting the decoder. 
>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Json decode : Monitoring errors properly

[Kevin Conway]

> I believe https://golang.org/pkg/encoding/json/#Decoder.Buffered was
added for this purpose.

I just caught on that this method is exactly what you showed in the
original message. I guess my input can be reduced to "I think that's your
only option when using the decoder".

I do think you've correctly identified they the buffer isn't guaranteed to
contain the whole object since the decoder has early exit error conditions.
I'm not sure how you'd change that without rewriting the decoder.

>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Json decode : Monitoring errors properly

[Kevin Conway]

> For example, the error returned may show : "*invalid character 'G'
looking for beginning of value*". But you can't see the original message.

I believe https://golang.org/pkg/encoding/json/#Decoder.Buffered was added
for this purpose. For example, https://play.golang.org/p/yAn2fypIELc

> 1- I'm not sure the buffer is always complete at the time I call the
ReadAll

ReadAll (https://golang.org/pkg/io/ioutil/#ReadAll) reads until the io.EOF.
If you're using long lived TCP connections on which you expect to have
multiple request/response cycles then you likely don't want to use ReadAll
since the stream won't encounter an EOF until some later point in time. The
Decoder is made for the streaming case is very likely the right choice over
reading things in with other tools.

> 2- It can be time-consuming

I don't know that you're necessarily going to avoid this problem using the
json.Decoder. As it was pointed out, the current implementation of Decoder
continues to read and buffer bytes from the stream until it has enough to
represent a whole object before it decodes it. There may be some early exit
cases from errors in the "decoderState" that is used to manage the internal
buffer but I don't expect they should be relied on for performance.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Json decode : Monitoring errors properly

[Tamás Gulácsi]

You haven't shown the code, so I don't know.
json.Decoder can decode json streams - one object at a time (separated by 
whitespace).

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Json decode : Monitoring errors properly

[Thomas S]

Sorry I don't get it ?

My decoder is already built with it.

Thank you,
Thomas

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Json decode : Monitoring errors properly

[Tamás Gulácsi]

Try json.NewDecoder.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Json decode : Monitoring errors properly

[Thomas S]

Hello,


I need help to improve the reliability of a proxy and numerous servers in 
Golang, currently in use by a certain number of users.

I receive json stream by TCP and websocket.
The only way to do it properly is to used json decode.
(notably because websocket payload system can split in multiple payloads. 
Whatever.) 

Decode function is defined as such :

func (dec *Decoder ) Decode(v 
interface{}) error 


The error specified what is problem encoutered by Decode, *but not the 
original message*.

For example, the error returned may show : "*invalid character 'G' looking 
for beginning of value*".
But you can't see the original message.

Untill now, I found only 1 way to do it. It's to read the buffer before 
reading it.
Example :

*var buff []byte*
*bb, _ = ioutil.ReadAll(reader.Buffered())*
*fmt.Println("Before :", string(bb))*


*var tmp interface{}*

*err := reader.Decode(&tmp) *



2 problems with this :

1- I'm not sure the buffer is always complete at the time I call the ReadAll
2- It can be time-consuming


Am I missing something to handle this better ?


Thanks a lot for your answers.

Thomas









-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.