Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-10 Thread Jonathan Hall
You're absolutely right. I just meant that the tokenizer interface wouldn't be completely replaced. There are still some corner cases where it will be necessary. On Friday, August 9, 2019 at 11:41:17 PM UTC+2, burak serdar wrote: > > On Fri, Aug 9, 2019 at 3:32 PM Jonathan Hall > wrote: > >

Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-09 Thread burak serdar
On Fri, Aug 9, 2019 at 3:32 PM Jonathan Hall wrote: > > I think you're right that most people are frustrated by the encoder, but as I > mentioned in another message just a bit ago, the same fundamental problem > exists with decoder, and for certain workloads, I believe it should be solved. > >

Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-09 Thread Jonathan Hall
I think you're right that most people are frustrated by the encoder, but as I mentioned in another message just a bit ago, the same fundamental problem exists with decoder, and for certain workloads, I believe it should be solved. Having said that, I think tackling the writer first definitely

Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-09 Thread Jonathan Hall
Oh, thanks for pointing that out. it is indeed very similar to my proposal. What do you think the chances of getting it resurrected and merged? Is more discussion still needed with respect to sync.Pool? On Friday, August 9, 2019 at 6:15:31 PM UTC+2, Ian Davis wrote: > > You may also be

Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-09 Thread Jonathan Hall
The problem is that when encoding, even with json.Encoder, the entire object is marshaled, into memory, before it is written to the io.Writer. My proposal allows writing the JSON output immediately, rather than waiting for the entire process to complete successfully first. The same problem

Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-09 Thread burak serdar
On Fri, Aug 9, 2019 at 2:10 PM Robert Engels wrote: > > I'm sorry, maybe I didn't understand your original concern. There is an > example of doing stream based processing in the json package (using Decoder). > > How is this not sufficient? > > The only problem I see with it is that it doesn't

Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-09 Thread Robert Engels
I'm sorry, maybe I didn't understand your original concern. There is an example of doing stream based processing in the json package (using Decoder).How is this not sufficient?The only problem I see with it is that it doesn't allow error correction/continuation, but in the modern world that seems

Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-09 Thread Ian Davis
On Fri, 9 Aug 2019, at 3:33 PM, Jonathan Hall wrote: > *I debated posting here, or straight to GitHub. If that's the better place, I > can move the thread there. * I have long wanted proper streaming support in > the `encoding/json` library. Lately I’ve been doing some digging to > understand

Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-09 Thread Jonathan Hall
An interesting observation. Although in a sense, we already have the decoding half of that low-level streaming API exposed by way of the `json.Decoder` type. Would you be suggesting that a separate, stream-based API makes sense even within the stdlib? I'm not really sure what that separate

Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-09 Thread Robert Engels
In other environments (e.g. Java), the streaming processors are distinct from the instance oriented - usually for good reason as the APIs are very different - the former is usually event based, and the latter returns realized instances as a whole or an error. The streaming processors can often

Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-09 Thread Jonathan Hall
Thanks for the reply. My responses inline below. On Friday, August 9, 2019 at 5:14:52 PM UTC+2, burak serdar wrote: > > On Fri, Aug 9, 2019 at 8:53 AM Jonathan Hall > wrote: > > > > Can you say more? Better in which way? > > Better in the way that it wouldn't change existing code. That

Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-09 Thread burak serdar
On Fri, Aug 9, 2019 at 8:53 AM Jonathan Hall wrote: > > Can you say more? Better in which way? Better in the way that it wouldn't change existing code. Also, I think the use cases for existing and proposed json encoders/decoders are different enough to justify a separate implementation. A wihle

Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-09 Thread Jonathan Hall
Can you say more? Better in which way? On Friday, August 9, 2019 at 4:46:19 PM UTC+2, burak serdar wrote: > > > Instead of modifying the existing Encoder/Decoder, wouldn't it be > better to do this as a separate encoder/decoder? > > > > > > > > > > -- > > You received this message because

Re: [go-nuts] RFC for opt-in streaming support in encoding/json package

2019-08-09 Thread burak serdar
On Fri, Aug 9, 2019 at 8:33 AM Jonathan Hall wrote: > > I debated posting here, or straight to GitHub. If that's the better place, I > can move the thread there. I have long wanted proper streaming support in the > `encoding/json` library. Lately I’ve been doing some digging to understand >