TL;DR: Is there any way I could stream data to a huge json file in small chunks, but adding/appending said data to one field only?
I mean, this json file represents one single pojo object, and I need to keep appending data to one of its fields. Thanks! (sent by smartphone / enviado pelo celular) On Wed, 15 Jun 2022, 22:20 Rodrigo Ottero, <[email protected]> wrote: > Hi folks, > > Apologies in advance if I am not clear enough, happy to provide > clarifications if required. > > I am having a hard time finding information about how to use > SequenceWriter to output a custom POJO to a JSON file in an incremental way. > > I found Bozhidar's solution to incrementally output data as a JSON array ( > https://dzone.com/articles/writing-big-json-files-with-jackson); that is > natively supported by SequenceWriter through the init(true) method. > > In my use case, we have a service that receives a SQL query and outputs it > as a CSV file. > > Since some queries may produce millions of rows, this CSV is written > incrementally as the result set is read, line by line, to prevent out of > memory issues. > > I was asked to add the option of outputting a JSON file instead of a CSV > file, but in the same incremental way. > > The gotcha is that I need to use a legacy JSON structure, something like: > > { > "metadata": { > "query": "SELECT * FROM ZZZ", > "processingTime": 100, > }, > "result": { > "columns": [ > "Column1", > "Column2", > "..." > ], > "result": [ > > > > > > > > > > > *[ "abc", "def", "..." > ], [ "foo", "bar", > "..." ], ...millions of lines...* > ] > } > } > > The part in red is the part I want to incrementally write/update on the > JSON file, while keeping the blue parts fixed. > > I was browsing the javadocs for SequenceWriter ( > https://developer.adobe.com/experience-manager/reference-materials/6-5/javadoc/com/fasterxml/jackson/databind/SequenceWriter.html) > and saw that the constructor accepts a DefaultSerializerProvider and a > JsonGenerator object, but I am not sure how I would use these resources to > achieve my objective. > > Would you have a piece of documentation or an example of how I could use > SequenceWriter to incrementally write the JSON file I need, based on a > custom POJO? > > Or could you please tell me if I should be looking for another approach, > in case SequenceWriter is not the answer for my use case? > > Many thanks, > Best regards, > Rodrigo > > -- > You received this message because you are subscribed to a topic in the > Google Groups "jackson-user" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/jackson-user/kXvRr6b7rg4/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jackson-user/01796d7e-5b37-4180-94ea-a7be908d4efan%40googlegroups.com > <https://groups.google.com/d/msgid/jackson-user/01796d7e-5b37-4180-94ea-a7be908d4efan%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "jackson-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/CAOmZcH%2BxxXQoSTw7cTMsnZ1eHtcQtmuMgbkYDxkWjYMHeoeH_Q%40mail.gmail.com.
