Re: [akka-user] Re: How to read XML inputstream (with SAX parser) received by akka-http?

[Endre Varga]

Btw, I am also interested in how it fares on a 5G input, so please tell us
how it went.

On Wed, Aug 10, 2016 at 9:02 PM, Endre Varga 
wrote:

> Hi,
>
> Christmas is early for you then, since this is exactly what I am working
> on: https://github.com/drewhk/akka-xml-stream/pull/1
>
> As you see, it is currently on my private repo, but it will be migrated
> and donated soon to the akka-stream-contrib project (and will gain an
> Apache Lincense of course). It is pretty minimal and most of the work is
> actually done by the Aalto XML parser (https://github.com/FasterXML/
> aalto-xml) but you can adapt it to your needs as you wish.
>
> Contributions are welcome, although I will rewrite it in Java to have a
> reference for Java people how to do this stuff (Scala API will stay of
> course).
>
> -Endre
>
> On Wed, Aug 10, 2016 at 5:59 PM, JY  wrote:
>
>> Nobody can help me to read 5GB of upload data with akka-http, and without
>> crashing my VM with OutOfMemoryException ?
>>
>> (I think the topic of this conversation is too specific compared to the
>> real explained problem, but I don't know how to modify the subject)
>>
>> Le mardi 2 août 2016 12:20:28 UTC+2, JY a écrit :
>>>
>>> I forgot to specify that flow I have to analyze weigh between 1 and 5
>>> GB, and I have to extract the value of a dozen tags, which is why we can't
>>> load all data into memory, and that we've choose SAX :)
>>>
>>>
>>>
>>> Le mardi 2 août 2016 11:23:18 UTC+2, JY a écrit :

 In fact, (I will try to be both more specific and more generic):

 I have to read (via InputStream because of SAX parser) the data when
 the first bits are received.

 I'm not able to wait and load all data into memory before to use it.

 I can not yet find the way to do it.

 I need a little help :)



 Le mardi 2 août 2016 10:31:01 UTC+2, Konrad Malawski a écrit :
>
> Yeah, that'd be good to collect usecases and demadn there.
> Though with SAX it won't be automagical ever, they always require
> manually writing the parsing AFAIR
>
> --
> Konrad `ktoso` Malawski
> Akka  @ Lightbend 
>
> On 2 August 2016 at 10:29:21, André (andre.r...@gmail.com) wrote:
>
> Maybe you can add your use case to https://github.com/akka/akk
> a/issues/16210.
>
> On Tuesday, August 2, 2016 at 10:12:13 AM UTC+2, JY wrote:
>>
>> I am new with AKKA and AKKA HTTP.
>>
>> I have to write a HTTP server application that reads a lot very big
>> XML data flows (via HTTP PUT method) and parses this Data with a SAX 
>> parser.
>>
>> How can I do that with akka-http?
>>
>> Thanx in advance for your help.
>>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/c
> urrent/additional/faq.html
> >> Search the archives: https://groups.google.com/grou
> p/akka-user
> ---
> You received this message because you are subscribed to the Google
> Groups "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to akka-user+...@googlegroups.com.
> To post to this group, send email to akka...@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: http://doc.akka.io/docs/akka/c
>> urrent/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+unsubscr...@googlegroups.com.
>> To post to this group, send email to akka-user@googlegroups.com.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: How to read XML inputstream (with SAX parser) received by akka-http?

[Endre Varga]

Hi,

Christmas is early for you then, since this is exactly what I am working
on: https://github.com/drewhk/akka-xml-stream/pull/1

As you see, it is currently on my private repo, but it will be migrated and
donated soon to the akka-stream-contrib project (and will gain an Apache
Lincense of course). It is pretty minimal and most of the work is actually
done by the Aalto XML parser (https://github.com/FasterXML/aalto-xml) but
you can adapt it to your needs as you wish.

Contributions are welcome, although I will rewrite it in Java to have a
reference for Java people how to do this stuff (Scala API will stay of
course).

-Endre

On Wed, Aug 10, 2016 at 5:59 PM, JY  wrote:

> Nobody can help me to read 5GB of upload data with akka-http, and without
> crashing my VM with OutOfMemoryException ?
>
> (I think the topic of this conversation is too specific compared to the
> real explained problem, but I don't know how to modify the subject)
>
> Le mardi 2 août 2016 12:20:28 UTC+2, JY a écrit :
>>
>> I forgot to specify that flow I have to analyze weigh between 1 and 5 GB,
>> and I have to extract the value of a dozen tags, which is why we can't load
>> all data into memory, and that we've choose SAX :)
>>
>>
>>
>> Le mardi 2 août 2016 11:23:18 UTC+2, JY a écrit :
>>>
>>> In fact, (I will try to be both more specific and more generic):
>>>
>>> I have to read (via InputStream because of SAX parser) the data when the
>>> first bits are received.
>>>
>>> I'm not able to wait and load all data into memory before to use it.
>>>
>>> I can not yet find the way to do it.
>>>
>>> I need a little help :)
>>>
>>>
>>>
>>> Le mardi 2 août 2016 10:31:01 UTC+2, Konrad Malawski a écrit :

 Yeah, that'd be good to collect usecases and demadn there.
 Though with SAX it won't be automagical ever, they always require
 manually writing the parsing AFAIR

 --
 Konrad `ktoso` Malawski
 Akka  @ Lightbend 

 On 2 August 2016 at 10:29:21, André (andre.r...@gmail.com) wrote:

 Maybe you can add your use case to https://github.com/akka/akk
 a/issues/16210.

 On Tuesday, August 2, 2016 at 10:12:13 AM UTC+2, JY wrote:
>
> I am new with AKKA and AKKA HTTP.
>
> I have to write a HTTP server application that reads a lot very big
> XML data flows (via HTTP PUT method) and parses this Data with a SAX 
> parser.
>
> How can I do that with akka-http?
>
> Thanx in advance for your help.
>
 --
 >> Read the docs: http://akka.io/docs/
 >> Check the FAQ: http://doc.akka.io/docs/akka/c
 urrent/additional/faq.html
 >> Search the archives: https://groups.google.com/grou
 p/akka-user
 ---
 You received this message because you are subscribed to the Google
 Groups "Akka User List" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to akka-user+...@googlegroups.com.
 To post to this group, send email to akka...@googlegroups.com.
 Visit this group at https://groups.google.com/group/akka-user.
 For more options, visit https://groups.google.com/d/optout.

 --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: How to read XML inputstream (with SAX parser) received by akka-http?

[Patrik Nordwall]

Take a look at Endre's streaming xml parser, for inspiration:
https://github.com/drewhk/akka-xml-stream/pull/1

ons 10 aug. 2016 kl. 19:57 skrev JY :

> Nobody can help me to read 5GB of upload data with akka-http, and without
> crashing my VM with OutOfMemoryException ?
>
> (I think the topic of this conversation is too specific compared to the
> real explained problem, but I don't know how to modify the subject)
>
> Le mardi 2 août 2016 12:20:28 UTC+2, JY a écrit :
>>
>> I forgot to specify that flow I have to analyze weigh between 1 and 5 GB,
>> and I have to extract the value of a dozen tags, which is why we can't load
>> all data into memory, and that we've choose SAX :)
>>
>>
>>
>> Le mardi 2 août 2016 11:23:18 UTC+2, JY a écrit :
>>>
>>> In fact, (I will try to be both more specific and more generic):
>>>
>>> I have to read (via InputStream because of SAX parser) the data when the
>>> first bits are received.
>>>
>>> I'm not able to wait and load all data into memory before to use it.
>>>
>>> I can not yet find the way to do it.
>>>
>>> I need a little help :)
>>>
>>>
>>>
>>> Le mardi 2 août 2016 10:31:01 UTC+2, Konrad Malawski a écrit :

 Yeah, that'd be good to collect usecases and demadn there.
 Though with SAX it won't be automagical ever, they always require
 manually writing the parsing AFAIR

 --
 Konrad `ktoso` Malawski
 Akka  @ Lightbend 

 On 2 August 2016 at 10:29:21, André (andre.r...@gmail.com) wrote:

 Maybe you can add your use case to
 https://github.com/akka/akka/issues/16210.

 On Tuesday, August 2, 2016 at 10:12:13 AM UTC+2, JY wrote:
>
> I am new with AKKA and AKKA HTTP.
>
> I have to write a HTTP server application that reads a lot very big
> XML data flows (via HTTP PUT method) and parses this Data with a SAX 
> parser.
>
> How can I do that with akka-http?
>
> Thanx in advance for your help.
>
 --
 >> Read the docs: http://akka.io/docs/
 >> Check the FAQ:
 http://doc.akka.io/docs/akka/current/additional/faq.html
 >> Search the archives:
 https://groups.google.com/group/akka-user
 ---
 You received this message because you are subscribed to the Google
 Groups "Akka User List" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to akka-user+...@googlegroups.com.
 To post to this group, send email to akka...@googlegroups.com.
 Visit this group at https://groups.google.com/group/akka-user.
 For more options, visit https://groups.google.com/d/optout.

 --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: How to read XML inputstream (with SAX parser) received by akka-http?

[JY]

Nobody can help me to read 5GB of upload data with akka-http, and without 
crashing my VM with OutOfMemoryException ?

(I think the topic of this conversation is too specific compared to the 
real explained problem, but I don't know how to modify the subject) 

Le mardi 2 août 2016 12:20:28 UTC+2, JY a écrit :
>
> I forgot to specify that flow I have to analyze weigh between 1 and 5 GB, 
> and I have to extract the value of a dozen tags, which is why we can't load 
> all data into memory, and that we've choose SAX :)
>
>
>
> Le mardi 2 août 2016 11:23:18 UTC+2, JY a écrit :
>>
>> In fact, (I will try to be both more specific and more generic):
>>
>> I have to read (via InputStream because of SAX parser) the data when the 
>> first bits are received.
>>
>> I'm not able to wait and load all data into memory before to use it.
>>
>> I can not yet find the way to do it.
>>
>> I need a little help :)
>>
>>
>>
>> Le mardi 2 août 2016 10:31:01 UTC+2, Konrad Malawski a écrit :
>>>
>>> Yeah, that'd be good to collect usecases and demadn there.
>>> Though with SAX it won't be automagical ever, they always require 
>>> manually writing the parsing AFAIR
>>>
>>> -- 
>>> Konrad `ktoso` Malawski
>>> Akka  @ Lightbend 
>>>
>>> On 2 August 2016 at 10:29:21, André (andre.r...@gmail.com) wrote:
>>>
>>> Maybe you can add your use case to 
>>> https://github.com/akka/akka/issues/16210.
>>>
>>> On Tuesday, August 2, 2016 at 10:12:13 AM UTC+2, JY wrote: 

 I am new with AKKA and AKKA HTTP.

 I have to write a HTTP server application that reads a lot very big XML 
 data flows (via HTTP PUT method) and parses this Data with a SAX parser.

 How can I do that with akka-http?

 Thanx in advance for your help.

>>> --
>>> >> Read the docs: http://akka.io/docs/
>>> >> Check the FAQ: 
>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>> >> Search the archives: 
>>> https://groups.google.com/group/akka-user
>>> ---
>>> You received this message because you are subscribed to the Google 
>>> Groups "Akka User List" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to akka-user+...@googlegroups.com.
>>> To post to this group, send email to akka...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/akka-user.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: How to read XML inputstream (with SAX parser) received by akka-http?

[JY]

I forgot to specify that flow I have to analyze weigh between 1 and 5 GB, 
and I have to extract the value of a dozen tags, which is why we can't load 
all data into memory, and that we've choose SAX :)



Le mardi 2 août 2016 11:23:18 UTC+2, JY a écrit :
>
> In fact, (I will try to be both more specific and more generic):
>
> I have to read (via InputStream because of SAX parser) the data when the 
> first bits are received.
>
> I'm not able to wait and load all data into memory before to use it.
>
> I can not yet find the way to do it.
>
> I need a little help :)
>
>
>
> Le mardi 2 août 2016 10:31:01 UTC+2, Konrad Malawski a écrit :
>>
>> Yeah, that'd be good to collect usecases and demadn there.
>> Though with SAX it won't be automagical ever, they always require 
>> manually writing the parsing AFAIR
>>
>> -- 
>> Konrad `ktoso` Malawski
>> Akka  @ Lightbend 
>>
>> On 2 August 2016 at 10:29:21, André (andre.r...@gmail.com) wrote:
>>
>> Maybe you can add your use case to 
>> https://github.com/akka/akka/issues/16210.
>>
>> On Tuesday, August 2, 2016 at 10:12:13 AM UTC+2, JY wrote: 
>>>
>>> I am new with AKKA and AKKA HTTP.
>>>
>>> I have to write a HTTP server application that reads a lot very big XML 
>>> data flows (via HTTP PUT method) and parses this Data with a SAX parser.
>>>
>>> How can I do that with akka-http?
>>>
>>> Thanx in advance for your help.
>>>
>> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com.
>> To post to this group, send email to akka...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Re: How to read XML inputstream (with SAX parser) received by akka-http?

[Konrad Malawski]

Yeah, that'd be good to collect usecases and demadn there.
Though with SAX it won't be automagical ever, they always require manually
writing the parsing AFAIR

-- 
Konrad `ktoso` Malawski
Akka  @ Lightbend 

On 2 August 2016 at 10:29:21, André (andre.ruedi...@gmail.com) wrote:

Maybe you can add your use case to https://github.com/akka/akka/issues/16210
.

On Tuesday, August 2, 2016 at 10:12:13 AM UTC+2, JY wrote:
>
> I am new with AKKA and AKKA HTTP.
>
> I have to write a HTTP server application that reads a lot very big XML
> data flows (via HTTP PUT method) and parses this Data with a SAX parser.
>
> How can I do that with akka-http?
>
> Thanx in advance for your help.
>
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups
"Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] Re: How to read XML inputstream (with SAX parser) received by akka-http?

[André]

Maybe you can add your use case 
to https://github.com/akka/akka/issues/16210.

On Tuesday, August 2, 2016 at 10:12:13 AM UTC+2, JY wrote:
>
> I am new with AKKA and AKKA HTTP.
>
> I have to write a HTTP server application that reads a lot very big XML 
> data flows (via HTTP PUT method) and parses this Data with a SAX parser.
>
> How can I do that with akka-http?
>
> Thanx in advance for your help.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.