If you only want to pull one row at a time, you need a feedback-loop to detect when the row has been processed so you can pull in new ones. It's either that or bounding the input and output buffers so that the aggregate matches your maximum work-in-progress.
On Wed, Oct 15, 2014 at 1:09 PM, Boris Lopukhov <[email protected]> wrote: > > What do you do if you have a row in the database that exceeds that max > size? You never read it? > > I can guarantee that the size of one row does not exceed the certain > maxsize > > > Can you write a SQL query that returns all rows up to a certain maxsize? > > I think so, but I'm going to read rows sequentially one row per a sql > request > > среда, 15 октября 2014 г., 13:48:15 UTC+4 пользователь √ написал: >> >> What do you do if you have a row in the database that exceeds that max >> size? You never read it? Can you write a SQL query that returns all rows up >> to a certain maxsize? >> On Oct 15, 2014 11:40 AM, "Boris Lopukhov" <[email protected]> wrote: >> >>> I need to write a simple application that reads rows from the table of >>> the postgres, indexes them in elasticsearch and if successful records in >>> postgres that data has been indexed. Rows have sizes of about 1 Kbyte to >>> 100MB. I'm trying to solve this with the akka-stream: >>> >>> FlowFrom(messagePublisher) >>> .mapFuture(elasticService.indexMessage) >>> .withSink(messagesSink) >>> .run >>> >>> >>> >>> Question: What is the best way to do a flow restriction for rows to >>> retrieve from the database so that the summary size of the in-process rows >>> does not exceed a certain value? >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ >>> current/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups.google.com/ >>> group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Cheers, √ -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
