Yup On Thu, Oct 26, 2017 at 3:51 PM, Rafał Krzewski <[email protected]> wrote:
> Hi Viktor, > > I'm passing the OutputStream to a single method that does a bunch of > writes synchronously and as soon as it returns I do os.close() so I should > be good, no? > > Cheers, > Rafał > > W dniu czwartek, 26 października 2017 14:02:32 UTC+2 użytkownik √ napisał: >> >> Be careful though, as you need to ensure proper happens-before >> relationship between the writes and the closing of the OutputStream. (and >> other writes of course) >> >> On Thu, Oct 26, 2017 at 1:56 PM, Rafał Krzewski <[email protected]> >> wrote: >> >>> That worked like a charm :) Thanks a lot Johan! >>> >>> Cheers, >>> Rafał >>> >>> W dniu czwartek, 26 października 2017 10:49:52 UTC+2 użytkownik Johan >>> Andrén napisał: >>>> >>>> Materialization is not async, so anything you do in >>>> mapMaterializedValue will block the materialization. Simplest solution is >>>> to not run the logic that feeds the OutputStream inside the >>>> materialization, but instead do that by keeping the materialized value >>>> through your chain of stages, and getting it as the returned value from >>>> run(). In some cases, maybe yours, you cannot do that because you are >>>> passing the blueprint to some API that will do the actual materialization, >>>> in that case you can instead fork off a future/thread to do the writing, to >>>> let the materialization complete. >>>> >>>> Something like this: >>>> >>>> def legacySource = >>>> StreamConverters.asOutputStream() >>>> .mapMaterializedValue { outputstream => >>>> Future { >>>> while(something) { >>>> outputstream.write(data) >>>> } >>>> }(use-a-blocking-specific-dispatcher-here-as-write-is-blocking) >>>> NotUsed >>>> } >>>> >>>> val route = get { >>>> complete { >>>> HttpEntity( >>>> ContentTypes.`application/octet-stream`, >>>> legacySource >>>> ) >>>> } >>>> } >>>> >>>> -- >>>> Johan >>>> Akka Team >>>> >>>> On Wednesday, October 25, 2017 at 1:42:58 PM UTC+2, Rafał Krzewski >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> I've tried using `StreamConverters.asOutputStream` and immediately >>>>> run into the deadlock described in the documentation [1] >>>>> >>>>> My use case is that I am am creating Excel spreadsheet using Apache >>>>> POI's streaming API [2] and then stream out the result using Akka HTTP. >>>>> >>>>> Under the hood POI is writing the row data into a temporary files on >>>>> disk. When all data is ready, one can write out the result to an arbitrary >>>>> `OutputStream` [3]. >>>>> >>>>> When interoperating with Akka HTTP I need to provide a `Source` that >>>>> is materialized by the framework. That's why my intuition was using >>>>> `mapMaterializedValue` to provide the code making use of the >>>>> `OutputStream` >>>>> at the site when I'm creating the source. Unfortunately this does not >>>>> work. >>>>> I was able to work around that by writing out the spreadsheet to another >>>>> temporary file, providing a `Source` to Akka HTTP side using `FileIO` and >>>>> some additional song and dance to clean up the temporary files in all >>>>> circumstances. >>>>> >>>>> >>>>> I am wondering if there is a way I could use >>>>> `StreamConverters.asOutputStream` correctly in this scenario that I >>>>> don't see? Or maybe another kind of API would be necessary here? I'm >>>>> thinking about something along the lines of: >>>>> >>>>> >>>>> `asOutputStream(f: OutputStream => Done, writeTimeout: FiniteDuration >>>>> = 5.seconds): Source[ByteString, Future[IOResult]]` >>>>> >>>>> >>>>> `f` would be invoked after the stream is ready for writing. After `f` >>>>> completes, the framework could ensure the stream is cleaned up properly. >>>>> The returned `IOResult.status` could be used to check whether `f` >>>>> completed >>>>> normally. If `f` fails to complete within specified timeout, any further >>>>> attempt to call methods on the `OutputStream` should result in an >>>>> `IOException`. The problem I see is that `f` could get permanently blocked >>>>> on some condition and thus steal a thread from >>>>> `akka.stream.blocking-io-dispatcher` but I don't think there is any >>>>> way to handled that on the JVM. >>>>> >>>>> >>>>> I am not sure if the above is feasible but if it were I'm sure people >>>>> would find it useful for interfacing with legacy code ;) >>>>> >>>>> >>>>> Cheers, >>>>> >>>>> Rafał >>>>> >>>>> >>>>> [1] https://doc.akka.io/docs/akka/current/scala/stream/stages-ov >>>>> erview.html#additional-sink-and-source-converters >>>>> >>>>> [2] https://poi.apache.org/spreadsheet/how-to.html#sxssf >>>>> >>>>> [3] https://poi.apache.org/apidocs/org/apache/poi/xssf/streaming >>>>> /SXSSFWorkbook.html#write-java.io.OutputStream- >>>>> >>>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/c >>> urrent/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups.google.com/grou >>> p/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Cheers, >> √ >> > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ > current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Cheers, √ -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
