Il giorno mercoledì 3 febbraio 2016 01:54:55 UTC+1, rrodseth ha scritto:
>
> No worries. I wish I had the time and expertise to help.
>
> I don't mean to be a pest, but since my credibility with management is at
> stake [ :) ] can anyone suggest any ways I can tackle the problem of
> groupBy followed by write to file?
>
> I might be able to get away with draining each substream to a collection
> and then using mapAsync to run a separate stream (per-key) to write the
> file, returning a Future as required by mapAsync. Not unlike where the
> activator template has ended up, but I'd really rather not.
>
> But I also find myself wondering if I could do something with fold, where
> the initial value contains the Sink for that substream.
>
> Roland also alluded to a hack(?) using prefixAndTail(0), but I don't quite
> see what that would look like.
>
Hi,
I can try to help using prefixAndTail, I came up with this so far:
FileIO.fromFile(logFile).
// parse chunks of bytes into lines
via(Framing.delimiter(ByteString(System.lineSeparator),
maximumFrameLength = 512, allowTruncation = true)).
map(_.utf8String).
map {
case line@LoglevelPattern(level) => (level, line)
case line@other => ("OTHER", line)
}.
// group them by log level
groupBy(5, _._1).
prefixAndTail(1).
// write lines of each group to a separate file
mapAsync(5) {
case (Seq((level, line)), source) =>
Source.single(line)
.concat(source.map(_._2))
.map(line => ByteString(line + "\n"))
.runWith(FileIO.toFile(new java.io.File(s"target/log-$level.txt")))
// Not sure that this will ever happen
case _ =>
Future.successful(0)
}.
mergeSubstreams.
runWith(Sink.onComplete { _ =>
system.shutdown()
})
What prefixAndTail does (if I got it right) emits a pair where the first
value is a sequence of n elements (prefix), and
the second value is the rest (tail) lifted to a source. Here I'm using
prefixAndTail(1) because I need to extract the level
from the first element to calculate the filename.
Hope it helps,
Francesco
> Clutching at straws!
>
> On Tue, Feb 2, 2016 at 12:27 AM, Roland Kuhn <[email protected]
> <javascript:>> wrote:
>
>> Yes, indeed, I didn’t have the time to get that link yesterday. If anyone
>> wants to work on that: contributions are *always* welcome! :-)
>>
>> Regards,
>>
>> Roland
>>
>> 1 feb 2016 kl. 23:59 skrev Richard Rodseth <[email protected]
>> <javascript:>>:
>>
>> For anyone following along, I believe this is the issue Roland refers to
>>
>> https://github.com/akka/akka/issues/18969
>>
>> On Mon, Feb 1, 2016 at 2:28 PM, Richard Rodseth <[email protected]
>> <javascript:>> wrote:
>>
>>> Ouch. Thanks.
>>>
>>> On Mon, Feb 1, 2016 at 1:49 PM, Roland Kuhn <[email protected]
>>> <javascript:>> wrote:
>>>
>>>> Hi Richard,
>>>>
>>>> this is not yet solved, and we have an issue tracking this in akka/akka
>>>> as well. It is not certain that we will be able to fix this before 2.4.2
>>>> comes out or whether the API addition that is necessary will have to wait
>>>> until 2.4.3.
>>>>
>>>> Regards,
>>>>
>>>> Roland
>>>>
>>>> 1 feb 2016 kl. 22:34 skrev Richard Rodseth <[email protected]
>>>> <javascript:>>:
>>>>
>>>> I'm concerned that this might fall through the cracks since the GitHub
>>>> issue is written against the Activator template rather than akka itself.
>>>> If it's already solved in 2.4.x please let me know.
>>>> If anyone can offer any guidance for a workaround, I'd appreciate it.
>>>> In the linked discussion Roland mentioned something about prefixAndTail
>>>> but
>>>> said you wouldn't want that in a tutorial.
>>>> I also think it would be helpful for those who have seen both the
>>>> pre-SubFlow and post-SubFlow templates if the reason for this unfortunate
>>>> regression was stated in the activator template.
>>>>
>>>> On Sun, Jan 31, 2016 at 6:23 PM, Richard Rodseth <[email protected]
>>>> <javascript:>> wrote:
>>>>
>>>>> I have run into this issue
>>>>> https://github.com/typesafehub/activator-akka-stream-scala/issues/37
>>>>>
>>>>> I want to group a stream and write each substream to a separate file.
>>>>> A pretty common use case, I'd imagine.
>>>>>
>>>>> The old version of the GroupLog example showed a groupBy() followed by
>>>>> a to()
>>>>>
>>>>> Because of the introduction of SubFlow, the new version had to
>>>>> introduce an unacceptable fold() into a strict collection
>>>>>
>>>>>
>>>>> https://github.com/typesafehub/activator-akka-stream-scala/blob/c63afb5eecf50726c4417157efd26d697b621e99/src/main/scala/sample/stream/GroupLogFile.scala
>>>>>
>>>>> What are my options? ETA for a fix?
>>>>> Thanks
>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>>>> >>>>>>>>>> Check the FAQ:
>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>> >>>>>>>>>> Search the archives:
>>>> https://groups.google.com/group/akka-user
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Akka User List" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected] <javascript:>.
>>>> To post to this group, send email to [email protected]
>>>> <javascript:>.
>>>> Visit this group at https://groups.google.com/group/akka-user.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
>>>>
>>>>
>>>> *Dr. Roland Kuhn*
>>>> *Akka Tech Lead*
>>>> Typesafe <http://typesafe.com/> – Reactive apps on the JVM.
>>>> twitter: @rolandkuhn
>>>> <http://twitter.com/#!/rolandkuhn>
>>>>
>>>>
>>>> --
>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>>>> >>>>>>>>>> Check the FAQ:
>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>> >>>>>>>>>> Search the archives:
>>>> https://groups.google.com/group/akka-user
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Akka User List" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected] <javascript:>.
>>>> To post to this group, send email to [email protected]
>>>> <javascript:>.
>>>> Visit this group at https://groups.google.com/group/akka-user.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
>>
>> --
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To post to this group, send email to [email protected]
>> <javascript:>.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>>
>> *Dr. Roland Kuhn*
>> *Akka Tech Lead*
>> Typesafe <http://typesafe.com/> – Reactive apps on the JVM.
>> twitter: @rolandkuhn
>> <http://twitter.com/#!/rolandkuhn>
>>
>> --
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To post to this group, send email to [email protected]
>> <javascript:>.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
