Hi Harit, according to the docs <http://doc.akka.io/api/akka-stream-and-http-experimental/1.0/?_ga=1.121420744.1150407535.1394739016#akka.stream.io.Framing$> allowTruncation controls whether to fail the stream if the last upstream line does not contain a delimiter.
The built-in delimiter framing <https://github.com/akka/akka/blob/97622c22b151eeb230302aff6479780690b42b8d/akka-stream/src/main/scala/akka/stream/io/Framing.scala#L34> stage does not handle all use cases. However it is a good example of such functionality. I would recommend taking a look at the source code and adapting it to your needs. On Tue, Oct 6, 2015 at 10:28 PM, Harit Himanshu < [email protected]> wrote: > Hello there, > > I am new to Akka-Stream and working on a use case where I need to parse > log files. These log files have lines separated by new line. > > I was looked at GroupLogFile.scala > <https://github.com/typesafehub/activator-akka-stream-scala/blob/master/src/main/scala/sample/stream/GroupLogFile.scala> > and > my code now looks like > > class LogFile(file: File, implicit val system: ActorSystem) { > Predef.assert(file.exists(), "log file must exists") > > implicit val materializer = ActorMaterializer() > val logger = Logger(LoggerFactory.getLogger(getClass)) > > val source: Source[ByteString, Future[Long]] = > Source.synchronousFile(file) > > // todo (harit): what should be maximumFrameLength > val flow: Flow[ByteString, String, Unit] = Flow[ByteString] > .via(Framing.delimiter(ByteString(System.lineSeparator), > maximumFrameLength = 1500, allowTruncation = true)) > .map(_.utf8String) > > def process() = { > logger.debug(s"processing $file") > source.via(flow).runForeach(println) > } > } > > The problem arises when a log line is more than 1500 characers (1500 > bytes), the stream fails. I asserted by running it against a log file and > it stopped on a line which had 1932 characters (1932 bytes) > I set allowTruncation = true, so I thought the line would truncate > everything after 1500 bytes and move on, but its not the case. > > *Question* > - How can I truncate over 1500 bytes, process (or print) the line and move > on with next line? without failing the stream? > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Martynas Mickevičius Typesafe <http://typesafe.com/> – Reactive <http://www.reactivemanifesto.org/> Apps on the JVM -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
