Another approach, if you don't have access to shared filesystem, is the MessageSequence pattern, where a message is broken up into multiple chunks.
Here is an implementation that we are using: https://gist.github.com/ericacm/7947424 On Friday, August 8, 2014 10:02:41 AM UTC-4, Konrad Malawski wrote: > > Hello Syed, > So the reason we wanted to verify 2.3.4 is that we have implemented a > separate way to propagate heartbeats in that version – shouldn't disconnect > nodes under load from the cluster as easily. > > The topic here discussed has been touched upon a few times in the last > weeks, refer to: > arbitrary and unpredictably large messages to a remote actor > <https://groups.google.com/forum/#!searchin/akka-user/large$20messages/akka-user/HXmpLAMWmjA/I3MTsGlLMIIJ> > Large messages s. distributed file system > <https://groups.google.com/forum/#!searchin/akka-user/large$20messages/akka-user/gO3J29ebJXU/kO2e7iXaYKkJ> > <- > my fav. suggestion at this point. > > You can also enable debug logging and remote lifecycle events > <https://groups.google.com/forum/#!searchin/akka-user/large$20messages/akka-user/I5xcmDF5X00/v2Q7hLcPCuQJ>, > > so we have more clues on what's actually happening. > > > > On Thu, Aug 7, 2014 at 11:30 PM, Syed Ahmed <[email protected] > <javascript:>> wrote: > >> FYI - Just tried with 2.3.4 and it doesn't change the behavior.. >> >> >> >> On Thursday, August 7, 2014 10:38:38 AM UTC-7, Syed Ahmed wrote: >>> >>> Hi Ryan, >>> In my test its a very large string and I just get the string back from >>> the message (i.e. it gets deserialized) -- Im not doing anything further. >>> This test is to just check on how large message can be send accross. I will >>> attempt again with 2.3.4 to see if it makes a difference >>> thx >>> -syed >>> >>> On Wednesday, August 6, 2014 3:12:27 PM UTC-7, Ryan Tanner wrote: >>>> >>>> When those large messages are received, what's supposed to happen? >>>> It's possible that whatever processing is triggered by that message is so >>>> CPU intensive that Akka's internal remoting can't get any time for its own >>>> messages. >>>> >>>> On Wednesday, August 6, 2014 12:08:20 PM UTC-6, Syed Ahmed wrote: >>>>> >>>>> Hello, >>>>> Im new to Akka/Akka Remoting.. >>>>> >>>>> We need to send large messages in the order 500-600MB. After viewing >>>>> and checking some mailer post I found that I have to change Akka Remote >>>>> netty tcp settings which I did and my application.conf has the >>>>> following... >>>>> >>>>> remote { >>>>> transport = "akka.remote.netty.NettyRemoteTransport" >>>>> netty.tcp { >>>>> hostname = "<ip address of host>" >>>>> port = 2553 >>>>> maximum-frame-size = 524288000 >>>>> send-buffer-size= 500000000 >>>>> receive-buffer-size=500000000 >>>>> } >>>>> } >>>>> >>>>> I create a client and remote server app using akka-remoting. Using >>>>> protobuff message serialization -- a client sends a large message to >>>>> server >>>>> and the server acknowledges with the timestamp when it received after >>>>> deserialized the same. >>>>> >>>>> With above settings things work fine for messages upto 200MB. When I >>>>> try slightly large message 250MB.. The server shows the following .. >>>>> >>>>> --- >>>>> [INFO] [08/05/2014 23:41:39.302] >>>>> [RemoteNodeApp-akka.remote.default-remote-dispatcher-4] >>>>> [akka.tcp://RemoteNodeApp@<remote-ip>:2552/system/transports/ >>>>> akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F% >>>>> 2FRemoteNodeApp%40<remote_ip>%3A36922-1] No response from remote. >>>>> Handshake timed out or transport failure detector triggered. >>>>> >>>>> --- >>>>> >>>>> and the client shows the following >>>>> >>>>> [WARN] [08/05/2014 23:41:39.326] >>>>> [LocalNodeApp-akka.remote.default-remote-dispatcher-6] >>>>> [akka.tcp://LocalNodeApp@<client_ip>:2553/system/endpointManager/ >>>>> reliableEndpointWriter-akka.tcp%3A%2F%2FRemoteNodeApp%4010.194.188.97%3A2552-0] >>>>> >>>>> Association with remote system >>>>> [akka.tcp://RemoteNodeApp@<server_ip>:2552] >>>>> has failed, address is now gated for [5000] ms. Reason is: >>>>> [Disassociated]. >>>>> >>>>> and the message is not sent at all.. Any idea what else is missing? >>>>> >>>>> BTW -- in the above I don't see any OOM errors or anything and the >>>>> client/remote app are still up and running. >>>>> >>>>> thx >>>>> -Syed >>>>> >>>>> >>>>> >>>>> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> Visit this group at http://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Cheers, > Konrad 'ktoso' Malawski > hAkker @ Typesafe > > <http://typesafe.com> > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
