Hi, Socket exception can be caused by wrong network configuration or firewall configuration. If node will not be able to send an response to other node then it can cause grid hanging.
(Un)marshall exceptions (that is not caused by network exceptions) is a signal that smth goes wrong. On Fri, Mar 23, 2018 at 8:48 AM, akshaym <[email protected]> wrote: > I have pushed the sample application to github > <https://github.com/akshaymhetre/SparkJobAsIgniteService> . Please check > it > once. > > Also, I am able to get rid of the hang issue with spark.close API call by > adding "igniteInstanceName" property. Not sure if its a right approach > though. > I came up with this solution, while debugging this issue. What I observed > is > that during saving dataframe to ignite, it needs Ignite context. It first > checks if the context is already there, if it is exists it uses that > context > to save dataframe in ignite and on spark close API call it tries to close > the same context. > As I am trying to run this spark job as an ignite service, I wanted it to > run continuously. So closing the ignite context was causing this issue. So > to make dataframe APIs to create new context everytime, I added > "igniteInstanceName" property to config which I am apsing to new ignite DF > APIs. > > Though it resolves the hang issue it is still showing some socket > connection > and unmarshalling exceptions. Do I need to worry about it? How can I get > rid > of those? > > Also, Any trade-offs if we use Spark As Ignite Service when executed with > Yarn? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > -- Best regards, Andrey V. Mashenkov
