Re: [akka-user] akka-cluster, speeding up join/memberUp events for end-to-end tests?

'Konstantinos Kougios' via Akka User List Thu, 25 Jun 2015 22:11:32 -0700

ok thanks, this now works and the test runs in 5.5 secs. With


gossip-interval= 100 ms

it goes down to 4.5s


On 25/06/15 13:11, Patrik Nordwall wrote:

You can only join an existing cluster, and bootstrap by joining one toitself.


for (c <- allClusters) c.join(cluster1.selfAddress)

On Thu, Jun 25, 2015 at 8:11 AM, 'Konstantinos Kougios' via Akka UserList <[email protected] <mailto:[email protected]>>wrote:


    Hi Patrik,
    On 23/06/15 08:07, Patrik Nordwall wrote:

    Then I think you can reduce the
    akka.cluster.seed-node-timeout = 2s

    it improved things, currently the overal time to startup the
    cluster is 7 secs (it used to be 13)


    It is 5 seconds by default.

    I normally use the join command instead of joinSeedNodes when
    testing.
    Cluster(system1).join(Cluster(system1).selfAddress)
    Cluster(system2).join(Cluster(system1).selfAddress)
    Cluster(system3).join(Cluster(system1).selfAddress)

    I tried that but it didn't work. It seems my cluster1 can see the
    rest, but my driverCluster can see itself. I tried doing it in
    different order and it seems the order I do things matters. Here
    is what my code looks like:

    val allClusters = List(cluster1, cluster2, driverCluster)
        //    Thread.sleep(1000) <-- I tried that too, but it didn't work

    for (c <- allClusters) join(c)

    private def join(cluster: Cluster): Unit = for (c <- allClusters)
    cluster.join(c.selfAddress)


    Then I am waiting like

    awaitCond(
        cluster1.state.members.size ==3&&cluster2.state.members.size 
==3&&driverCluster.state.members.size ==3,
        60seconds
    )


    If I add some printlns to print the members of each cluster in
    that await, I am not getting consistent results. driverCluster
    seems to see itself only.

    Thanks,

    Kostas


    Cheers,
    Patrik

    On Mon, Jun 22, 2015 at 8:44 PM, 'Konstantinos Kougios' via Akka
    User List <[email protected]
    <mailto:[email protected]>> wrote:

        Thanks Patrik but still it takes the same amount of time.
        While waiting I get these in the logs

        [INFO] [06/22/2015 19:43:31.255]
        [testSystem-akka.actor.default-dispatcher-4]
        [akka://testSystem/deadLetters] Message
        [akka.cluster.InternalClusterAction$InitJoinNack] from
        Actor[akka://testSystem/system/cluster/core/daemon#-2049681964 
<tel:2049681964>]
        to Actor[akka://testSystem/deadLetters] was not delivered.
        [2] dead letters encountered. This logging can be turned off
        or adjusted with configuration settings
        'akka.log-dead-letters' and
        'akka.log-dead-letters-during-shutdown'.
        [INFO] [06/22/2015 19:43:32.172]
        [testSystem-akka.actor.default-dispatcher-3]
        [akka://testSystem/deadLetters] Message
        [akka.cluster.InternalClusterAction$InitJoin$] from
        
Actor[akka://testSystem/system/cluster/core/daemon/firstSeedNodeProcess-1#1602924580]
        to Actor[akka://testSystem/deadLetters] was not delivered.
        [3] dead letters encountered. This logging can be turned off
        or adjusted with configuration settings
        'akka.log-dead-letters' and
        'akka.log-dead-letters-during-shutdown'.
        [INFO] [06/22/2015 19:43:33.171]
        [testSystem-akka.actor.default-dispatcher-19]
        [akka://testSystem/deadLetters] Message
        [akka.cluster.InternalClusterAction$InitJoin$] from
        
Actor[akka://testSystem/system/cluster/core/daemon/firstSeedNodeProcess-1#1602924580]
        to Actor[akka://testSystem/deadLetters] was not delivered.
        [4] dead letters encountered. This logging can be turned off
        or adjusted with configuration settings
        'akka.log-dead-letters' and
        'akka.log-dead-letters-during-shutdown'.
        [INFO] [06/22/2015 19:43:34.173]
        [testSystem-akka.actor.default-dispatcher-18]
        [akka://testSystem/deadLetters] Message
        [akka.cluster.InternalClusterAction$InitJoin$] from
        
Actor[akka://testSystem/system/cluster/core/daemon/firstSeedNodeProcess-1#1602924580]
        to Actor[akka://testSystem/deadLetters] was not delivered.
        [5] dead letters encountered. This logging can be turned off
        or adjusted with configuration settings
        'akka.log-dead-letters' and
        'akka.log-dead-letters-during-shutdown'.
        [INFO] [06/22/2015 19:43:35.173]
        [testSystem-akka.actor.default-dispatcher-18]
        [akka://testSystem/deadLetters] Message
        [akka.cluster.InternalClusterAction$InitJoin$] from
        
Actor[akka://testSystem/system/cluster/core/daemon/firstSeedNodeProcess-1#1602924580]
        to Actor[akka://testSystem/deadLetters] was not delivered.
        [6] dead letters encountered. This logging can be turned off
        or adjusted with configuration settings
        'akka.log-dead-letters' and
        'akka.log-dead-letters-during-shutdown'.
        [INFO] [06/22/2015 19:43:36.180]
        [testSystem-akka.actor.default-dispatcher-17]
        [Cluster(akka://testSystem)] Cluster Node
        [akka.tcp://[email protected]:2700
        <mailto:akka.tcp://[email protected]:2700>] - Node
        [akka.tcp://[email protected]:2700
        <mailto:akka.tcp://[email protected]:2700>] is JOINING,
        roles [index-server]



        On 22/06/15 12:15, Patrik Nordwall wrote:

        You could try to decrease the interval of these periodic tasks:

        akka.cluster.gossip-interval = 500 ms
        akka.cluster.leader-actions-interval = 500 ms

        On Fri, Jun 19, 2015 at 10:11 PM, 'Konstantinos Kougios' via
        Akka User List <[email protected]
        <mailto:[email protected]>> wrote:

            Thanks Patrik, this worked. For me I know that 3 members
            should join, so I was able to do this like

            TestKit.awaitCond(
                cluster1.state.members.size ==3&&cluster2.state.members.size 
==3&&driverCluster.state.members.size ==3,
                20seconds
            )

            Still though my test takes 11 secs to run due to some
            cluster initial comm delays, any ideas how to give them
            a kick start?


            On 16/06/15 10:15, Patrik Nordwall wrote:

            Try awaitAssert in TestKit. Something like:

            val addresses = Set(Cluster(system1).selfAddress,
            Cluster(system2).selfAddress)
            within(10.seconds) {
            awaitAssert {
            Cluster(system1).state.members.map(_.address) should
            be(addresses)
            Cluster(system1).state.members.map(_.status) should
            be(Set(MemberStatus.Up))
            Cluster(system2).state.members.map(_.address) should
            be(addresses)
            Cluster(system2).state.members.map(_.status) should
            be(Set(MemberStatus.Up))
            }
                }

            Regards,
            Patrik

            On Mon, Jun 15, 2015 at 8:00 AM, Kostas kougios
            <[email protected]
            <mailto:[email protected]>> wrote:

                Hi,

                I am creating an end to end test for my cluster by
                creating 2 of them in the same jvm :

                valcluster1=Cluster(SystemName,server1Config)
                valcluster2=Cluster(SystemName,server2Config)

                Thread.sleep(10000)


                As you can see, I have to use a sleep() to allow
                the cluster to to talk to each other.

                Is there a way to speed this up?