Just to clarify a comment from Owen, conserve-sockets=true does show improved performance over conserve-sockets=false in certain specific scenarios, but this isn't a hard and fast rule that applies everywhere, even excluding the cases where using conserve-sockets=true can lead to distributed deadlocks. As an example, with the new default setting of false, the UpgradeTest job in the CI pipeline takes about 10% longer, indicating that, at least for the scenarios being tested there, there is some performance impact. That being said, all of the geode-benchmarks tests explicitly set conserve-sockets=false, so from the point of view of what we're actually testing in terms of performance, no impact will be seen due to this change. ________________________________ From: Jacob Barrett <jabarr...@vmware.com> Sent: Thursday, November 19, 2020 8:02 AM To: dev@geode.apache.org <dev@geode.apache.org> Subject: Re: [PROPOSAL] Change the default value of conserve-sockets to false
I would argue that is doesn’t change the outward behavior of the product. It does change the internal workings of the product. It does improve the performance and reliability. As long as changes to the internals don’t have effect on the outward facing behavior of the product I see no problem making these internal changes. Yes this may lead to more resource utilization but so can and have so other changes along the way between majors. I would also expect in the scenarios where this would make the most impact on resources they are already configured to use this feature. +1 make the change. > On Nov 19, 2020, at 1:16 AM, Ju@N <jujora...@gmail.com> wrote: > > I'm all in for changing the default to *false* but, unfortunately and for > all the reasons already stated in the thread, I'm hesitant to include this > change as part of a minor release. > Best regards. > > On Thu, 19 Nov 2020 at 02:48, John Blum <jb...@vmware.com> wrote: > >> The downside of conserve-sockets = false is that you are (essentially) >> back to a Thread|Socket / Request model (though Geode limits this system >> resource consumption to a degree by the use of Thread Pools in p2p >> distribution layer) and thus, you can run out of file descriptors (per >> newly opened Socket) pretty quickly if you are not careful. >> >> conserve-sockets set to true limits the use of finite system resources why >> risking deadlocks (i.e. A -> B -> C -> A), which is also contingent on ACKS >> (and the infamous ReplyProcessor21; at least at 1 time, not sure if it is >> still in play, but probably!). >> >> conserve-sockets set to false uses more system resources but avoids >> deadlocks. >> >> If this change is made, I'd minimally make sure to document that users >> should adjust their (soft & hard) ulimits accordingly, based on use cases, >> load, etc. >> >> Personally, this has caused enough grief in the past (both ways, >> actually!) that I 'd say this is a major version change. >> >> -j >> >> >> ________________________________ >> From: Nabarun Nag <n...@vmware.com> >> Sent: Wednesday, November 18, 2020 6:09 PM >> To: dev@geode.apache.org <dev@geode.apache.org> >> Subject: Re: [PROPOSAL] Change the default value of conserve-sockets to >> false >> >> +1 >> >> * As nearly all of the production use-cases need "conserve-sockets" to >> be set to false, I think we can aim for changing the default value to false >> for 1.14.0 release. >> * We can highlight this change in the release notes and emails. >> >> Regards, >> Nabarun >> >> ________________________________ >> From: Udo Kohlmeyer <u...@vmware.com> >> Sent: Wednesday, November 18, 2020 6:00 PM >> To: dev@geode.apache.org <dev@geode.apache.org> >> Subject: Re: [PROPOSAL] Change the default value of conserve-sockets to >> false >> >> Hi there Donal, >> >> Thank you for raising this. It is not an uncommon request to change the >> default value of this field. >> >> This has been discussed many times in the past. I would LOVE to approve >> this change, but this would mean that users that don’t set this property >> might suddenly have this property changed. We are not sure that this would >> mean for these users. BUT >> >> That said, there have been very little (to none) complaints about the >> product stability when `conserve-sockets=false` are set. >> >> +1 - if we are allowed to make this change outside of a major version >> change. >> >> --Udo >> >> From: Donal Evans <doev...@vmware.com> >> Date: Thursday, November 19, 2020 at 12:04 PM >> To: dev@geode.apache.org <dev@geode.apache.org> >> Subject: [PROPOSAL] Change the default value of conserve-sockets to false >> Hi Geode dev, >> >> First, from the docs[1], a brief explanation of the purpose of the >> conserve-sockets property: >> >> "The conserve-sockets setting indicates whether application threads share >> sockets with other threads or use their own sockets for member >> communication. This setting has no effect on communication between a server >> and its clients, but it does control the server’s communication with its >> peers or a gateway sender’s communication with a gateway receiver." >> >> The current default value for the conserve-sockets property is true, which >> at first glance makes sense, since in an ideal world, existing sockets >> could be shared between threads and there would be no need to create and >> destroy new sockets for each process, which can be somewhat >> resource-intensive. However, in practice, there are several known issues >> with using the default setting of true. From the docs[1]: >> >> "For distributed regions, the put operation, and destroy and invalidate >> for regions and entries, can all be optimized with conserve-sockets set to >> false. For partitioned regions, setting conserve-sockets to false can >> improve general throughput. >> Note: When you have transactions operating on EMPTY, NORMAL or PARTITION >> regions, make sure that conserve-sockets is set to false to avoid >> distributed deadlocks." >> >> and[2]: >> >> "WAN deployments increase the messaging demands on a Geode system. To >> avoid hangs related to WAN messaging, always set `conserve-sockets=false` >> for Geode members that participate in a WAN deployment." >> >> Given that it is generally accepted as best practice to set >> conserve-sockets to false for almost all use cases of Geode beyond the most >> simple, it would make sense to also change the default value to false, to >> prevent people having to encounter a problem, search for the solution, then >> change the setting to what is almost always the "correct" value. >> >> I have done some experimenting to see what it would take to make this >> proposal a reality, and the changes required are very minimal, with only >> two existing DUnit tests that need to be modified to explicitly set the >> value of conserve-sockets that were previously relying on the default being >> true. >> >> Any feedback on this proposal would be very welcome, and if the response >> is positive, I can create a PR with the changes as soon as a decision is >> reached. >> >> Thanks, >> Donal >> >> [1] >> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgeode.apache.org%2Fdocs%2Fguide%2F113%2Fmanaging%2Fmonitor_tune%2Fperformance_controls_controlling_socket_use.html&data=04%7C01%7Cdoevans%40vmware.com%7C6986324c9c2c4051425808d88ca4872a%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637413985573024235%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3099YogxVmgVipro6vrU9P4de501hbrGPT9dGns47VA%3D&reserved=0 >> [2] >> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgeode.apache.org%2Fdocs%2Fguide%2F113%2Fmanaging%2Fmonitor_tune%2Fsockets_and_gateways.html&data=04%7C01%7Cdoevans%40vmware.com%7C6986324c9c2c4051425808d88ca4872a%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637413985573034231%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=v6FuDfn3RWXCRlJDrqGMdBz18j32GjdIpbW41SDg6BI%3D&reserved=0 >> > > > -- > Ju@N
