Re: Reminder: [VOTE] Apache 2.1.9 and 2.0.26 releases

Emmanuel Lécharny Sun, 13 Oct 2024 14:56:04 -0700

Not so easy...

Keep in mind that we may have 2 process (a server and a client)interacting, so it's not possible to synchronize the two.

Here, the real problem is that if the UseReadOperation flag is set on anopened Session , then it means we can't guarantee that something writtenon the sessionOpened event will be available by the client doing a read().

A solution would rather be to inject the UseReadOperation flag in thedefault session configuration *before* connecting to the server:


        try {
            IoConnector connector = new NioSocketConnector();
            connector.setHandler(new IoHandlerAdapter());
            connector.getSessionConfig().setUseReadOperation(true);

ConnectFuture connectFuture = connector.connect(newInetSocketAddress("localhost", port));

            connectFuture.awaitUninterruptibly();
...


That fixes the issue.

On 13/10/2024 17:44, Jeff Genender wrote:

Gary… yep!  Countdown latch would od the trick for that.  I use that in all my 
thread based tests.

Jeff

On Oct 13, 2024, at 6:10 AM, Gary Gregory <garydgreg...@gmail.com> wrote:

Great analysis, thank you.

Could the test be better synchronized using a count down latch?

Gary

On Sun, Oct 13, 2024, 1:09 AM Emmanuel Lécharny <elecha...@gmail.com> wrote:

AFAIU, when you fetch an ephemeral port with a new ServerSocket(0), what
happens is that the address:port is in TIME_WAIT after the socket
closure, but as it's also using SO_REUSEADDR, the process can use it,
which guarantees it's available, and no other process can use it.

You just need to be sure no other test is going to call the
AvailablePortFinder.getNextAvailable() method.

So I retract what I said about the lock being caused by the wrong port
being used.


I analysed further the test and the Mina code, and I found there is a
race condition when using the read() command:

- On one thread, we do write a message when the session is opened
- On the other thread we set the session configuration UseReadOperation
flag to true, then do a read.


What happens - and I can reproduce the lock in debugging mode easily -
is that the read is executed too late, the message being already
propagated *before* the check for the UseReadOperation flag is switched
to 'true', so when read() is called, there is nothing to be read.

I can also reproduce the lock 100% by adding a Thread.sleep(10) before
setting the session configuration to accept read operation:

         try {
             IoConnector connector = new NioSocketConnector();
             connector.setHandler(new IoHandlerAdapter());
             ConnectFuture connectFuture = connector.connect(new
InetSocketAddress("localhost", port));
             connectFuture.awaitUninterruptibly();

             if (connectFuture.getException() != null) {
                 throw connectFuture.getException();
             }

             Thread.sleep( 10 );

connectFuture.getSession().getConfig().setUseReadOperation(true);
...


To get it working with the sleep(10), it's enough to add a
Thread.sleep(11) in the sessionOpened handler method:

             @Override
             public void sessionOpened(IoSession session) throws Exception
{
                 Thread.sleep( 11 );
...

Bottom line, the test is flaky. We should never expect a read to succeed
when the data being written are pushed by the sessionConnected()
handler's method.
There is no way we can get this fixed in MINA core code, because the
session only exists when it's opened, so too late for a configuration to
be injected in time - unless we are lucky - for it to be taken in
consideration when trying to do a read() on the client side.

In other words, this feature is badly designed. We should either
configure it when we create the server, so that it's available when the
session is created, or expect the user not to write anything in the
sessionOpened/sessionCreated handler's methods and assume that it will
be available on the client immediately...

I'll create a ticket for that issue. In the mean time, I don't think
it's a blocker for a release, because it's also present in every version
of MINA :/


On 12/10/2024 22:36, Gary Gregory wrote:

What I've done elsewhere (mostly at work) is to configure the main code
(like a server) to accept port 0 from a test, instead of a "free"
discovered port. This port ends up being used by a ServerSocket
constructor or equivalent SSL class. The main class under test needs to
have a getPort() method or methods depending on how deeply buried the
server socket is, that queries the allocated ServerSocket. The test can
then query the main code after it knows the socket is bound.

HTH,
Gary

On Sat, Oct 12, 2024, 3:16 PM Guillaume Nodet <gno...@apache.org
<mailto:gno...@apache.org>> wrote:



    Le sam. 12 oct. 2024 à 06:24, Emmanuel Lécharny <elecha...@gmail.com
    <mailto:elecha...@gmail.com>> a écrit :

        Indeed. But the point is that you can't fetch an available port
        and use
        it immediately and be sure it's not used by another process
        without some
        kind of global lock on your system...

        I don't see any other way to grab an available port from the OS,
        and
        guarantee it's not used when you have to bind a socket on it and
        release
        it immediately for use by another socket.


    If you bind a server socket on port 0, the OS will allocate a free
    port. So just need to get the bound port and configure the client to
    use it ?

        Back to the issue: can you grab a StackTrace to see exactly
        where it
        blocks, in case there is another issue?

        Many thanks!

        On 11/10/2024 13:59, Gary Gregory wrote:

But it's not really doing that because it's NOT using the

        allocated

ServerSocket. Once that method exists, any other process in

the

underlying OS can grab that port for itself.

Gary

On Fri, Oct 11, 2024, 7:53 AM Emmanuel Lécharny

        <elecha...@gmail.com <mailto:elecha...@gmail.com>

<mailto:elecha...@gmail.com <mailto:elecha...@gmail.com>>>

wrote:




    On 11/10/2024 10:01, Gary Gregory wrote:

Hi Emanuel,

Why not use ephemeral ports?


    this is exactly what the

        AvailablePortFinder.getNextAvailable()

    method does:

          public static int getNextAvailable() {
              try (ServerSocket serverSocket = new

        ServerSocket(0)){

                  // Here, we simply return an available port

        found by

    the system
                  return serverSocket.getLocalPort();
              } catch (IOException ioe) {
                  throw new

        NoSuchElementException(ioe.getMessage());

              }


    It might mean a bit of work update the

tests to query the port and then use it, but it seems

        less probe to

problems. I'll try again in a few hours.

Gary

On Thu, Oct 10, 2024, 10:17 PM Emmanuel Lécharny

    <elecha...@gmail.com <mailto:elecha...@gmail.com>

        <mailto:elecha...@gmail.com <mailto:elecha...@gmail.com>>

<mailto:elecha...@gmail.com

        <mailto:elecha...@gmail.com> <mailto:elecha...@gmail.com
        <mailto:elecha...@gmail.com>>>> wrote:


    Thanks Gary!

    DIRMINA777Test may fell due to some port collision

        (we use a

    AvailablePortFinder.getNextAvailable() call to

        find a port

    that the

    Acceptor can use, but as some other tests may run

    concurrently, the

    same
    port can be in use by another Acceptor, and the

        readFuture

    can wait

    forever because the IoHandler that responds to

        connection is

    not doing

    what is expected.

    It's not perfect, and the only solution is to run

        the test

    another time.


    Could you give it another try?

    Side note: the

        AvailablePortFinder.getNextAvailable() method,

    which is

    only used for tests, should probably be

        syncrhonized to avoid

    being

    called by many threads at the same time.

    Thanks!

    On 10/10/2024 15:10, Gary Gregory wrote:

+ 1 Release MINA 2.1.9

Release MINA 2.0.26: Running
org.apache.mina.transport.socket.nio.x hangs or

        takes

forever, I killed it after 30 minutes.

- Tested the src ZIPs
- ASCs OK
- SHA512 files are not machine verifiable with

        `shasum

    --check`

    due to

their incompatible format.
- mvn clean verify
- Using:
openjdk version "11.0.24" 2024-07-16
OpenJDK Runtime Environment Homebrew (build

        11.0.24+0)

OpenJDK 64-Bit Server VM Homebrew (build

        11.0.24+0, mixed

    mode)


Apache Maven 3.9.9

        (8e8579a9e76f7d015ee5ec7bfcdc97d260186937)

Maven home:

/usr/local/Cellar/maven/3.9.9/libexec

Java version: 11.0.24, vendor: Homebrew,

runtime:

          /usr/local/Cellar/openjdk@11

/11.0.24/libexec/openjdk.jdk/Contents/Home

Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "15.0.1", arch:

        "x86_64",

    family: "mac"


Darwin **** 24.0.0 Darwin Kernel Version

        24.0.0: Tue Sep 24

    23:36:30 PDT

2024; root:xnu-11215.1.12~1/RELEASE_X86_64

x86_64


Gary


On Wed, Oct 9, 2024 at 10:53 AM Emmanuel

Lécharny

    <elecha...@gmail.com <mailto:elecha...@gmail.com>

        <mailto:elecha...@gmail.com <mailto:elecha...@gmail.com>>

    <mailto:elecha...@gmail.com <mailto:elecha...@gmail.com>

        <mailto:elecha...@gmail.com <mailto:elecha...@gmail.com>>>

<mailto:elecha...@gmail.com

        <mailto:elecha...@gmail.com> <mailto:elecha...@gmail.com
        <mailto:elecha...@gmail.com>>

    <mailto:elecha...@gmail.com <mailto:elecha...@gmail.com>

        <mailto:elecha...@gmail.com <mailto:elecha...@gmail.com>>>>>

wrote:


    Hi!

    I have started this vote 9 days ago, still

        haven't get any

    vote yet...


    On 30/09/2024 19:21, Emmanuel Lecharny

wrote:

hi!

WARNING: there are 2 votes to cast!


This is a vote for a double release:
* MINA 2.1.9
* MINA 2.0.26

Those versions are a maintenance

        realase, fixing a

    bug in

    the way we

treat Strings when reading a IoBuffer:

DIRMINA-1181:Exception thrown when

        attempting to decode

    certain

    UTF-16 chars




Temporary tags have been created (they

        can be

    removed if

    the vote

    is not

approved) :

* MINA 2.1.9:

https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60>
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60>>
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60>
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60>>>
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60>
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60>>
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60>
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60
<
https://github.com/apache/mina/commit/8df31da1597056b73f5d6dbf11c75ce13227ba60


* MINA 2.0.26:

https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49>
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49>>
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49>
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49>>>
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49>
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49>>
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49>
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49
<
https://github.com/apache/mina/commit/4d1cf35024ae565827b63c11bd0b42a62a1c3e49





The final artifacts are stored in a

        staging repository:

* MINA 2.1.9:

https://repository.apache.org/content/repositories/orgapachemina-1105 <
https://repository.apache.org/content/repositories/orgapachemina-1105> <
https://repository.apache.org/content/repositories/orgapachemina-1105 <
https://repository.apache.org/content/repositories/orgapachemina-1105>> <
https://repository.apache.org/content/repositories/orgapachemina-1105 <
https://repository.apache.org/content/repositories/orgapachemina-1105> <
https://repository.apache.org/content/repositories/orgapachemina-1105 <
https://repository.apache.org/content/repositories/orgapachemina-1105>>> <
https://repository.apache.org/content/repositories/orgapachemina-1105 <
https://repository.apache.org/content/repositories/orgapachemina-1105> <
https://repository.apache.org/content/repositories/orgapachemina-1105 <
https://repository.apache.org/content/repositories/orgapachemina-1105>> <
https://repository.apache.org/content/repositories/orgapachemina-1105 <
https://repository.apache.org/content/repositories/orgapachemina-1105> <
https://repository.apache.org/content/repositories/orgapachemina-1105 <
https://repository.apache.org/content/repositories/orgapachemina-1105>>>>

* MINA 2.0.26:

https://repository.apache.org/content/repositories/orgapachemina-1104 <
https://repository.apache.org/content/repositories/orgapachemina-1104> <
https://repository.apache.org/content/repositories/orgapachemina-1104 <
https://repository.apache.org/content/repositories/orgapachemina-1104>> <
https://repository.apache.org/content/repositories/orgapachemina-1104 <
https://repository.apache.org/content/repositories/orgapachemina-1104> <
https://repository.apache.org/content/repositories/orgapachemina-1104 <
https://repository.apache.org/content/repositories/orgapachemina-1104>>> <
https://repository.apache.org/content/repositories/orgapachemina-1104 <
https://repository.apache.org/content/repositories/orgapachemina-1104> <
https://repository.apache.org/content/repositories/orgapachemina-1104 <
https://repository.apache.org/content/repositories/orgapachemina-1104>> <
https://repository.apache.org/content/repositories/orgapachemina-1104 <
https://repository.apache.org/content/repositories/orgapachemina-1104> <
https://repository.apache.org/content/repositories/orgapachemina-1104 <
https://repository.apache.org/content/repositories/orgapachemina-1104>>>>




The distributions are available for

        download on :

* MINA 2.1.9:

        https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9
        <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9>

    <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9

        <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9>>

          <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9
        <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9>

    <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9

        <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9>>>

      <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9

        <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9>

    <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9

        <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9>>

          <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9
        <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9>

    <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9

        <https://dist.apache.org/repos/dist/dev/mina/mina/2.1.9>>>>

* MINA 2.0.26:

        https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26
        <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26>

    <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26

        <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26>>

          <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26
        <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26>

    <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26

        <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26>>>

          <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26
        <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26>

    <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26

        <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26>>

          <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26
        <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26>

    <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26

        <https://dist.apache.org/repos/dist/dev/mina/mina/2.0.26>>>>



Let us vote :
[ ] +1 | Release MINA 2.1.9
[ ] ± | Abstain
[ ] -1 | Do *NOT* release MINA 2.1.9


[ ] +1 | Release MINA 2.0.26
[ ] ± | Abstain
[ ] -1 | Do *NOT* release MINA 2.0.26


    --
    *Emmanuel Lécharny* P. +33 (0)6 08 33 32 61
elecha...@apache.org

        <mailto:elecha...@apache.org> <mailto:elecha...@apache.org
        <mailto:elecha...@apache.org>>

    <mailto:elecha...@apache.org

        <mailto:elecha...@apache.org> <mailto:elecha...@apache.org
        <mailto:elecha...@apache.org>>>

    <mailto:elecha...@apache.org

        <mailto:elecha...@apache.org> <mailto:elecha...@apache.org
        <mailto:elecha...@apache.org>>

    <mailto:elecha...@apache.org

        <mailto:elecha...@apache.org> <mailto:elecha...@apache.org
        <mailto:elecha...@apache.org>>>>

---------------------------------------------------------------------

    To unsubscribe, e-mail:

dev-unsubscr...@mina.apache.org

        <mailto:dev-unsubscr...@mina.apache.org>
        <mailto:dev-unsubscr...@mina.apache.org
        <mailto:dev-unsubscr...@mina.apache.org>>

    <mailto:dev-unsubscr...@mina.apache.org

        <mailto:dev-unsubscr...@mina.apache.org>

    <mailto:dev-unsubscr...@mina.apache.org

        <mailto:dev-unsubscr...@mina.apache.org>>>

    <mailto:dev-unsubscr...@mina.apache.org

        <mailto:dev-unsubscr...@mina.apache.org>

    <mailto:dev-unsubscr...@mina.apache.org

        <mailto:dev-unsubscr...@mina.apache.org>>

    <mailto:dev-unsubscr...@mina.apache.org

        <mailto:dev-unsubscr...@mina.apache.org>

    <mailto:dev-unsubscr...@mina.apache.org

        <mailto:dev-unsubscr...@mina.apache.org>>>>

    For additional commands, e-mail:

dev-h...@mina.apache.org <mailto:dev-h...@mina.apache.org>

        <mailto:dev-h...@mina.apache.org <mailto:

dev-h...@mina.apache.org>>

    <mailto:dev-h...@mina.apache.org

        <mailto:dev-h...@mina.apache.org>

    <mailto:dev-h...@mina.apache.org

        <mailto:dev-h...@mina.apache.org>>>

    <mailto:dev-h...@mina.apache.org

        <mailto:dev-h...@mina.apache.org>

    <mailto:dev-h...@mina.apache.org

        <mailto:dev-h...@mina.apache.org>>

    <mailto:dev-h...@mina.apache.org

        <mailto:dev-h...@mina.apache.org>

    <mailto:dev-h...@mina.apache.org

        <mailto:dev-h...@mina.apache.org>>>>


    --
    *Emmanuel Lécharny* P. +33 (0)6 08 33 32 61
elecha...@apache.org <mailto:elecha...@apache.org>

        <mailto:elecha...@apache.org <mailto:elecha...@apache.org>>

    <mailto:elecha...@apache.org

        <mailto:elecha...@apache.org> <mailto:elecha...@apache.org
        <mailto:elecha...@apache.org>>>


    --
    *Emmanuel Lécharny* P. +33 (0)6 08 33 32 61
elecha...@apache.org <mailto:elecha...@apache.org>

        <mailto:elecha...@apache.org <mailto:elecha...@apache.org>>


        --
        *Emmanuel Lécharny* P. +33 (0)6 08 33 32 61
        elecha...@apache.org <mailto:elecha...@apache.org>

---------------------------------------------------------------------

        To unsubscribe, e-mail: dev-unsubscr...@mina.apache.org
        <mailto:dev-unsubscr...@mina.apache.org>
        For additional commands, e-mail: dev-h...@mina.apache.org
        <mailto:dev-h...@mina.apache.org>



    --
    ------------------------
    Guillaume Nodet


--
*Emmanuel Lécharny* P. +33 (0)6 08 33 32 61
elecha...@apache.org


--
*Emmanuel Lécharny* P. +33 (0)6 08 33 32 61
elecha...@apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@mina.apache.org
For additional commands, e-mail: dev-h...@mina.apache.org

Re: Reminder: [VOTE] Apache 2.1.9 and 2.0.26 releases

Reply via email to