[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988737#comment-15988737 ] Konstantin Dudkov commented on IGNITE-3054: --- I add additional test with real channel > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Semen Boikov > Fix For: 2.1 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15986837#comment-15986837 ] Konstantin Dudkov commented on IGNITE-3054: --- completely removed NioSslSocket wrapper, modified test > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Semen Boikov > Fix For: 2.1 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15984885#comment-15984885 ] Konstantin Dudkov commented on IGNITE-3054: --- Implemented input/ouput streams in BlockingSslHandler and test for it > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Semen Boikov > Fix For: 2.1 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568031#comment-15568031 ] Dmitry Karachentsev commented on IGNITE-3054: - Semen, thanks for review, added your recommendations to onDisconnected() method. The case you worry about could happen only if message was received for disconnected client, so we can simply drop it (I've added new check). This behavior is provided by connection logic that will not add new worker if there was not stopped current one. Also I need access to worker from listener, because there are operations that require to change it's state. > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Dmitry Karachentsev > Fix For: 1.8 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561816#comment-15561816 ] Dmitry Karachentsev commented on IGNITE-3054: - Fixed all issues and tested on grid with 1 server and 400 clients. > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Dmitry Karachentsev > Fix For: 1.8 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15507060#comment-15507060 ] Dmitry Karachentsev commented on IGNITE-3054: - Fixed race, but haven't found the root case of restart suites overtime. The same issues can be observed on master too, f.e. that tests terminated by timeout and sometimes they couldn't be stopped due to uninterruptible wait. No new failures found. > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Dmitry Karachentsev > Fix For: 1.8 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15496787#comment-15496787 ] Dmitry Karachentsev commented on IGNITE-3054: - Fixed BlockingSslHandler and client discovery message processing. > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Dmitry Karachentsev > Fix For: 1.8 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490621#comment-15490621 ] Dmitry Karachentsev commented on IGNITE-3054: - All issues were fixed, now discovery meets all time requirements. Reran all TC tests to recheck. > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Dmitry Karachentsev > Fix For: 1.8 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15487839#comment-15487839 ] Dmitry Karachentsev commented on IGNITE-3054: - * Fixed bugs that lead to ExecutionTimeout of the tests. * Now SSL works OK, but on Linux machine often receive exceptions on handshake, like: *"javax.net.ssl.SSLException: Received fatal alert: "* with random codes, or real reasons, but it looks like data mess on connection process. Exactly the same issue like [here|http://stackoverflow.com/questions/34671460/getting-various-sslexceptions-when-running-loadtest-on-an-ssl-enabled-url-call]. Trying to understand the nature of that error. > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Dmitry Karachentsev > Fix For: 1.8 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15484760#comment-15484760 ] Dmitry Karachentsev commented on IGNITE-3054: - * Increased SSL connection establishing speed, now it's just a bit slower than old implementation. * NIO implementation works slower in general (for client nodes and SSL) that causes SPI tests exceed 50 min limit. Working on this. * SSL still not fully stable, sometimes is thrown "SocketException: Software caused connection abort". Investigating. > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Dmitry Karachentsev > Fix For: 1.8 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15481566#comment-15481566 ] Dmitry Karachentsev commented on IGNITE-3054: - * Fixed review notes except the first one, because GridNioSession is always the same: that passed in onMessage() or used in right ClientNioMessageWorker. * Fixed SPI tests (added new method to TcpDiscoverySpi to handle sending messages to Session). * Some small fixes. * Working on failing SSL tests, error happens during SSL handshake. > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Dmitry Karachentsev > Fix For: 1.8 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318855#comment-15318855 ] Dmitry Karachentsev commented on IGNITE-3054: - 1. Implemented SSL support. 2. Fixed bugs. 3. Checked resource consuming. Actually I've just started server node and 200 client nodes, but don't see any difference in memory (probably there weren't counted threads stacks in Visual VM) or CPU consumption. Significant difference is observed only in number of live threads (as expected). No message processing velocity was measured. Here are some numbers: Initial (no clients): Threads: 76 Heap: 50 M Master (200 clients): Threads: 486 Heap: 120 M NIO (200 clients): Threads: 93 Heap: 100 M 4. Verified compatibility with previous version (master branch used). 5. Added pretty ugly workaround: Sometimes OutputStream from NIO socket may block thread infinitely, so in this case used SocketChannel directly. 6. Now trying to resolve bug that causes to fail random tests. On server start when SocketServerChannel.bind() is called it sometimes throws SocketException from native bind0() method: java.net.SocketException: Invalid argument at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:463) at sun.nio.ch.Net.bind(Net.java:455) ... I'm trying to find it's implementation for Linux but still no luck. Continue investigating. > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Dmitry Karachentsev >Priority: Blocker > Fix For: 1.7 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308043#comment-15308043 ] Dmitry Karachentsev commented on IGNITE-3054: - Working on SSL support. To keep it compatible, and in same time support switching clients to unblocking mode, there needs to be added SSL handshake support from GridNioSslHandler and encrypt/decrypt methods for server connections. This solution picked because of it's unable to use GridNioServer, it's designed to work in non-blocking mode, but we want to keep current server communication logic untouched (use blocking mode and thread per connection). In previous implementation SSLServerSocketFactory was used, which is not applicable for NIO. Left: * SSL support. * Performance tests. > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Dmitry Karachentsev >Priority: Blocker > Fix For: 1.7 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15306756#comment-15306756 ] Dmitry Karachentsev commented on IGNITE-3054: - Made some refactoring and changed strategy for switching to async mode: used flag on handshake procedure. This allows correctly handle client reconnects and doesn't brake compatibility. > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Dmitry Karachentsev >Priority: Blocker > Fix For: 1.7 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-3054) Rework client connection handling from thread-per-client to NIO model.
[ https://issues.apache.org/jira/browse/IGNITE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15306295#comment-15306295 ] Dmitry Karachentsev commented on IGNITE-3054: - Implemented async NIO. Now clients join topology and communicate via NIO selector. Server, on node join request, decides to switch into async mode, according to client version, and responds to client with recipe status OK v2. After switch client starts adding message length to each packet to make server be able to assembly it from chunks. With these changes now is avoided supplying of two threads: client message worker and socket reader, that were created for each client connection before. Approx. roadmap for task: * Handle correctly client reconnect, or make sure if it's not necessary due to establishing new connection. * Refactor and obvious optimizations. * Adding SSL support (seems not too difficult - just add SSL filter to GridNioServer). * Performance tests. > Rework client connection handling from thread-per-client to NIO model. > -- > > Key: IGNITE-3054 > URL: https://issues.apache.org/jira/browse/IGNITE-3054 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 1.5.0.final >Reporter: Vladimir Ozerov >Assignee: Dmitry Karachentsev >Priority: Blocker > Fix For: 1.7 > > > Currently both servers and clients has the same operational model - > thread-per-connection. While being more or less fine for servers, this could > be a problem for clients when their total number is too high (e.g. 1000 or > even more). > We should rework client handling model and employ standard NIO technique: one > or several acceptor threads + thread pool to server requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
