lingbin commented on a change in pull request #2603: Add nio support for mysql protocol implementation URL: https://github.com/apache/incubator-doris/pull/2603#discussion_r362178093
########## File path: fe/src/main/java/org/apache/doris/mysql/nio/NMysqlChannel.java ########## @@ -0,0 +1,111 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +package org.apache.doris.mysql.nio; + +import org.apache.doris.mysql.MysqlChannel; +import org.apache.doris.qe.ConnectProcessor; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.xnio.StreamConnection; +import org.xnio.channels.Channels; + +import java.io.IOException; +import java.net.InetSocketAddress; +import java.nio.ByteBuffer; + +/** + * mysql Channel based on nio. + */ +public class NMysqlChannel extends MysqlChannel { + protected final Logger LOG = LogManager.getLogger(this.getClass()); + private StreamConnection conn; + + public NMysqlChannel(StreamConnection connection) { + super(); + this.conn = connection; + if (connection.getPeerAddress() instanceof InetSocketAddress) { + InetSocketAddress address = (InetSocketAddress) connection.getPeerAddress(); + remoteHostPortString = address.getHostString() + ":" + address.getPort(); + remoteIp = address.getAddress().getHostAddress(); + } else { + // Reach here, what's it? + remoteHostPortString = connection.getPeerAddress().toString(); + remoteIp = connection.getPeerAddress().toString(); + } + } + + /** + * read packet until all data is ready, unless block. + * + * @param dstBuf + * @return + * @throws IOException + */ + @Override + protected int readAll(ByteBuffer dstBuf) throws IOException { + int ret = Channels.readBlocking(conn.getSourceChannel(), dstBuf); + // return -1 when remote peer close the channel + if (ret == -1) { + return 0; + } + return ret; Review comment: There may be a problem here. When the MySQL request message is large, it will be split into multiple TCP packets, but it is only read once here, and `readBlocking()` does not guarantee that the content of the parameter `dstBuf` is filled(see [jboss doc](https://docs.jboss.org/xnio/3.1/api/org/xnio/channels/Channels.html#readBlocking(C,%20java.nio.ByteBuffer))), so it is easy causes fewer data to be read. In other words, a complete MySQL logical packet can not be read, which will cause an error to occur in the upper layer. For example, when the local TCP buffer is full, the remaining part of the logical packet will only be reached after reading the data that has arrived first, and then it can be read. Here, the length we want to read should be `dstBuf.remaining()`. A simple solution to achieve this is to add a loop logic(like the existing implementation), to make sure this function will not return until enough data is read, but this may take a long time for the IO thread because it is not sure what time the remaining packets are will arrive. However, the problem is not big in practice. A slightly more complicated but better solution is to try to read every time a readable event arrives(likes what you did now). If there is not a complete logical packet, then just record the data. Processing is not done until a logical packet has been read, but this may require slightly larger changes. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
