[
https://issues.apache.org/jira/browse/AVRO-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204670#comment-17204670
]
Julian Wexler commented on AVRO-2519:
-------------------------------------
The problem actually IS in the {{NettyTransceiver}}: if an exception occurs
during the constructor, the {{NioClientSocketChannelFactory}} is not properly
closed.
One possible workaround:
{code:java}
NettyTransceiver client = null;
NioClientSocketChannelFactory factory = new NioClientSocketChannelFactory(
Executors.newCachedThreadPool(new
NettyTransceiver.NettyTransceiverThreadFactory("Avro Netty Boss")),
Executors.newCachedThreadPool(new
NettyTransceiver.NettyTransceiverThreadFactory("Avro Netty I/O")));
while (true) {
try {
client = new NettyTransceiver(new
InetSocketAddress(InetAddress.getByName("127.0.0.1"), 7999), factory, 3000L);
}
catch (Exception e) {
e.printStackTrace();
}
finally {
if (client != null) client.close();
}
}
factory.shutdown();
{code}
does not leak resources and can run like forever on my machine.
The leak in question is no longer present in 1.10 (due to migration from netty
3 to netty 4, and replaced by another resource leak), but I could help provide
a patch for 1.9, if there is going to be a 1.9.3 release at all.
> Use NettyTransceiver repeatedly connect to remote server which doesn't
> support current RPC or unavailable will cause " java.lang.OutOfMemoryError:
> Direct buffer memory"
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: AVRO-2519
> URL: https://issues.apache.org/jira/browse/AVRO-2519
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.9.0
> Environment: java version "12.0.1" 2019-04-16
> Java(TM) SE Runtime Environment (build 12.0.1+12)
> Java HotSpot(TM) 64-Bit Server VM (build 12.0.1+12, mixed mode, sharing)
> Reporter: Xun Wang
> Priority: Critical
> Attachments: avro-rpc-quickstart test.zip
>
>
> I just modify the code from
> https://github.com/phunt/avro-rpc-quickstart。Execute the modified code can
> 100% reproduce this issue. I have attached the test project file. When use
> NettyTransceiver repeatedly connect to remote server which doesn't support
> current RPC or unavailable will cause " java.lang.OutOfMemoryError: Direct
> buffer memory"
> The source to execute:
> /**
> * Licensed to the Apache Software Foundation (ASF) under one
> * or more contributor license agreements. See the NOTICE file
> * distributed with this work for additional information
> * regarding copyright ownership. The ASF licenses this file
> * to you under the Apache License, Version 2.0 (the
> * "License"); you may not use this file except in compliance
> * with the License. You may obtain a copy of the License at
> *
> * http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
>
> package example;
>
> import java.io.IOException;
> import java.net.InetAddress;
> import java.net.InetSocketAddress;
>
> import org.apache.avro.ipc.netty.NettyServer;
> import org.apache.avro.ipc.netty.NettyTransceiver;
> import org.apache.avro.ipc.Server;
> import org.apache.avro.ipc.specific.SpecificRequestor;
> import org.apache.avro.ipc.specific.SpecificResponder;
> import org.apache.avro.util.Utf8;
>
> import example.proto.Mail;
> import example.proto.Message;
>
> /**
> * Start a server, attach a client, and send a message.
> */
> public class Main {
> public static class MailImpl implements Mail {
> // in this simple example just return details of the message
> public Utf8 send(Message message) {
> System.out.println("Sending message");
> return new Utf8("Sending message to " + message.getTo().toString() + " from "
> + message.getFrom().toString()
> + " with body " + message.getBody().toString());
> }
> }
>
> private static Server server;
>
> private static void startServer() throws IOException {
> server = new NettyServer(new SpecificResponder(Mail.class, new MailImpl()),
> new InetSocketAddress(65111));
> // the server implements the Mail protocol (MailImpl)
> }
>
> public static void main(String[] args) throws IOException {
> // if (args.length != 3) {
> // System.out.println("Usage: <to> <from> <body>");
> // System.exit(1);
> // }
>
> System.out.println("Starting server");
> // usually this would be another app, but for simplicity
> // startServer();
> System.out.println("Server started");
>
> while (true) {
> NettyTransceiver client = null;
> try {
> client = new NettyTransceiver(new
> InetSocketAddress(InetAddress.getByName("129.28.206.27"), 7999), 3000l);
> // client code - attach to the server and send a message
> Mail proxy = (Mail) SpecificRequestor.getClient(Mail.class, client);
> System.out.println("Client built, got proxy");
>
> // fill in the Message record and send it
> Message message = new Message();
> message.setTo(new Utf8(args[0]));
> message.setFrom(new Utf8(args[1]));
> message.setBody(new Utf8(args[2]));
> System.out.println("Calling proxy.send with message: " + message.toString());
> System.out.println("Result: " + proxy.send(message));
> } catch (Exception e) {
> System.out.println(e.getMessage());
> } finally {
> // cleanup
> if(client != null) {
> client.close();
> }
> }
>
> }
>
> // server.close();
> }
> }
> The error log:
> Starting server
> Server started
> Error connecting to /129.28.206.27:7999
> Error connecting to /129.28.206.27:7999
> Error connecting to /129.28.206.27:7999
> Error connecting to /129.28.206.27:7999
> Error connecting to /129.28.206.27:7999
> Error connecting to /129.28.206.27:7999
> Error connecting to /129.28.206.27:7999
> Exception in thread "main" java.lang.OutOfMemoryError: Direct buffer memory
> at java.base/java.nio.Bits.reserveMemory(Bits.java:175)
> at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118)
> at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
> at
> org.jboss.netty.channel.socket.nio.SocketSendBufferPool$Preallocation.<init>(SocketSendBufferPool.java:156)
> at
> org.jboss.netty.channel.socket.nio.SocketSendBufferPool.<init>(SocketSendBufferPool.java:42)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:45)
> at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
> at
> org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)
> at
> org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)
> at
> org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
> at
> org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.<init>(NioClientSocketChannelFactory.java:151)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.<init>(NioClientSocketChannelFactory.java:116)
> at
> org.apache.avro.ipc.netty.NettyTransceiver.<init>(NettyTransceiver.java:127)
> at example.Main.main(Main.java:69)
>
> The JVM parameter：
> -XX:MaxDirectMemorySize=8m -Dio.netty.leakDetection.level=advanced
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
