Hi, Shekar, Please see my replies below:
On Thu, Aug 4, 2016 at 12:18 PM, Shekar Tippur <ctip...@gmail.com> wrote: > Sorry .. got trigger happy. > Couple of observations: > 1. rmi port seem to be random ones Even with task.opts and setting the port > to 1099 > You are right. Samza containers will choose a random port to start the JmxServer. We do have an issue w/ port conflicts in 0.10.0 and earlier versions. It is recently fixed in 0.10.1. The 0.10.1 RC is out for the vote now. It would be good if you can try it out. However, your exception stack trace indicates that it is not a port conflict, but a refusal of connection. Did you try telnet to localhost to see whether it can open the connection? > 2. Do I need to start rmiregistry manually? > No, you shouldn't need to start any other process manually for that. P.S. do you have a full log from your SamzaAppMaster? That would be helpful in debugging this issue. > 3. I dont have any firewall settings. > > On Thu, Aug 4, 2016 at 12:14 PM, Shekar Tippur <ctip...@gmail.com> wrote: > > > Yi > > Thanks for responding.. I have tried diff options for task.opts and > > without it as well. > > I have tried this > > > > task.opts=-Dcom.sun.management.jmxremote -Dcom.sun.management. > jmxremote.port=1099 > > -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management. > jmxremote.ssl=false > > -Djava.rmi.server.hostname=localhost > > > > > > > > Here is the property file: > > ########################################################## > > > > # Licensed to the Apache Software Foundation (ASF) under one > > > > # or more contributor license agreements. See the NOTICE file > > > > # distributed with this work for additional information > > > > # regarding copyright ownership. The ASF licenses this file > > > > # to you under the Apache License, Version 2.0 (the > > > > # "License"); you may not use this file except in compliance > > > > # with the License. You may obtain a copy of the License at > > > > # > > > > # http://www.apache.org/licenses/LICENSE-2.0 > > > > # > > > > # Unless required by applicable law or agreed to in writing, > > > > # software distributed under the License is distributed on an > > > > # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY > > > > # KIND, either express or implied. See the License for the > > > > # specific language governing permissions and limitations > > > > # under the License. > > > > > > job.factory.class=org.apache.samza.job.yarn.YarnJobFactory > > > > job.name=Argos > > > > #job.coordinator.system=kafka > > > > #job.coordinator.replication.factor=2 > > > > # YARN > > > > yarn.package.path=http://localhost:8000/argos_aggregate.tar.gz > > > > #task.opts="-Dcom.sun.management.jmxremote -Dcom.sun.management. > jmxremote.port=7091 > > -Dcom.sun.management.jmxremote.authenticate=false > > -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=loc > > alhost" > > > > # Task > > > > task.class=com.project.argos.task.ArgosParserStreamTask > > > > task.inputs=kafka.argos-raw > > > > task.outputs=argos-parser > > > > task.checkpoint.factory=org.apache.samza.checkpoint.kafka.Ka > > fkaCheckpointManagerFactory > > > > task.checkpoint.system=kafka > > > > # Normally, this would be 3, but we have only one broker. > > > > task.checkpoint.replication.factor=2 > > > > task.drop.deserialization.errors=true > > > > task.drop.serialization.errors=true > > > > task.ignored.exceptions=org.codehaus.jackson.JsonParseException > > > > > > # Samza10 properties > > > > #systems.kafka.producer.bootstrap.servers=localhost:6667 > > > > systems.kafka.producer.bootstrap.servers=host1:6667,host1:6667 > > > > job.coordinator.system=kafka > > > > > > > > # Metrics > > > > metrics.reporters=snapshot,jmx > > > > metrics.reporter.snapshot.class=org.apache.samza.metrics. > > reporter.MetricsSnapshotReporterFactory > > > > metrics.reporter.snapshot.stream=kafka.metrics > > > > metrics.reporter.jmx.class=org.apache.samza.metrics.reporter > > .JmxReporterFactory > > > > > > # Serializers > > > > serializers.registry.json.class=org.apache.samza.serializers > > .JsonSerdeFactory > > > > serializers.registry.string.class=org.apache.samza.serialize > > rs.StringSerdeFactory > > > > serializers.registry.metrics.class=org.apache.samza.serializ > > ers.MetricsSnapshotSerdeFactory > > > > > > # Systems > > > > systems.kafka.samza.factory=org.apache.samza.system.kafka.Ka > > fkaSystemFactory > > > > systems.kafka.samza.msg.serde=json > > > > > > systems.kafka.consumer.zookeeper.connect=host1:2181, > host2:2181,host3:2181 > > > > systems.kafka.consumer.auto.offset.reset=largest > > > > systems.kafka.producer.producer.type=sync > > > > # Normally, we'd set this much higher, but we want things to look snappy > > in the demo. > > > > systems.kafka.producer.batch.num.messages=1 > > > > systems.kafka.streams.metrics.samza.msg.serde=metrics > > > > systems.kafka.streams.ArgosRaw.samza.msg.serde=json > > > > systems.kafka.streams.PageViewEvent.samza.msg.serde=json > > > > > > > > # enable wildcard lookup on cache misses, will impact performance > > > > redis.wildcard.lookup=true > > > > > > # Property File > > > > load.property=true > > > > > > > > system.redis=host4 > > > > systems.kafka.producer.metadata.broker.list=host1:6667host2:6667 > > > > ########################################################## > > > > On Thu, Aug 4, 2016 at 11:19 AM, Yi Pan <nickpa...@gmail.com> wrote: > > > >> Hi, Shekar, > >> > >> Did you check your firewall configuration? Could you also paste your > >> configuration, especially task.opts? > >> > >> -Yi > >> > >> On Wed, Aug 3, 2016 at 5:56 PM, Shekar Tippur <ctip...@gmail.com> > wrote: > >> > >> > I am trying to submit a Samza job to yarn and I get a error: > >> > > >> > Exception in thread "main" java.io.IOException: Cannot bind to URL > >> > [rmi://localhost:44960/jmxrmi]: javax.naming.ServiceUnavailabl > >> eException > >> > [Root exception is java.rmi.ConnectException: Connection refused to > >> host: > >> > localhost; nested exception is: > >> > > >> > java.net.ConnectException: Connection refused] > >> > > >> > at > >> > javax.management.remote.rmi.RMIConnectorServer.newIOException( > >> > RMIConnectorServer.java:827) > >> > > >> > at > >> > javax.management.remote.rmi.RMIConnectorServer.start( > >> > RMIConnectorServer.java:432) > >> > > >> > at org.apache.samza.metrics.JmxServer.<init>(JmxServer. > scala:89 > >> ) > >> > > >> > at org.apache.samza.metrics.JmxServer.<init>(JmxServer. > scala:43 > >> ) > >> > > >> > at > >> > org.apache.samza.job.yarn.SamzaAppMaster$.main( > SamzaAppMaster.scala:87) > >> > > >> > at > >> > org.apache.samza.job.yarn.SamzaAppMaster.main(SamzaAppMaster.scala) > >> > > >> > Caused by: javax.naming.ServiceUnavailableException [Root exception > is > >> > java.rmi.ConnectException: Connection refused to host: localhost; > nested > >> > exception is: > >> > > >> > java.net.ConnectException: Connection refused] > >> > > >> > at > >> > com.sun.jndi.rmi.registry.RegistryContext.bind(RegistryConte > >> xt.java:147) > >> > > >> > at > >> > com.sun.jndi.toolkit.url.GenericURLContext.bind( > >> > GenericURLContext.java:228) > >> > > >> > at javax.naming.InitialContext.bind(InitialContext.java:425) > >> > > >> > at > >> > javax.management.remote.rmi.RMIConnectorServer.bind( > >> > RMIConnectorServer.java:644) > >> > > >> > at > >> > javax.management.remote.rmi.RMIConnectorServer.start( > >> > RMIConnectorServer.java:427) > >> > > >> > ... 4 more > >> > > >> > Caused by: java.rmi.ConnectException: Connection refused to host: > >> > localhost; nested exception is: > >> > > >> > java.net.ConnectException: Connection refused > >> > > >> > at sun.rmi.transport.tcp.TCPEndpoint.newSocket( > >> > TCPEndpoint.java:619) > >> > > >> > at > >> > sun.rmi.transport.tcp.TCPChannel.createConnection( > TCPChannel.java:216) > >> > > >> > at > >> > sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) > >> > > >> > at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:342) > >> > > >> > at sun.rmi.registry.RegistryImpl_Stub.bind(Unknown Source) > >> > > >> > at > >> > com.sun.jndi.rmi.registry.RegistryContext.bind(RegistryConte > >> xt.java:141) > >> > > >> > ... 8 more > >> > > >> > Caused by: java.net.ConnectException: Connection refused > >> > > >> > at java.net.PlainSocketImpl.socketConnect(Native Method) > >> > > >> > at > >> > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSock > >> etImpl.java: > >> > 350) > >> > > >> > at > >> > java.net.AbstractPlainSocketImpl.connectToAddress( > >> > AbstractPlainSocketImpl.java:206) > >> > > >> > at > >> > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocket > >> Impl.java:188) > >> > > >> > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > >> > > >> > at java.net.Socket.connect(Socket.java:589) > >> > > >> > at java.net.Socket.connect(Socket.java:538) > >> > > >> > at java.net.Socket.<init>(Socket.java:434) > >> > > >> > at java.net.Socket.<init>(Socket.java:211) > >> > > >> > at > >> > sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket( > >> > RMIDirectSocketFactory.java:40) > >> > > >> > at > >> > sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket( > >> > RMIMasterSocketFactory.java:148) > >> > > >> > at sun.rmi.transport.tcp.TCPEndpoint.newSocket( > >> > TCPEndpoint.java:613) > >> > > >> > ... 13 more > >> > > >> > Any idea on what could be the issue? > >> > > >> > - Shekar > >> > > >> > > > > >
