Thanks Aldrin/Corey. I will try it tomorrow morning. From: Aldrin Piri <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Tuesday, September 29, 2015 at 6:44 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: nifi Cluster setup issue
Oops, definitely missed what Corey sent out. Please specify the nifi.cluster.manager.address as he suggests. On Tue, Sep 29, 2015 at 9:40 PM, Aldrin Piri <[email protected]<mailto:[email protected]>> wrote: Chakrader, You would also need to set the nifi.web.http.host for the manager as well. Each member of the cluster provides how they can be accessed in the protocol. This would explain what you are seeing in the node from the master/manager. Please try also setting the manager and let us know if this gets your cluster up and running. On Tue, Sep 29, 2015 at 8:48 PM, Chakrader Dewaragatla <[email protected]<mailto:[email protected]>> wrote: Aldrin - I redeployed with nifi with default settings and modified the required settings needed for cluster setup documented in https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html. I tried to change nifi.web.http.host property on Node (slave) with its ip. On slave I notice following error: 2015-09-30 00:44:21,855 INFO [main] o.a.nifi.controller.StandardFlowService Connecting Node: [id=75baec43-adf1-4e17-98fd-49111a5a0c76, apiAddress=10.233.2.42, apiPort=8080, socketAddress=10.233.2.42, socketPort=3002] On Master: As usual: 2015-09-30 00:44:51,036 INFO [Process Pending Heartbeats] org.apache.nifi.cluster.heartbeat Received heartbeat for node [id=644370b1-4d8f-4004-ac6c-8bd614a1890b, apiAddress=localhost, apiPort=8080, socketAddress=10.233.2.42, socketPort=3002]. Here is my complete conf file : Master conf file: # Core Properties # nifi.version=0.3.0 nifi.flow.configuration.file=./conf/flow.xml.gz nifi.flow.configuration.archive.dir=./conf/archive/ nifi.flowcontroller.autoResumeState=true nifi.flowcontroller.graceful.shutdown.period=10 sec nifi.flowservice.writedelay.interval=500 ms nifi.administrative.yield.duration=30 sec # If a component has no work to do (is "bored"), how long should we wait before checking again for work? nifi.bored.yield.duration=10 millis nifi.authority.provider.configuration.file=./conf/authority-providers.xml nifi.templates.directory=./conf/templates nifi.ui.banner.text= nifi.ui.autorefresh.interval=30 sec nifi.nar.library.directory=./lib nifi.nar.working.directory=./work/nar/ nifi.documentation.working.directory=./work/docs/components # H2 Settings nifi.database.directory=./database_repository nifi.h2.url.append=;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE # FlowFile Repository nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository nifi.flowfile.repository.directory=./flowfile_repository nifi.flowfile.repository.partitions=256 nifi.flowfile.repository.checkpoint.interval=2 mins nifi.flowfile.repository.always.sync=false nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager nifi.queue.swap.threshold=20000 nifi.swap.in.period=5 sec nifi.swap.in.threads=1 nifi.swap.out.period=5 sec nifi.swap.out.threads=4 # Content Repository nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository nifi.content.claim.max.appendable.size=10 MB nifi.content.claim.max.flow.files=100 nifi.content.repository.directory.default=./content_repository nifi.content.repository.archive.max.retention.period=12 hours nifi.content.repository.archive.max.usage.percentage=50% nifi.content.repository.archive.enabled=true nifi.content.repository.always.sync=false nifi.content.viewer.url=/nifi-content-viewer/ # Provenance Repository Properties nifi.provenance.repository.implementation=org.apache.nifi.provenance.PersistentProvenanceRepository # Persistent Provenance Repository Properties nifi.provenance.repository.directory.default=./provenance_repository nifi.provenance.repository.max.storage.time=24 hours nifi.provenance.repository.max.storage.size=1 GB nifi.provenance.repository.rollover.time=30 secs nifi.provenance.repository.rollover.size=100 MB nifi.provenance.repository.query.threads=2 nifi.provenance.repository.index.threads=1 nifi.provenance.repository.compress.on.rollover=true nifi.provenance.repository.always.sync=false nifi.provenance.repository.journal.count=16 # Comma-separated list of fields. Fields that are not indexed will not be searchable. Valid fields are: # EventType, FlowFileUUID, Filename, TransitURI, ProcessorID, AlternateIdentifierURI, ContentType, Relationship, Details nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename, ProcessorID, Relationship # FlowFile Attributes that should be indexed and made searchable nifi.provenance.repository.indexed.attributes= # Large values for the shard size will result in more Java heap usage when searching the Provenance Repository # but should provide better performance nifi.provenance.repository.index.shard.size=500 MB # Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from # the repository. If the length of any attribute exceeds this value, it will be truncated when the event is retrieved. nifi.provenance.repository.max.attribute.length=65536 # Volatile Provenance Respository Properties nifi.provenance.repository.buffer.size=100000 # Component Status Repository nifi.components.status.repository.implementation=org.apache.nifi.controller.status.history.VolatileComponentStatusRepository nifi.components.status.repository.buffer.size=1440 nifi.components.status.snapshot.frequency=1 min # Site to Site properties nifi.remote.input.socket.host= nifi.remote.input.socket.port= nifi.remote.input.secure=true # web properties # nifi.web.war.directory=./lib nifi.web.http.host= nifi.web.http.port=8080 nifi.web.https.host= nifi.web.https.port= nifi.web.jetty.working.directory=./work/jetty nifi.web.jetty.threads=200 # security properties # nifi.sensitive.props.key= nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL nifi.sensitive.props.provider=BC nifi.security.keystore= nifi.security.keystoreType= nifi.security.keystorePasswd= nifi.security.keyPasswd= nifi.security.truststore= nifi.security.truststoreType= nifi.security.truststorePasswd= nifi.security.needClientAuth= nifi.security.user.credential.cache.duration=24 hours nifi.security.user.authority.provider=file-provider nifi.security.support.new.account.requests= nifi.security.ocsp.responder.url= nifi.security.ocsp.responder.certificate= # cluster common properties (cluster manager and nodes must have same values) # nifi.cluster.protocol.heartbeat.interval=5 sec nifi.cluster.protocol.is.secure=false nifi.cluster.protocol.socket.timeout=30 sec nifi.cluster.protocol.connection.handshake.timeout=45 sec # if multicast is used, then nifi.cluster.protocol.multicast.xxx properties must be configured # nifi.cluster.protocol.use.multicast=false nifi.cluster.protocol.multicast.address= nifi.cluster.protocol.multicast.port= nifi.cluster.protocol.multicast.service.broadcast.delay=500 ms nifi.cluster.protocol.multicast.service.locator.attempts=3 nifi.cluster.protocol.multicast.service.locator.attempts.delay=1 sec # cluster node properties (only configure for cluster nodes) # nifi.cluster.is.node=false nifi.cluster.node.address= nifi.cluster.node.protocol.port= nifi.cluster.node.protocol.threads=2 # if multicast is not used, nifi.cluster.node.unicast.xxx must have same values as nifi.cluster.manager.xxx # nifi.cluster.node.unicast.manager.address= nifi.cluster.node.unicast.manager.protocol.port= # cluster manager properties (only configure for cluster manager) # nifi.cluster.is.manager=true nifi.cluster.manager.address= nifi.cluster.manager.protocol.port=3001 nifi.cluster.manager.node.firewall.file= nifi.cluster.manager.node.event.history.size=10 nifi.cluster.manager.node.api.connection.timeout=30 sec nifi.cluster.manager.node.api.read.timeout=30 sec nifi.cluster.manager.node.api.request.threads=10 nifi.cluster.manager.flow.retrieval.delay=5 sec nifi.cluster.manager.protocol.threads=10 nifi.cluster.manager.safemode.duration=0 sec # kerberos # nifi.kerberos.krb5.file= Slave conf file : # Core Properties # nifi.version=0.3.0 nifi.flow.configuration.file=./conf/flow.xml.gz nifi.flow.configuration.archive.dir=./conf/archive/ nifi.flowcontroller.autoResumeState=true nifi.flowcontroller.graceful.shutdown.period=10 sec nifi.flowservice.writedelay.interval=500 ms nifi.administrative.yield.duration=30 sec # If a component has no work to do (is "bored"), how long should we wait before checking again for work? nifi.bored.yield.duration=10 millis nifi.authority.provider.configuration.file=./conf/authority-providers.xml nifi.templates.directory=./conf/templates nifi.ui.banner.text= nifi.ui.autorefresh.interval=30 sec nifi.nar.library.directory=./lib nifi.nar.working.directory=./work/nar/ nifi.documentation.working.directory=./work/docs/components # H2 Settings nifi.database.directory=./database_repository nifi.h2.url.append=;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE # FlowFile Repository nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository nifi.flowfile.repository.directory=./flowfile_repository nifi.flowfile.repository.partitions=256 nifi.flowfile.repository.checkpoint.interval=2 mins nifi.flowfile.repository.always.sync=false nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager nifi.queue.swap.threshold=20000 nifi.swap.in.period=5 sec nifi.swap.in.threads=1 nifi.swap.out.period=5 sec nifi.swap.out.threads=4 # Content Repository nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository nifi.content.claim.max.appendable.size=10 MB nifi.content.claim.max.flow.files=100 nifi.content.repository.directory.default=./content_repository nifi.content.repository.archive.max.retention.period=12 hours nifi.content.repository.archive.max.usage.percentage=50% nifi.content.repository.archive.enabled=true nifi.content.repository.always.sync=false nifi.content.viewer.url=/nifi-content-viewer/ # Provenance Repository Properties nifi.provenance.repository.implementation=org.apache.nifi.provenance.PersistentProvenanceRepository # Persistent Provenance Repository Properties nifi.provenance.repository.directory.default=./provenance_repository nifi.provenance.repository.max.storage.time=24 hours nifi.provenance.repository.max.storage.size=1 GB nifi.provenance.repository.rollover.time=30 secs nifi.provenance.repository.rollover.size=100 MB nifi.provenance.repository.query.threads=2 nifi.provenance.repository.index.threads=1 nifi.provenance.repository.compress.on.rollover=true nifi.provenance.repository.always.sync=false nifi.provenance.repository.journal.count=16 # Comma-separated list of fields. Fields that are not indexed will not be searchable. Valid fields are: # EventType, FlowFileUUID, Filename, TransitURI, ProcessorID, AlternateIdentifierURI, ContentType, Relationship, Details nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename, ProcessorID, Relationship # FlowFile Attributes that should be indexed and made searchable nifi.provenance.repository.indexed.attributes= # Large values for the shard size will result in more Java heap usage when searching the Provenance Repository # but should provide better performance nifi.provenance.repository.index.shard.size=500 MB # Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from # the repository. If the length of any attribute exceeds this value, it will be truncated when the event is retrieved. nifi.provenance.repository.max.attribute.length=65536 # Volatile Provenance Respository Properties nifi.provenance.repository.buffer.size=100000 # Component Status Repository nifi.components.status.repository.implementation=org.apache.nifi.controller.status.history.VolatileComponentStatusRepository nifi.components.status.repository.buffer.size=1440 nifi.components.status.snapshot.frequency=1 min # Site to Site properties nifi.remote.input.socket.host= nifi.remote.input.socket.port= nifi.remote.input.secure=true # web properties # nifi.web.war.directory=./lib nifi.web.http.host=10.233.2.42 nifi.web.http.port=8080 nifi.web.https.host= nifi.web.https.port= nifi.web.jetty.working.directory=./work/jetty nifi.web.jetty.threads=200 # security properties # nifi.sensitive.props.key= nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL nifi.sensitive.props.provider=BC nifi.security.keystore= nifi.security.keystoreType= nifi.security.keystorePasswd= nifi.security.keyPasswd= nifi.security.truststore= nifi.security.truststoreType= nifi.security.truststorePasswd= nifi.security.needClientAuth= nifi.security.user.credential.cache.duration=24 hours nifi.security.user.authority.provider=file-provider nifi.security.support.new.account.requests= nifi.security.ocsp.responder.url= nifi.security.ocsp.responder.certificate= # cluster common properties (cluster manager and nodes must have same values) # nifi.cluster.protocol.heartbeat.interval=5 sec nifi.cluster.protocol.is.secure=false nifi.cluster.protocol.socket.timeout=30 sec nifi.cluster.protocol.connection.handshake.timeout=45 sec # if multicast is used, then nifi.cluster.protocol.multicast.xxx properties must be configured # nifi.cluster.protocol.use.multicast=false nifi.cluster.protocol.multicast.address= nifi.cluster.protocol.multicast.port= nifi.cluster.protocol.multicast.service.broadcast.delay=500 ms nifi.cluster.protocol.multicast.service.locator.attempts=3 nifi.cluster.protocol.multicast.service.locator.attempts.delay=1 sec # cluster node properties (only configure for cluster nodes) # nifi.cluster.is.node=true nifi.cluster.node.address=10.233.2.42 nifi.cluster.node.protocol.port=3002 nifi.cluster.node.protocol.threads=2 # if multicast is not used, nifi.cluster.node.unicast.xxx must have same values as nifi.cluster.manager.xxx # nifi.cluster.node.unicast.manager.address=10.233.2.40 nifi.cluster.node.unicast.manager.protocol.port=3001 # cluster manager properties (only configure for cluster manager) # nifi.cluster.is.manager=false nifi.cluster.manager.address= nifi.cluster.manager.protocol.port= nifi.cluster.manager.node.firewall.file= nifi.cluster.manager.node.event.history.size=10 nifi.cluster.manager.node.api.connection.timeout=30 sec nifi.cluster.manager.node.api.read.timeout=30 sec nifi.cluster.manager.node.api.request.threads=10 nifi.cluster.manager.flow.retrieval.delay=5 sec nifi.cluster.manager.protocol.threads=10 nifi.cluster.manager.safemode.duration=0 sec # kerberos # nifi.kerberos.krb5.file= From: Aldrin Piri <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Tuesday, September 29, 2015 at 4:26 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: nifi Cluster setup issue Chakrader, I suspect that the nifi.web.http.host property is not using the same address as that specified and is transmitting "localhost" (the system's response to a localhost hostname lookup from Java). While the clustering protocol communicates via the properties you list, the actual command-control and replication of requests from master to slave nodes is carried out via the REST API which also runs on the web tier. The system's hostname, as previously determined, is transmitted as part of the clustering handshake. Either the system needs to have it report a valid hostname or a host needs to be specified for nifi.web.http.host. In either case of hostname or specified host, each must be network reachable from the master and able to be bound to locally within your server. Let us know if you need additional direction and we'd be happy to help you through the process. Thanks! On Tue, Sep 29, 2015 at 6:56 PM, Chakrader Dewaragatla <[email protected]<mailto:[email protected]>> wrote: Hi – We are exploring nifi for our workflow management, I have a cluster setup with 3 nodes. One as master and rest as slaves. I see following error when I try to access the nifi workflow webpage. 2015-09-29 22:46:13,263 WARN [NiFi Web Server-23] o.a.n.c.m.impl.HttpRequestReplicatorImpl Node request for [id=7481fca5-930c-4d4b-84a3-66cc62b4e2d3, apiAddress=localhost, apiPort=8080, socketAddress=localhost, socketPort=3002] encountered exception: java.util.concurrent.ExecutionException: com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection refused 2015-09-29 22:46:13,263 WARN [NiFi Web Server-23] o.a.n.c.m.impl.HttpRequestReplicatorImpl Node request for [id=0abd8295-34a3-4bf7-ab06-1b6b94014740, apiAddress=localhost, apiPort=8080, socketAddress=10.233.2.42, socketPort=3002] encountered exception: java.util.concurrent.ExecutionException: com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection refused 2015-09-29 22:46:13,264 INFO [NiFi Web Server-23] o.a.n.c.m.e.NoConnectedNodesException org.apache.nifi.cluster.manager.exception.NoResponseFromNodesException: No nodes were able to process this request.. Returning Conflict response. Master is not hybrid, I wonder why it is trying to self connect 3002. Master settings: # cluster manager properties (only configure for cluster manager) # nifi.cluster.is.manager=true nifi.cluster.manager.address=10.233.2.40 nifi.cluster.manager.protocol.port=3001 nifi.cluster.manager.node.firewall.file= nifi.cluster.manager.node.event.history.size=10 nifi.cluster.manager.node.api.connection.timeout=30 sec nifi.cluster.manager.node.api.read.timeout=30 sec nifi.cluster.manager.node.api.request.threads=10 nifi.cluster.manager.flow.retrieval.delay=5 sec nifi.cluster.manager.protocol.threads=10 nifi.cluster.manager.safemode.duration=0 sec Slave settings: # cluster node properties (only configure for cluster nodes) # nifi.cluster.is.node=true nifi.cluster.node.address=10.233.2.42 nifi.cluster.node.protocol.port=3002 nifi.cluster.node.protocol.threads=2 # if multicast is not used, nifi.cluster.node.unicast.xxx must have same values as nifi.cluster.manager.xxx # nifi.cluster.node.unicast.manager.address=10.233.2.40 nifi.cluster.node.unicast.manager.protocol.port=3001 ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________
