[
https://issues.apache.org/jira/browse/MINIFI-454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471844#comment-16471844
]
Paul Verardi commented on MINIFI-454:
-------------------------------------
We have it working now but we have three questions:
* 1: Is there a built in way for our c2 server to interface with an s3 bucket.
The use case for this being if we provision a new tenant and throw a config.yml
file into an s3 bucket, we would want the c2 server to pull down this file and
place it into the correct directory
* 2: We seem to have an issue with our certs after every ec2 reboot of our c2
server. We have to do more testing, to see the issue, but for now, we have to
regenerate the certs after every reboot, is there a known cause for this?
* 3: can c2 be used to pass a keystore/truststore to minifi agents similarly
to how it works for config.yml, for example if one expires, we wouldn't have to
manually go and replace it on all minifi agents.
> MiNiFi agent has trouble interfacing with C2
> --------------------------------------------
>
> Key: MINIFI-454
> URL: https://issues.apache.org/jira/browse/MINIFI-454
> Project: Apache NiFi MiNiFi
> Issue Type: Improvement
> Components: Agent Configuration/Installation, Command and Control
> Reporter: Aldrin Piri
> Assignee: Aldrin Piri
> Priority: Major
> Attachments: authorities.yaml, authorizations.yaml, bootstrap.conf,
> c2.properties, minifi-c2-context.xml
>
>
> A user reported issues with interfacing minifi and the c2 server in the ASF
> HipChat room.
>
> {quote}[2:25 PM] Paul Verardi: Hello
> [2:26 PM] Paul Verardi: I am having some issues connecting my MiNiFi Agent
> to my C2 Server
> [2:27 PM] Paul Verardi: If anyone is available to answer a few questions,
> that would be great
> [2:29 PM] Marc parisi: Paul, C2 server or NiFi instance?
> [2:30 PM] Paul Verardi: C2 Server
> [2:30 PM] Paul Verardi: I am trying to dynamically pull config.yml files
> from a C2 server into my Minifi agent
> [2:30 PM] Marc parisi: MiNiFi java or MiNiFi C++?
> [2:30 PM] Paul Verardi: java
> [2:30 PM] Marc parisi: Is this the C2 server that's embedded within the
> minifi java agent, @aldrin The original one?
> [2:31 PM] Paul Verardi: yes, I am running the C2 Server from the github
> project which includes the Minifi java agent
> [2:31 PM] Paul Verardi: however, my minifi java agent is from the
> hortonworks zip
> [2:31 PM] Paul Verardi: not from this github repo
> [2:31 PM] Paul Verardi: I am using this github repo for the c2 only, but
> they look to be the same
> [2:32 PM] Aldrin Piri: do you have any errors that are cropping up in the
> process
> [2:32 PM] Paul Verardi: Yessir
> [2:33 PM] Paul Verardi: 2018-05-07 17:48:28,601 WARN [pool-2-thread-1]
> o.a.n.m.b.c.i.PullHttpChangeIngestor Hit an exception while trying to pull
> java.net.SocketTimeoutException: connect timed out
> [2:33 PM] Paul Verardi: in my minifi-bootstrap.log file
> [2:33 PM] Paul Verardi: I can paste the entire stack trace if you would like
> [2:33 PM] Aldrin Piri: configs might be more of interest
> [2:34 PM] Paul Verardi: I believe my issue has to do with the config
> properties, either in the bootstrap.conf on the minifi agent, or in one of
> the few files we have to modify on the c2 server
> [2:34 PM] Paul Verardi: I am running my MiNiFi agent on a windows 2016 base
> AWS EC2 machine
> [2:34 PM] Paul Verardi: and my c2 on a linux ami based EC2 machine
> [2:36 PM] Paul Verardi: And I am attempting to use the
> PullHttpChangeIngestor instead of the other 2 options
> [2:40 PM] Aldrin Piri: have you opened up the correct ports on your aws ec2
> instance?
> [2:40 PM] Aldrin Piri: and security group
> [2:41 PM] Aldrin Piri: or are they both in the same AZ?
> [2:41 PM] Paul Verardi: Thats where I started with the troubleshooting
> [2:41 PM] Paul Verardi: they are both in east 2 ohio yes
> [2:41 PM] Paul Verardi: and I allow all traffic from all ports just to test
> [2:41 PM] Paul Verardi: I will paste the error I get when I allow all traffic
> [2:42 PM] Paul Verardi: 2018-05-07 13:00:13,079 WARN [pool-2-thread-1]
> o.a.n.m.b.c.i.PullHttpChangeIngestor Hit an exception while trying to pull
> javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
> [2:42 PM] Paul Verardi: If I allow all traffic in the security group
> [2:48 PM] Aldrin Piri: hmm, if you could share your configs, that would be
> helpful
> [2:48 PM] Paul Verardi: sure
> [2:48 PM] Paul Verardi: on both the c2 and the minifi agent?
> [2:48 PM] Aldrin Piri: certainly seems like something is just a little awry
> and thus the timeouts, but would need to see specifics. thanks
> [2:49 PM] Aldrin Piri: yes, please
> [2:50 PM] Paul Verardi:
> File uploaded: authorities.yaml
>
> [https://s3-eu-west-1.amazonaws.com/uploads-eu.hipchat.com/136413/5598132/RPdz8QZjxMMBh1Z/authorities.yaml]
> [2:50 PM] Paul Verardi:
> File uploaded: authorizations.yaml
>
> [https://s3-eu-west-1.amazonaws.com/uploads-eu.hipchat.com/136413/5598132/3Ly5VJP9m9833BZ/authorizations.yaml]
> [2:50 PM] Paul Verardi:
> File uploaded: c2.properties
> [https://s3-eu-west-1.amazonaws.com/uploads-eu.hipchat.com/136413/5598132/mtFxmU2YQ6sH8eH/c2.properties]
> [2:50 PM] Paul Verardi:
> File uploaded: minifi-c2-context.xml
>
> [https://s3-eu-west-1.amazonaws.com/uploads-eu.hipchat.com/136413/5598132/CZafKHOEMnQMQWZ/minifi-c2-context.xml]
> [2:52 PM] Paul Verardi:
> File uploaded: bootstrap.conf
>
> [https://s3-eu-west-1.amazonaws.com/uploads-eu.hipchat.com/136413/5598132/TvFhlhYvIY5nqCJ/bootstrap.conf]
> [2:52 PM] Paul Verardi: any others you need?
> [2:52 PM] Aldrin Piri: that looks good, give me a few to evaluate
> [2:53 PM] Paul Verardi: sure, I blocked out the tls passwords in the
> bootstrap.conf, but my versions have the actual pw in them
> [2:53 PM] Aldrin Piri: yep, no worries
> [3:22 PM] Paul Verardi: Lost internet for a second, missed any replies since
> my last post
> [3:24 PM] Otto Fowler: no
> [3:25 PM] Aldrin Piri: hey paul, sorry I got pulled into meetings. is it
> okay if I take your files and attach them to a JIRA for tracking
> [3:25 PM] Aldrin Piri: either we are insufficient on docs or there is a bug
> [3:26 PM] Paul Verardi: yeah, let me give you a new c2.properties file though
> [3:26 PM] Aldrin Piri: okay, great
> [3:26 PM] Aldrin Piri: or
> [3:26 PM] Aldrin Piri: if you just want to create a JIRA
> [3:26 PM] Aldrin Piri: and upload it all there that is fine as well
> [3:26 PM] Aldrin Piri: let me know your preference
> [3:27 PM] Paul Verardi:
> File uploaded: c2.properties
>
> [https://s3-eu-west-1.amazonaws.com/uploads-eu.hipchat.com/136413/5598132/mtFxmU2YQ6sH8eH/c2.properties]
> [3:28 PM] Paul Verardi: I have never created via public JIRA
> [3:28 PM] Paul Verardi: if you could, that would be great
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
