[GitHub] metron issue #583: METRON-949 Change from error to warning message for Metro...

[anandsubbu]

Github user anandsubbu commented on the issue:

https://github.com/apache/metron/pull/583
  
@merrimanr, let me know if the above comment answers your question.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Trying to spin up Metron in EC2: Failed

[Nick Allen]

Good.  Glad to hear that.

On Tue, May 23, 2017 at 6:50 PM, Laurens Vets  wrote:

> Hmmm, deploying Metron in AWS from another EC2 instance seems to work
> fine...
>
> I have a fully deployed 10 node Metron install now.
>
> On 2017-05-19 10:52, Nick Allen wrote:
>
>> I am not sure what the issue is.  Nothing is jumping out at me.
>>
>> Weird issues can arise when the machine that you are running the Ansible
>> deployment from has intermittent connectivity issues to the AWS data
>> center.
>>
>> Is Amazon's us-west-2 data center the one closest to you?  You might try a
>> different/closer data center.  You can alter the data center here [1].
>>
>> [1]
>> https://github.com/apache/metron/blob/master/metron-deployme
>> nt/amazon-ec2/conf/defaults.yml#L27-L39
>>
>> Other than that, you could try standing up a small node in AWS and then
>> use
>> that to run the Metron deployment from.  That always works much more
>> smoothly.
>>
>>
>>
>>
>> On Wed, May 17, 2017 at 4:41 PM, Laurens Vets  wrote:
>>
>> Also, it's only 1 out of 10 hosts that failing it seems.
>>>
>>>
>>> On 2017-05-17 13:25, Laurens Vets wrote:
>>>
>>> When you say Metron host, do you mean the host I'm pushing from to ec2
 or the master ambari server in ec2?

 On 2017-05-17 11:51, Ryan Merriman wrote:

 That happens when you don't have the zookeeper url configured correctly.
> Can you check the contents of the /etc/sysconfig/metron file on the
> Metron
> host?
>
> On Wed, May 17, 2017 at 1:36 PM, Laurens Vets 
> wrote:
>
> For testing purposes, I decided to spin up the default Metron AWS
> config.
>
>> This resulted in a hang from ansible here:
>>
>> TASK [librdkafka : include] **
>> **
>> task path: /home/laurens/SAPSource/metron/metron-deployment/roles/
>> librdkafka/tasks/main.yml:18
>> included: /home/laurens/SAPSource/metron/metron-deployment/roles/
>> librdkafka/tasks/dependencies.yml for ec2-34-210-194-189.us-west-2.c
>> ompute.amazonaws.com
>>
>> TASK [librdkafka : Install prerequisites]
>> **
>> 
>> task path: /home/laurens/SAPSource/metron/metron-deployment/roles/
>> librdkafka/tasks/dependencies.yml:18
>>  ESTABLISH
>> CONNECTION
>> FOR USER: centos on PORT 22 TO ec2-34-210-194-189.us-west-2.c
>> ompute.amazonaws.com
>> /usr/lib/python2.7/dist-packages/Crypto/Cipher/blockalgo.py:141:
>> FutureWarning: CTR mode needs counter parameter, not IV
>>   self._cipher = factory.new(key, *args, **kwargs)
>>  EXEC ( umask 22
>> &&
>> mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp
>> -1495041091.74-92163853889508
>> )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp
>> -1495041091.74-92163853889508
>> )" )
>>  PUT
>> /tmp/tmpwnH61y
>> TO /home/centos/.ansible/tmp/ansible-tmp-1495041091.74-92163853
>> 889508/yum
>>  EXEC /bin/sh -c
>> 'sudo -H -S -n -u root /bin/sh -c '"'"'echo
>> BECOME-SUCCESS-rmswjjyhfdywqvwtvqwcmbsqpsbohvxh;
>> LANG=en_CA.UTF-8 LC_ALL=en_CA.UTF-8 LC_MESSAGES=en_CA.UTF-8
>> /usr/bin/python
>> -tt /home/centos/.ansible/tmp/ansible-tmp-1495041091.74-92163853
>> 889508/yum;
>> rm -rf "/home/centos/.ansible/tmp/ansible-tmp-1495041091.74-9216385
>> 3889508/"
>> > /dev/null 2>&1'"'"''
>>
>> Looking in the machine logs, I see the following for Kafka and Metron
>> REST:
>>
>> Kafka:
>> [2017-05-17 17:03:14,831] INFO KafkaConfig values:
>> advertised.host.name = null
>> metric.reporters = []
>> quota.producer.default = 9223372036854775807
>> offsets.topic.num.partitions = 50
>> log.flush.interval.messages = 9223372036854775807
>> auto.create.topics.enable = true
>> controller.socket.timeout.ms = 3
>> log.flush.interval.ms = null
>> principal.builder.class = class org.apache.kafka.common.securi
>> ty.auth.DefaultPrincipalBuilder
>> replica.socket.receive.buffer.bytes = 65536
>> min.insync.replicas = 1
>> replica.fetch.wait.max.ms = 500
>> num.recovery.threads.per.data.dir = 1
>> ssl.keystore.type = JKS
>> sasl.mechanism.inter.broker.protocol = GSSAPI
>> default.replication.factor = 1
>> ssl.truststore.password = null
>> log.preallocate = false
>> sasl.kerberos.principal.to.local.rules = [DEFAULT]
>> fetch.purgatory.purge.interval.requests = 1
>> ssl.endpoint.identification.algorithm = null
>> replica.socket.timeout.ms = 3
>> message.max.bytes = 100
>> num.io.threads = 8
>> offsets.commit.required.acks = -1
>> 

Re: AWS cloudformation script for Metron deployment

[Satish Abburi]

I agree – good to have cloud neutral. I still think, we might end up with some 
custom wrapper for specific cloud provider.

Here is a good comparison of the two popular cloud vendor templates.
https://www.cloudenablers.com/blog/comparing-aws-cloudformation-and-azure-resource-manager/


On 5/23/17, 12:59 PM, "Simon Elliston Ball"  wrote:

This might be a neater solution than the current ansible build, however, it 
might also be worth considering something more cloud neutral, since more and 
more of the project is moving into ambari, blueprints and cloud break might 
also be an option worth looking at.

Simon 

> On 23 May 2017, at 21:49, Satish Abburi  wrote:
> 
>  
> Want to check with community before we invest to create one? If we don’t 
have one, can we leverage any existing scripts to create this.
>  
> Thanks,
> Satish
>  

AWS cloudformation script for Metron deployment

[Satish Abburi]

Want to check with community before we invest to create one? If we don’t have 
one, can we leverage any existing scripts to create this.

Thanks,
Satish

[GitHub] metron issue #591: METRON-959 Trivial documentation formatting bugs

[JonZeolla]

Github user JonZeolla commented on the issue:

https://github.com/apache/metron/pull/591
  
@cestella (or another PMC member) can you take a look at 
[INFRA_14159](https://issues.apache.org/jira/browse/INFRA-14159)?  It is 
waiting on user right now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #581: METRON-844: Install Metron Management UI with Ambari MPac...

[ottobackwards]

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/581
  
Great contribution, and it works in Full Dev as you would expect.
I have a couple of code review questions however before +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---