Pcap only requires hdfs configs because it writes to HDFS. Parsers and the
indexing specific jars need them because they either write to hdfs (es and
solr jars) or write/read from hbase (all of them). Essentially these jars
are the main uber jars used to run the storm topologies.

The rest of the jars don't require them because we take the configs from
the environment rather than modifying the jars in the utilities that live
in those jars.


On Thu, May 19, 2016 at 21:15 Nick Allen <[email protected]> wrote:

> Which jars really need the hbase-site, core-site, and hdfs-site
> configuration files injected?  Currently Solr, Elasticsearch, Pcap and
> Parsers all do.  I can’t reason why those are the particular ones that need
> them though.
>
>
> - name: Add *-site.xml files to topology jars
>   shell: "cd {{ item.config_path }} && jar -uf {{ metron_directory
> }}/lib/{{ item.jar_name }} {{ item.file_name }}"
>   with_items:
>       - { config_path: "{{ hbase_config_path }}", file_name:
> "hbase-site.xml", jar_name: "{{ metron_solr_jar_name }}" }
>       - { config_path: "{{ hdfs_config_path }}",  file_name:
> "core-site.xml",  jar_name: "{{ metron_solr_jar_name }}" }
>       - { config_path: "{{ hdfs_config_path }}",  file_name:
> "hdfs-site.xml",  jar_name: "{{ metron_solr_jar_name }}, }
>       - { config_path: "{{ hbase_config_path }}", file_name:
> "hbase-site.xml", jar_name: "{{ metron_elasticsearch_jar_name }}" }
>       - { config_path: "{{ hdfs_config_path }}",  file_name:
> "core-site.xml",  jar_name: "{{ metron_elasticsearch_jar_name }}" }
>       - { config_path: "{{ hdfs_config_path }}",  file_name:
> "hdfs-site.xml",  jar_name: "{{ metron_elasticsearch_jar_name }}" }
>       - { config_path: "{{ hdfs_config_path }}",  file_name:
> "core-site.xml",  jar_name: "{{ metron_pcap_jar_name }}" }
>       - { config_path: "{{ hdfs_config_path }}",  file_name:
> "hdfs-site.xml",  jar_name: "{{ metron_pcap_jar_name }}" }
>       - { config_path: "{{ hbase_config_path }}", file_name:
> "hbase-site.xml", jar_name: "{{ metron_parsers_jar_name }}" }
>       - { config_path: "{{ hdfs_config_path }}",  file_name:
> "core-site.xml",  jar_name: "{{ metron_parsers_jar_name }}" }
>       - { config_path: "{{ hdfs_config_path }}",  file_name:
> "hdfs-site.xml",  jar_name: "{{ metron_parsers_jar_name }}" }
>

Reply via email to