Pcap only requires hdfs configs because it writes to HDFS. Parsers and the indexing specific jars need them because they either write to hdfs (es and solr jars) or write/read from hbase (all of them). Essentially these jars are the main uber jars used to run the storm topologies.
The rest of the jars don't require them because we take the configs from the environment rather than modifying the jars in the utilities that live in those jars. On Thu, May 19, 2016 at 21:15 Nick Allen <[email protected]> wrote: > Which jars really need the hbase-site, core-site, and hdfs-site > configuration files injected? Currently Solr, Elasticsearch, Pcap and > Parsers all do. I can’t reason why those are the particular ones that need > them though. > > > - name: Add *-site.xml files to topology jars > shell: "cd {{ item.config_path }} && jar -uf {{ metron_directory > }}/lib/{{ item.jar_name }} {{ item.file_name }}" > with_items: > - { config_path: "{{ hbase_config_path }}", file_name: > "hbase-site.xml", jar_name: "{{ metron_solr_jar_name }}" } > - { config_path: "{{ hdfs_config_path }}", file_name: > "core-site.xml", jar_name: "{{ metron_solr_jar_name }}" } > - { config_path: "{{ hdfs_config_path }}", file_name: > "hdfs-site.xml", jar_name: "{{ metron_solr_jar_name }}, } > - { config_path: "{{ hbase_config_path }}", file_name: > "hbase-site.xml", jar_name: "{{ metron_elasticsearch_jar_name }}" } > - { config_path: "{{ hdfs_config_path }}", file_name: > "core-site.xml", jar_name: "{{ metron_elasticsearch_jar_name }}" } > - { config_path: "{{ hdfs_config_path }}", file_name: > "hdfs-site.xml", jar_name: "{{ metron_elasticsearch_jar_name }}" } > - { config_path: "{{ hdfs_config_path }}", file_name: > "core-site.xml", jar_name: "{{ metron_pcap_jar_name }}" } > - { config_path: "{{ hdfs_config_path }}", file_name: > "hdfs-site.xml", jar_name: "{{ metron_pcap_jar_name }}" } > - { config_path: "{{ hbase_config_path }}", file_name: > "hbase-site.xml", jar_name: "{{ metron_parsers_jar_name }}" } > - { config_path: "{{ hdfs_config_path }}", file_name: > "core-site.xml", jar_name: "{{ metron_parsers_jar_name }}" } > - { config_path: "{{ hdfs_config_path }}", file_name: > "hdfs-site.xml", jar_name: "{{ metron_parsers_jar_name }}" } >
