Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/1157#discussion_r209655011
--- Diff:
metron-platform/metron-pcap/src/main/java/org/apache/metron/pcap/finalizer/PcapFinalizer.java
---
@@ -99,10 +104,55 @@ protected PcapResultsWriter getResultsWriter() {
LOG.warn("Unable to cleanup files in HDFS", e);
}
}
+ LOG.info("Done finalizing results");
return new PcapPages(outFiles);
}
- protected abstract void write(PcapResultsWriter resultsWriter,
Configuration hadoopConfig, List<byte[]> data, Path outputPath) throws
IOException;
+ /**
+ * Figure out how many threads to use in the thread pool. If it's a
string and ends with "C",
+ * then strip the C and treat it as an integral multiple of the number
of cores. If it's a
+ * string and does not end with a C, then treat it as a number in string
form.
+ */
+ private static int getNumThreads(String numThreads) {
+ String numThreadsStr = ((String) numThreads).trim().toUpperCase();
+ if (numThreadsStr.endsWith("C")) {
+ Integer factor = Integer.parseInt(numThreadsStr.replace("C", ""));
--- End diff --
Should we add a catch block for when a user enters an invalid value? We
should catch and provide a helpful exception message like "Invalid value for
property 'pcap.finalizer.threadpool.size'; value='3CCC'".
---
