[
https://issues.apache.org/jira/browse/METRON-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547858#comment-16547858
]
ASF GitHub Bot commented on METRON-1614:
----------------------------------------
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/1108#discussion_r203383137
--- Diff:
metron-platform/metron-pcap/src/main/java/org/apache/metron/pcap/finalizer/PcapRestFinalizer.java
---
@@ -16,27 +16,21 @@
* limitations under the License.
*/
-package org.apache.metron.pcap;
+package org.apache.metron.pcap.finalizer;
-import java.util.List;
+import java.util.Map;
import org.apache.hadoop.fs.Path;
-import org.apache.metron.job.Pageable;
+import org.apache.metron.pcap.config.PcapOptions;
-public class PcapFiles implements Pageable<Path> {
-
- private List<Path> files;
-
- public PcapFiles(List<Path> files) {
- this.files = files;
- }
+/**
+ * Write to HDFS.
+ */
+public class PcapRestFinalizer extends PcapFinalizer {
@Override
- public Iterable<Path> asIterable() {
- return files;
+ protected String getOutputFileName(Map<String, Object> config, int
partition) {
+ Path finalOutputPath =
PcapOptions.FINAL_OUTPUT_PATH.getTransformed(config, Path.class);
+ return String.format("%s/page-%s", finalOutputPath, partition);
--- End diff --
This constant string should be be a `static final String` with a comment
about the file format in English.
> Create job status abstraction
> -----------------------------
>
> Key: METRON-1614
> URL: https://issues.apache.org/jira/browse/METRON-1614
> Project: Metron
> Issue Type: Sub-task
> Reporter: Ryan Merriman
> Assignee: Michael Miklavcic
> Priority: Major
>
> It is possible to use different job engines such as MR or Spark. There should
> be an abstraction that allows us to track status independent of the
> underlying job engine. Initially we will use YARN/MR to run pcap query jobs.
> We will also need a way to persist this information.
> Pcap job submission should be asynchronous. Some kind of id should be
> returned upon successful job submission rather than blocking and waiting on
> the job to complete.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
