[
https://issues.apache.org/jira/browse/METRON-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547861#comment-16547861
]
ASF GitHub Bot commented on METRON-1614:
----------------------------------------
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/1108#discussion_r203382448
--- Diff:
metron-platform/metron-pcap/src/main/java/org/apache/metron/pcap/finalizer/PcapFinalizer.java
---
@@ -0,0 +1,122 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.metron.pcap.finalizer;
+
+import com.google.common.collect.Iterables;
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.LocatedFileStatus;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.RemoteIterator;
+import org.apache.metron.common.hadoop.SequenceFileIterable;
+import org.apache.metron.job.Finalizer;
+import org.apache.metron.job.JobException;
+import org.apache.metron.job.Pageable;
+import org.apache.metron.pcap.PcapPages;
+import org.apache.metron.pcap.config.PcapOptions;
+import org.apache.metron.pcap.writer.PcapResultsWriter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
--- End diff --
It'd be good to explain exactly what this does. I'm not super worried about
javadocs for the method, but general discussion of the purpose of this would be
helpful.
> Create job status abstraction
> -----------------------------
>
> Key: METRON-1614
> URL: https://issues.apache.org/jira/browse/METRON-1614
> Project: Metron
> Issue Type: Sub-task
> Reporter: Ryan Merriman
> Assignee: Michael Miklavcic
> Priority: Major
>
> It is possible to use different job engines such as MR or Spark. There should
> be an abstraction that allows us to track status independent of the
> underlying job engine. Initially we will use YARN/MR to run pcap query jobs.
> We will also need a way to persist this information.
> Pcap job submission should be asynchronous. Some kind of id should be
> returned upon successful job submission rather than blocking and waiting on
> the job to complete.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
