[
https://issues.apache.org/jira/browse/METRON-777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16117987#comment-16117987
]
ASF GitHub Bot commented on METRON-777:
---------------------------------------
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/metron/pull/530#discussion_r131787199
--- Diff:
bundles-lib/src/main/java/org/apache/metron/bundles/util/FileSystemManagerFactory.java
---
@@ -0,0 +1,99 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.bundles.util;
+
+import org.apache.accumulo.start.classloader.vfs.UniqueFileReplicator;
+import org.apache.commons.vfs2.*;
+import org.apache.commons.vfs2.cache.SoftRefFilesCache;
+import org.apache.commons.vfs2.impl.DefaultFileSystemManager;
+import org.apache.commons.vfs2.impl.FileContentInfoFilenameFactory;
+import org.apache.commons.vfs2.provider.hdfs.HdfsFileProvider;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+
+public class FileSystemManagerFactory {
+ private static final Logger LOG =
LoggerFactory.getLogger(FileSystemManagerFactory.class);
+
+ /**
+ * Create a FileSystem manager suitable for our purposes.
+ * This manager supports files of the following types:
+ * * res - resource files
+ * * jar
+ * * tar
+ * * bz2
+ * * tgz
+ * * zip
+ * * HDFS
+ * * FTP
+ * * HTTP/S
+ * * file
+ * @return
+ * @throws FileSystemException
+ */
+ public static FileSystemManager createFileSystemManager() throws
FileSystemException {
+ return createFileSystemManager(null);
+ }
+ public static FileSystemManager createFileSystemManager(String
jarExtensionToRegister) throws FileSystemException {
+ DefaultFileSystemManager vfs = new DefaultFileSystemManager();
+
+ if(jarExtensionToRegister != null ||
!StringUtils.isBlank(jarExtensionToRegister) ) {
+ vfs.addExtensionMap(jarExtensionToRegister, "jar");
+ vfs.addProvider(jarExtensionToRegister, new
org.apache.commons.vfs2.provider.jar.JarFileProvider());
+ }
+
+ vfs.addProvider("res", new
org.apache.commons.vfs2.provider.res.ResourceFileProvider());
+ vfs.addProvider("zip", new
org.apache.commons.vfs2.provider.zip.ZipFileProvider());
+ vfs.addProvider("gz", new
org.apache.commons.vfs2.provider.gzip.GzipFileProvider());
+ vfs.addProvider("ram", new
org.apache.commons.vfs2.provider.ram.RamFileProvider());
+ vfs.addProvider("file", new
org.apache.commons.vfs2.provider.local.DefaultLocalFileProvider());
+ vfs.addProvider("jar", new
org.apache.commons.vfs2.provider.jar.JarFileProvider());
+ vfs.addProvider("http", new
org.apache.commons.vfs2.provider.http.HttpFileProvider());
+ vfs.addProvider("https", new
org.apache.commons.vfs2.provider.https.HttpsFileProvider());
+ vfs.addProvider("ftp", new
org.apache.commons.vfs2.provider.ftp.FtpFileProvider());
+ vfs.addProvider("ftps", new
org.apache.commons.vfs2.provider.ftps.FtpsFileProvider());
+ vfs.addProvider("war", new
org.apache.commons.vfs2.provider.jar.JarFileProvider());
+ vfs.addProvider("par", new
org.apache.commons.vfs2.provider.jar.JarFileProvider());
+ vfs.addProvider("ear", new
org.apache.commons.vfs2.provider.jar.JarFileProvider());
+ vfs.addProvider("sar", new
org.apache.commons.vfs2.provider.jar.JarFileProvider());
+ vfs.addProvider("ejb3", new
org.apache.commons.vfs2.provider.jar.JarFileProvider());
+ vfs.addProvider("tmp", new
org.apache.commons.vfs2.provider.temp.TemporaryFileProvider());
+ vfs.addProvider("tar", new
org.apache.commons.vfs2.provider.tar.TarFileProvider());
+ vfs.addProvider("tbz2", new
org.apache.commons.vfs2.provider.tar.TarFileProvider());
+ vfs.addProvider("tgz", new
org.apache.commons.vfs2.provider.tar.TarFileProvider());
+ vfs.addProvider("bz2", new
org.apache.commons.vfs2.provider.bzip2.Bzip2FileProvider());
+ vfs.addProvider("hdfs", new HdfsFileProvider());
+ vfs.addExtensionMap("jar", "jar");
+ vfs.addExtensionMap("zip", "zip");
+ vfs.addExtensionMap("gz", "gz");
+ vfs.addExtensionMap("tar", "tar");
+ vfs.addExtensionMap("tbz2", "tar");
+ vfs.addExtensionMap("tgz", "tar");
+ vfs.addExtensionMap("bz2", "bz2");
--- End diff --
Why do we not also need to support "bundle" type? (Both here and in the
comments L37-46)
Yes, it's an alias for jar, but so are a lot of the other items above.
If the aliasing is being done somewhere else, I would argue that here is
the correct place.
> Create a plugin system for Metron based on 'NAR'
> ------------------------------------------------
>
> Key: METRON-777
> URL: https://issues.apache.org/jira/browse/METRON-777
> Project: Metron
> Issue Type: New Feature
> Reporter: Otto Fowler
> Assignee: Otto Fowler
>
> The success of the Metron project will be greatly dependent on community
> participation, and with that the ability to adapt and extend Metron without
> having to maintain a fork of the project.
> As organizations and individuals look to extend the Metron system with custom
> parsers, enrichments, and stellar functions that may be proprietary in
> nature, the ability to develop and deploy these extensions outside the Metron
> code base is critically important.
> To that end, and after community discussion and proposal we create or
> formalize the 'plugin' development story in Metron.
> The proposal is to adapt the Apache Nifi NAR system for use in Metron. This
> will provide the system with:
> * archetype(s) for developer projects and independent development
> * defined packaging and metadata for 'plugin' products
> * loading and instantiation with classloader isolation capabilities
> * removing the necessity for shading plugin jars
> These capabilities will also enable other features, such as plugin lifecycle,
> plugin configuration+redeployment, and other things.
> The plugin archetypes and their installation will be a followon
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
