I agree that the current place isn't a good one, for both the reasons you mention on the jira (and because the people maintaining this code don't primarily work on Hadoop). IMO the SwiftFS driver should live in the swift source tree (as part of open stack).
I'm not -1 on it living in-tree, it's just not my 1st choice. If you want to create a top-level directory for 3rd party (read non-local, non-hdfs file systems) file systems - go for it. It would be an improvement on the current situation (o.a.h.fs.ftp also brings in dependencies that most people don't need). I don't think we need to come up with a new top-level "kitchen sink" directory to handle all Hadoop extensions, there are a few well-defined extension points that can likely be handled independently so logically grouping them separately makes sense to me (and perhaps we'll decide some extensions are better in-tree and some not). On Tue, Feb 12, 2013 at 1:51 PM, Steve Loughran <[email protected]> wrote: > On 12 February 2013 21:35, Eli Collins <[email protected]> wrote: > >> >> To answer your original question, there's already precedent for >> extension code in the project, eg file system extensions like s3 live >> in o.a.h.fs. The directory depends on the type of extension (eg 3rd >> party codec integration lives in o.a.h.io.compress). Probably best to >> discuss the particular case. >> > > I know that s3n & S3 are in the main JAR, but I don't think that should be > copied without good reason, because it's also added another dependency > (jets3t) to everything. > > The case of where to put SwiftFS driver has cropped up in HADOOP-8545, but > I don't think a single JIRA is the place to define policy like this > -hopefully general is. > > -Steve
