related: fuse over webhdfs: https://github.com/aw-altiscale/webhdfs
On Thu, Sep 17, 2015 at 8:04 PM, Tony Kurc <trk...@gmail.com> wrote: > has anyone tried using fuse [1] to accomplish something like what Kathy was > asking? > > [1] https://wiki.apache.org/hadoop/MountableHDFS > > On Thu, Sep 17, 2015 at 1:26 PM, Joe Witt <joe.w...@gmail.com> wrote: > >> Kathy, >> >> In short, no. But this is a good idea. Our content repository >> implementations today expect access to a 'local' file system. That >> local file system could in theory be something like an NFS mount but >> we'd need to implement it differently to work with HDFS. This would >> be a great JIRA to create if you're interested. >> >> Thanks >> Joe >> >> On Thu, Sep 17, 2015 at 10:17 AM, Olsen, Katherine <kol...@scitor.com> >> wrote: >> > Hi, >> > >> > >> > >> > I'm attempting to deploy a NiFi instance on Mesos using Apache >> Marathon. Currently whenever the NiFi instance is re-started, a new Mesos >> container is pulled down and serves as the run time environment. This is >> all good except that the new container has an empty flow file repository. >> My preferred solution to avoid this is to use a location on the distributed >> file system (HDFS) as the flow file repository location. However, when I >> attempted to change the configuration file setting in nifi.properties to >> write to HDFS as follows, it simply created a folder named hdfs:// with >> associated sub-directories inside the local Mesos container that is running >> NiFi. >> > >> > >> > >> > nifi.flowfile.repository.directory = >> hdfs://ip_address:8020/nifi/flowfile_repository >> > >> > >> > >> > Is it currently possible to get Nifi to be respectful of the hdfs naming >> convention and save its flow file repository to a distributed file system >> like HDFS? I'm using NiFi version 0.2.1, Mesos version 0.23.0, and >> Marathon version 0.10.0. >> > >> > >> > >> > Thanks, >> > >> > Kathy >> > >> > CONFIDENTIALITY NOTICE (v1.0): This e-mail message, including any >> attachments, is for the sole use of the intended recipient(s) and may >> contain proprietary, confidential or privileged information or otherwise be >> protected by law. Any unauthorized review, use, disclosure or distribution >> is prohibited. If you are not the intended recipient, please notify the >> sender and destroy all copies and the original message. >> > >> > >> -- Sean
