Hi Mike,
On 21/10/2019 16:37, Gilbert, Mike wrote:
Thank you so much for taking the time to answer. That is very exciting
news and I am definitely interested in alpha testing version 4. Would
you please provide the quick-start doc?
Unfortunately, RobinHood is not quite at that point where quickstarting
it is
as easy as git cloning a repo + running a few shell commands.
Right now, about half of the code that will make up the core of RobinHood v4
rests upstream in the main github repository, in the 'v4' branch. The
other half
is still in the process of being reviewed on gerrithub.
To put it blankly, without the second half, the project is not quite
usable, nor
useful.
You can still download it:
git clone https://github.com/cea-hpc/robinhood.git
Install its build dependencies:
* meson
* gcc
* mongo-c-driver
* check (optional, needed to run the project's tests)
And then build it:
cd robinhood
meson build
cd build
ninja
But after that, you are pretty much done.
Besides, RobinHood v4 makes the main project (RobinHood) a library, as
opposed
to a library + a set of tools. That means the tools -- at the moment:
rbh-sync
and rbh-find -- live in separate repositories and have to be built as
well (the
process is similar: git clone / meson / ninja). But as with RobinHood,
most of
the code (actually all of it) has not yet been reviewed. So you can
still get
the code from gerrithub (rbh-sync
<https://review.gerrithub.io/q/status:open+project:cea-hpc/rbh-sync> and
rbh-find
<https://review.gerrithub.io/q/status:open+project:cea-hpc/rbh-find>),
merge it in a local repository
and build the projects. But even after that, the best you will be able
to do is run
rbh-find using the filesystem itself as robinhood's backend (which
basically is the
same thing as running find itself). [*]
At this point, I would understand if you had lost interest in testing
RobinHood
v4. The truth is, this is very much still a work in progress. As soon as the
core functionnalities make it upstream, I will post detailed
instructions on how
to try them out here.
In the meantime, if you want to speed things up, you can review the code on
gerrithub:
* For the main project
<https://review.gerrithub.io/q/status:open+project:cea-hpc/robinhood+branch:v4>
;
* For rbh-sync
<https://review.gerrithub.io/q/status:open+project:cea-hpc/rbh-sync>
(the tool that replaces robinhood --scan) ;
* For rbh-find
<https://review.gerrithub.io/q/status:open+project:cea-hpc/rbh-find>.
[*] that sentence probably only makes sense if you are familiar with the new
version of the project (or attended my presentation at LAD). Briefly:
RobinHood v4 aims at supporting more than just SQL DBs to store your
filesystem's metadata. For example: NoSQL DBs (mongodb or
postgresql), but
not just them either. With RobinHood v4, pretty much any storage
technology
can implement RobinHood's "backend interface" and be used as a
"RobinHood
backend". As of now, MongoDB is the only fully supported RobinHood
backend,
but there is partial (ie. read-only) support for POSIX filesystems.
---
That might seem a bit depressing, and I would prefer this mail to end on
a more
positive note. So here is what a typical test session of a fully featured
RobinHood v4.0.0 will look like (assuming you downloaded, built and
installed
robinhood, rbh-sync and rbh-find):
# Install mongodb on your machine and start it
...
# Scan your filesystem and mirror its metadata into a mongodb database
rbh-sync rbh:posix:/path/to/mnt rbh:mongo:test
# Enjoy just how fast and easy to use the new rbh-find is
rbh-find rbh:mongo:test -name 'a*'
rbh-find rbh:mongo:test -type f -name 'a*' -atime +30 -print0
rbh-find rbh:mongo:test ! -type f -count
For those that worry "rbh:mongo:<...>" will be a pain to type. I plan on
providing a completion script with rbh-find (and probably rbh-sync as
well). =)
Cheers,
Quentin
------------------------------------------------------------------------
*From:* thomas.leibov...@cea.fr <thomas.leibov...@cea.fr>
*Sent:* Monday, October 21, 2019 12:32 AM
*To:* Gilbert, Mike <gilber...@llnl.gov>
*Cc:* 'robinhood-support@lists.sourceforge.net'
<robinhood-support@lists.sourceforge.net>; quentin.bou...@cea.fr
<quentin.bou...@cea.fr>
*Subject:* RE: Documentation?
Hello Mike,
In 2014, it was robinhood version 2.5.
Robinhood is now at version 3.1.5. This version (3.0) brings generic
policies (you can schedule/execute any action on the filesystem) and
advanced action scheduling options. This version is now stable, so
there is no big change in this branch except bug fixes or adaptation
for new lustre features.
You can take a look at the Changelog for more details about robinhood
evolutions: https://github.com/cea-hpc/robinhood/wiki/ChangeLog
Beyond robinhood 3, we are currently focused on developing robinhood
4. This is a full (disrupting) rewrite of the project.
There was a presentation at this year’s LAD:
https://www.eofs.eu/_media/events/lad19/robinhood-v4-progress-report.pdf
This also explains why there is no major evolution in robinhood 3 today.
Rbh4 is first implemented on MongoDB for higher performance and
scalability, and will use technologies like kafka for scalable
changelog processing.
If you are interested in the project, you are welcome to participate
to code the review of version 4:
https://review.gerrithub.io/q/project:cea-hpc%252Frobinhood+branch:v4
If you are interested in alpha testing, please tell us, we will
provide you with a quick start doc.
So as you can see, the project is more alive than ever, and is going
to come out with great new things for the next years!
Regards,
Thomas
*De :*Gilbert, Mike via robinhood-support
[mailto:robinhood-support@lists.sourceforge.net]
*Envoyé :* vendredi 18 octobre 2019 18:59
*À :* robinhood-support@lists.sourceforge.net
*Objet :* [robinhood-support] Documentation?
Hello,
We've recently decided to re-evaluate Robinhood for use in our center
after several years. After reading through the Wiki on GitHub and
doing some testing with version 3.1.5, it doesn't appear that much has
changed since we last evaluated it in 2014. I've also noticed that
there hasn't been a user group meeting since 2017. Is Robinhood still
being actively managed?
Best regards,
Mike Gilbert
LLNL HPC Sysadmin
_______________________________________________
robinhood-support mailing list
robinhood-support@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/robinhood-support
