Quentin,
Thank you very much for the detailed explanation and information. Although it’s
unfortunate that v4 isn’t ready for testing, I’m excited to see such an
innovative direction with the project! I will pass this information along to
the rest of our team.
Best,
Mike
From: "quentin.bou...@cea.fr" <quentin.bou...@cea.fr>
Date: Tuesday, October 22, 2019 at 12:17 AM
To: "Gilbert, Mike" <gilber...@llnl.gov>
Cc: "thomas.leibov...@cea.fr" <thomas.leibov...@cea.fr>,
"'robinhood-support@lists.sourceforge.net'"
<robinhood-support@lists.sourceforge.net>
Subject: Re: Documentation?
Hi Mike,
On 21/10/2019 16:37, Gilbert, Mike wrote:
Thank you so much for taking the time to answer. That is very exciting news and
I am definitely interested in alpha testing version 4. Would you please provide
the quick-start doc?
Unfortunately, RobinHood is not quite at that point where quickstarting it is
as easy as git cloning a repo + running a few shell commands.
Right now, about half of the code that will make up the core of RobinHood v4
rests upstream in the main github repository, in the 'v4' branch. The other half
is still in the process of being reviewed on gerrithub.
To put it blankly, without the second half, the project is not quite usable, nor
useful.
You can still download it:
git clone https://github.com/cea-hpc/robinhood.git
Install its build dependencies:
* meson
* gcc
* mongo-c-driver
* check (optional, needed to run the project's tests)
And then build it:
cd robinhood
meson build
cd build
ninja
But after that, you are pretty much done.
Besides, RobinHood v4 makes the main project (RobinHood) a library, as opposed
to a library + a set of tools. That means the tools -- at the moment: rbh-sync
and rbh-find -- live in separate repositories and have to be built as well (the
process is similar: git clone / meson / ninja). But as with RobinHood, most of
the code (actually all of it) has not yet been reviewed. So you can still get
the code from gerrithub
(rbh-sync<https://review.gerrithub.io/q/status:open+project:cea-hpc/rbh-sync>
and
rbh-find<https://review.gerrithub.io/q/status:open+project:cea-hpc/rbh-find>),
merge it in a local repository
and build the projects. But even after that, the best you will be able to do is
run
rbh-find using the filesystem itself as robinhood's backend (which basically is
the
same thing as running find itself). [*]
At this point, I would understand if you had lost interest in testing RobinHood
v4. The truth is, this is very much still a work in progress. As soon as the
core functionnalities make it upstream, I will post detailed instructions on how
to try them out here.
In the meantime, if you want to speed things up, you can review the code on
gerrithub:
* For the main
project<https://review.gerrithub.io/q/status:open+project:cea-hpc/robinhood+branch:v4>
;
* For
rbh-sync<https://review.gerrithub.io/q/status:open+project:cea-hpc/rbh-sync>
(the tool that replaces robinhood --scan) ;
* For
rbh-find<https://review.gerrithub.io/q/status:open+project:cea-hpc/rbh-find>.
[*] that sentence probably only makes sense if you are familiar with the new
version of the project (or attended my presentation at LAD). Briefly:
RobinHood v4 aims at supporting more than just SQL DBs to store your
filesystem's metadata. For example: NoSQL DBs (mongodb or postgresql), but
not just them either. With RobinHood v4, pretty much any storage technology
can implement RobinHood's "backend interface" and be used as a "RobinHood
backend". As of now, MongoDB is the only fully supported RobinHood backend,
but there is partial (ie. read-only) support for POSIX filesystems.
---
That might seem a bit depressing, and I would prefer this mail to end on a more
positive note. So here is what a typical test session of a fully featured
RobinHood v4.0.0 will look like (assuming you downloaded, built and installed
robinhood, rbh-sync and rbh-find):
# Install mongodb on your machine and start it
...
# Scan your filesystem and mirror its metadata into a mongodb database
rbh-sync rbh:posix:/path/to/mnt rbh:mongo:test
# Enjoy just how fast and easy to use the new rbh-find is
rbh-find rbh:mongo:test -name 'a*'
rbh-find rbh:mongo:test -type f -name 'a*' -atime +30 -print0
rbh-find rbh:mongo:test ! -type f -count
For those that worry "rbh:mongo:<...>" will be a pain to type. I plan on
providing a completion script with rbh-find (and probably rbh-sync as well). =)
Cheers,
Quentin
________________________________
From: thomas.leibov...@cea.fr<mailto:thomas.leibov...@cea.fr>
<thomas.leibov...@cea.fr><mailto:thomas.leibov...@cea.fr>
Sent: Monday, October 21, 2019 12:32 AM
To: Gilbert, Mike <gilber...@llnl.gov><mailto:gilber...@llnl.gov>
Cc:
'robinhood-support@lists.sourceforge.net<mailto:robinhood-support@lists.sourceforge.net>'
<robinhood-support@lists.sourceforge.net><mailto:robinhood-support@lists.sourceforge.net>;
quentin.bou...@cea.fr<mailto:quentin.bou...@cea.fr>
<quentin.bou...@cea.fr><mailto:quentin.bou...@cea.fr>
Subject: RE: Documentation?
Hello Mike,
In 2014, it was robinhood version 2.5.
Robinhood is now at version 3.1.5. This version (3.0) brings generic policies
(you can schedule/execute any action on the filesystem) and advanced action
scheduling options. This version is now stable, so there is no big change in
this branch except bug fixes or adaptation for new lustre features.
You can take a look at the Changelog for more details about robinhood
evolutions: https://github.com/cea-hpc/robinhood/wiki/ChangeLog
Beyond robinhood 3, we are currently focused on developing robinhood 4. This is
a full (disrupting) rewrite of the project.
There was a presentation at this year’s LAD:
https://www.eofs.eu/_media/events/lad19/robinhood-v4-progress-report.pdf
This also explains why there is no major evolution in robinhood 3 today.
Rbh4 is first implemented on MongoDB for higher performance and scalability,
and will use technologies like kafka for scalable changelog processing.
If you are interested in the project, you are welcome to participate to code
the review of version 4:
https://review.gerrithub.io/q/project:cea-hpc%252Frobinhood+branch:v4
If you are interested in alpha testing, please tell us, we will provide you
with a quick start doc.
So as you can see, the project is more alive than ever, and is going to come
out with great new things for the next years!
Regards,
Thomas
De : Gilbert, Mike via robinhood-support
[mailto:robinhood-support@lists.sourceforge.net]
Envoyé : vendredi 18 octobre 2019 18:59
À :
robinhood-support@lists.sourceforge.net<mailto:robinhood-support@lists.sourceforge.net>
Objet : [robinhood-support] Documentation?
Hello,
We've recently decided to re-evaluate Robinhood for use in our center after
several years. After reading through the Wiki on GitHub and doing some testing
with version 3.1.5, it doesn't appear that much has changed since we last
evaluated it in 2014. I've also noticed that there hasn't been a user group
meeting since 2017. Is Robinhood still being actively managed?
Best regards,
Mike Gilbert
LLNL HPC Sysadmin
_______________________________________________
robinhood-support mailing list
robinhood-support@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/robinhood-support
