Re: [robinhood-support] Segmentation fault

Hi Paul, If you can back trace the core that would help. Also strace the run and possibly even monitor VM pressure if you believe the issue to be memory related. Oom killer should be noted in your system logging. On Mar 7, 2014 1:37 PM, "Paul Jackson" wrote: > I'm not sure what your long-running

Re: [robinhood-support] "sleep" connection with database

You'll need to up the logging level to determine why it's not exiting, that said the "sleep"ing processes are just idle connections. On Mon, May 5, 2014 at 8:31 PM, Lucas wrote: > Hello， > When I used Robinhood2.4.0 for partial scan, sometimes it would > never exit. I checked MySQL da

Re: [robinhood-support] Missing Attribute Warnings in RH LHSM Logs

Hi Aurelien, I'm seeing the same warning with a 2.5.1 installation, just in my case it's complaining about fullpath. In my case I'm not removing data what so ever and the FID resolution should yield the needed information. -cf On Thu, May 29, 2014 at 7:21 AM, Aurélien Degrémont < aurelien.degr

Re: [robinhood-support] Missing Attribute Warnings in RH LHSM Logs

copy tools (cp) I don't see this issue. -cf On Thu, May 29, 2014 at 10:48 AM, Kilian Cavalotti < kilian.cavalotti.w...@gmail.com> wrote: > Hi Colin, > > On Thu, May 29, 2014 at 8:49 AM, Colin Faber wrote: > > I'm seeing the same warning with a 2.5.1 installation

Re: [robinhood-support] Metadata inode numbers

Hi Sreedhar, What version of lustre are you running? -cf On Wed, Sep 17, 2014 at 11:14 AM, Sreedhar wrote: > Hi, > > Last week we had a multi disk failure on our lustre metadata server and > had a major corruption of metadata. Upn e2fsck most of the files were put > in lost and found. I got f

[robinhood-support] Scaling

Hi Folks, I'm wondering if some of you with the larger machines can give me some feedback on the scale of your RBH installations. Has anyone utilized RBH with 500 million - a billion files? How about RBH against Lustre DNE setups? Any experience out there in production environments? Also, for

[robinhood-support] HSM flow?

Hi Folks, I'm wondering if anyone could explain (or has documentation on) a detailed breakdown of the HSM flow, from file identification to copytool interaction, to verification/realisation that copytool made changes to the file system. -cf

Re: [robinhood-support] HSM flow?

Is it polling the hsm action / state on the file that's being migrated? Is there any documentation available describing how rbh handles hsm events? -cf On Mon, Jul 13, 2015 at 9:45 AM, Colin Faber wrote: > Hi Folks, > > I'm wondering if anyone could explain (or has docu

[robinhood-support] Best practices documents?

Hi Folks, I'm wondering if anyone has put together a best practices document for Robinhood setup / installation / tuning? Or something similar in the existing documents (which I didn't see) Is there a best practices database setup document that you folks follow? Tuning? etc? Thanks in advance!

Re: [robinhood-support] SQL errors with hard links

Maybe check to see if the hard links are being recorded correctly? Can you update the file with additional hard links and see if nlinks is updated to the correct count in the db record? On Nov 2, 2015 7:58 PM, "Andrew Elwell" wrote: > Hi again - I mentioned this a while ago but don't see any che

[robinhood-support] Can't seem to break 7500 records / sec

So I'm playing around with RBH on some reasonably powerful hardware, I've tried various database configurations of robinhood 3, against mariadb with both myisam and innodb tables but keep running into the same road blocks performance wise. My test is simple, on a quiet file system, generate 1 mill

Re: [robinhood-support] Can't seem to break 7500 records / sec

rations that can be processed in parallel by the Lustre > client > by increasing "max_rpc_in_flight" and ko2iblnd peer_credits on robinhood > host and on MDS accordingly. > > Thanks forward for any update, > Thomas > > > Le 09/12/2015 23:31, Colin Faber a écrit : >

Re: [robinhood-support] How to improve changelog processing rates

Hi Nathan, I'm actively working on a similar type of benchmarking activities. From what I've found so far, as most of my slowdowns are within the database processing itself. This can be observed by utilizing db benchmarking tools (sysbench with oltp testing). >From what I've found robinhood usual

[robinhood-support] LUG meet up?

Hi Folks, I'm wondering who else will be going to LUG this year and if you would all like to meet up (much like we did at LAD for RUG). I found the RUG extremely useful and generally interesting. Anyways, if there's no formal meeting scheduled for robinhood discussions Seagate will sponsor one (

Re: [robinhood-support] LUG meet up?

of Robinhood. > > Frederick > > On Thu, Mar 3, 2016 at 7:36 PM Colin Faber wrote: > >> Hi Folks, >> >> I'm wondering who else will be going to LUG this year and if you would >> all like to meet up (much like we did at LAD for RUG). >> >> I found t

Re: [robinhood-support] LUG meet up?

conflict with the LUG presentations, Aurélien > Degrémont and I would be pleased to attend as well. > > If people are interested I can present an update on the upcoming > Robinhood v3. > > Regards > > -- > Henri > > On 08/mars - 19:08 Colin Faber wrote: &g

Re: [robinhood-support] SQL errors with hard links

ecord remains locked in a transaction when it should be removed. The simple fix here, to me seems to just ignore the error, as it appears the FID in question is removed successfully from the DB. -cf On Mon, Nov 2, 2015 at 8:20 PM, Colin Faber wrote: > Maybe check to see if the hard links a

Re: [robinhood-support] LUG meet up?

Hi Folks, After receiving some feedback from various folks interested in participating in this, my thought is that we get together Wednesday after LUG. We can either use the primary meeting room for LUG, or I can reserve a room within the building. As it seems there is much interest in meeting, a

Re: [robinhood-support] robinhood 2.5.5 crushes the client running Lustre scan

It's very possible the problem is lru and inode caching. Try flushing your inode cache every so often during your initial scan. This should keep memory freed up. On Mon, May 2, 2016 at 11:55 AM, Thomas LEIBOVICI (mail perso) < thomasleibov...@free.fr> wrote: > Le 02/05/2016 16:39, Grigory Shamov

[robinhood-support] File class matching and -F

With 3.0, should a file class be matched and executed on if -F is present? FileClass empty_files { Definition { type == file and size == 0 } } Shouldn't this match any files that are zero bytes? Shouldn't this hit and be actionable with -F present? --

Re: [robinhood-support] File class matching and -F

ort, rbh-find, rbh-du, ... ? > > Thanks > Thomas > > > 09/05/2016 19:08, Colin Faber wrote: > > With 3.0, should a file class be matched and executed on if -F is present? > > > FileClass empty_files { > Definition { > type == file and

Re: [robinhood-support] File class matching and -F

ot;-F" option overrides this check, and allow applying policies even if no > full scan has been done. > There is no relationship with fileclasses. > > Regards, > Thomas > > > On 09/05/2016 21:29, Colin Faber wrote: > > Hi Thomas, > > Yes, sorry robinhood its

[robinhood-support] 2.5.5-4 extremely long DB GET

Hey guys, I'm looking at a recently upgraded 2.5.5-4 instance on a very powerful system, Right now it's got about 38 million records in the DB. There are other identical systems with that many or more records in the DB handling different MDT's The bizarre issue I'm seeing here, is that RBH DB ac

Re: [robinhood-support] File class matching and -F

robinhood was >> not aware of the whole filesystem contents (if no full scan has been done). >> "-F" option overrides this check, and allow applying policies even if no >> full scan has been done. >> There is no relationship with fileclasses. >> >> Rega

Re: [robinhood-support] number of stripe items doesn't match stripe count

You could try and determine the file by SQL query against ENTRIES and NAMES table. On May 22, 2016 9:12 PM, "Nathan Dauchy - NOAA Affiliate" < nathan.dau...@noaa.gov> wrote: > Greetings All, > > Has anyone seen this error from "rbh-find" before, and is there an easy > way to figure out what file h

Re: [robinhood-support] License Agreement

Hi Michael, Maybe this will help: https://github.com/cea-hpc/robinhood/blob/master/LICENSE.en.txt On Tue, Jul 12, 2016 at 9:16 AM, Michael Skiba wrote: > To Whom It May Concern: > > > >I am looking for the License Agreement to make sure that it > is ok with Oracle’s Legal depart

Re: [robinhood-support] very long first scan; stats reporting last action hours ago

Hi Jessica, Is the database indicating any work? (login to the db, issue: select processlist;) -cf On Thu, Aug 4, 2016 at 8:23 AM, Jessica Otey wrote: > Hello all, > I just installed robinhood v3 and yesterday afternoon kicked off the first > scan (against a lustre 2.4.3 system). > The scan cl

Re: [robinhood-support] very long first scan; stats reporting last action hours ago

Whoops, I mean "show processlist;" not select. On Thu, Aug 4, 2016 at 11:04 AM, Colin Faber wrote: > Hi Jessica, > > Is the database indicating any work? (login to the db, issue: select > processlist;) > > -cf > > On Thu, Aug 4, 2016 at 8:23 AM, Jessica Otey

Re: [robinhood-support] verifying changelogs is working on robinhood v3

Hi Jessica, Verify your changelog flags. Possibly rmdir/mkdir flags are missing? On Aug 19, 2016 11:36 AM, "Jessica Otey" wrote: > All, > Now that I've got a version 3 install successfully up and running, I'd > like to leverage lustre's changelogs to have robinhood update its database > in near

Re: [robinhood-support] [lustre-discuss] Robinhood v3.0 released!

Congratulations Thomas and team! Good work! On Fri, Sep 16, 2016 at 8:28 AM, LEIBOVICI Thomas wrote: > Final release of Robinhood v3.0 is now available! > > It can be downloaded from > here:https://sourceforge.net/projects/robinhood/files/robinhood/3.0-final >

Re: [robinhood-support] Robinhood - Lustre changelog reader id

The consumer ids only increment upwards, to reclaim your cl1 you'll need to wipe the changelog_* and changelog llog files (d/O/*) files on your MDT with ldiskfs type mount​. -cf -- Check out the vibrant tech community on o

Re: [robinhood-support] scan causing mysql to crash

This seems more like a MySQL issue, have you checked to see if there is an update for MySQL? On Nov 28, 2016 1:46 PM, "Robbins, Lewis" wrote: > Hello, > > We had been successfully running robinhood-tmps 2.5.x for about a year, > and recently started having robinhood/mysql issues, where a scan wi

Re: [robinhood-support] scan causing mysql to crash

intended recipient, please > contact the sender immediately and destroy or permanently delete all copies > of the original message. > > > -- > *From:* Robbins, Lewis > *Sent:* Monday, November 28, 2016 4:21 PM > *To:* Colin Faber > > *Cc:* r

Re: [robinhood-support] [lustre-discuss] Robinhood exhausting RPC resources against 2.5.5 lustre file systems

Have you been able to replicate the issue with a simple test? find /mnt/lustre -exec lfs path2fid {} \; ? On May 17, 2017 12:04 PM, "Jessica Otey" wrote: > All, > > We have observed an unfortunate interaction between Robinhood and two > Lustre 2.5.5 file systems (both of which originated as 1

Re: [robinhood-support] [lustre-discuss] Robinhood exhausting RPC resources against 2.5.5 lustre file systems

Very likely then the test isn't stressing the system enough. What tunings are you using on your robinhood installation? What tunings on the lustre client itself? -cf On Wed, May 17, 2017 at 2:52 PM, Jessica Otey wrote: > Update #1. Robinhood change log consumption is also producing the same >

Re: [robinhood-support] Policy to rsync files

You could use the HSM rules to do this, and have the HSM action be the rsync command On Fri, Jun 30, 2017 at 4:14 AM, Martin BALVERS wrote: > Hi, > > Would it be possible to create a policy that rsyncs/copies to a specific > location when a file in lustre is added/modified/deleted. > What I want

Re: [robinhood-support] weird (but 'legal') usernames

Hi John, You could try running the command with debug enabled to see what SQL it's actually using to see how far that username makes it into the system. From what I recall there shouldn't be anything which prevents it, however if there is a LIKE statement, it's possible to get erroneous results as

Re: [robinhood-support] Aborted connection to DB (Got an error reading communication packets)

mysql client <-> server mismatch ? Did you try recompiling RBH against a newer client? On Fri, Oct 11, 2019 at 2:28 PM Forstell, Todd wrote: > I recently updated our database from Mariadb 5.5.6 to MySQL 5.7.27 and the > robinhood --readlog command seems to be causing the error listed in the > su

Re: [robinhood-support] Feedback on RobinHood

Regarding changelog performance. I've seen up to 20% or more performance degradation (from optimal /w changelogs disabled) depending on how large your mask is. I suspect this is a result of the additional writes required to populate all of the records. -cf On Thu, Jan 12, 2023 at 8:04 AM thomas.l

Re: [robinhood-support] Feedback on RobinHood

FYI LU-14699 in 2.15 implements a garbage collection for changelog which should address this case. On Thu, Jan 12, 2023, 10:22 AM Scott Nolin via robinhood-support < robinhood-support@lists.sourceforge.net> wrote: > We have not looked at changelog performance impact (for us, likely does > not mat