Re: fdisk implementation [read this before deciding what to do]

2009-02-17 Thread Vincent Stemen
On Tue, Feb 17, 2009 at 05:54:40PM +, Jost Tobias Springenberg wrote:
> Is there any chance of this moving forward ?
> As mentioned in my previous  post I really need this feature :). If you
> need any help or the like I would be more than happy to help and get
> this project on track again!
> Otherwise I might go back to were I started off and finish a new fdisk
> with less features myself.

Sorry about the delay on getting back on fdisk/disklabel.  We have had
some family interruptions.  My dad died just before Christmas, than my
mom went in the hospital a few weeks ago for congestive heart failure.
So it's been a bit chaotic lately.  We will try to bump up the priority
on the partitioning tools.  Chet started working on some cleanup on the
code today to prepare the new fdisk for others to evaluate.  Give is
a another couple days and I will try to get back with some more concrete
answers.


> 
> Greetings,
>   Tobi
> 
> 
> 5 Hamilton Avenue, San Jose, CA 95125.
> Status: O
> Content-Length: 902
> Lines: 33
> 
> sd.org
> To: users@crater.dragonflybsd.org
> Status: O
> Content-Length: 820
> Lines: 26
> 
> Jost Tobias Springenberg wrote:
> > Is there any chance of this moving forward ?
> > As mentioned in my previous  post I really need this feature :). If you
> > need any help or the like I would be more than happy to help and get
> > this project on track again!
> > Otherwise I might go back to were I started off and finish a new fdisk
> > with less features myself.
> > 
> > Greetings,
> > Tobi
> > 
> 
> I need it also - even my laptop has all four *BSD on one HDD. One of the 
> towers has those four, plus Solaris, Slackware, Minix, Syllable...
> 
> However ... I don't code 'C', and ASM is probably not in your plans...
> 
> But I sure can *test* on a wide range of MB, controllers, media, and 
> 'foreign' OS disk layouts.
> 
> The 'feature' I need most is an fdisk (and a disklabel) that at least do 
> not harm what they cannot understand.
> 
> ;-)
> 
> Bill


Re: the 'why' of pseudofs

2009-02-17 Thread Michael Neumann
Am Wed, 18 Feb 2009 05:25:10 +0800
schrieb Bill Hacker :

> Folks,
> 
> Google was no help, and I have only the last 54,000 or so of the 
> DragonFlyBSD newsgroup messages to hand on on the PowerBook, wherein
> a message-body search on pfs, PFS, pseudofs turned up only about 240
> or so messages, or Mark One eyeball processing..
> 
> That now done, I find:
> 
> Several of these cover conception, gestation, birth, and education  - 
> the 'what' or 'how' of pseudofs / PFS, so to speak.
> 
> ONE of which lists the pro /con vs PFS_NOT. And that one not really 
> hard-edged.
> 
> NONE of which tell me with any degree of absolute-ish-ness, if you
> will..
> 
> ... that one cannot, or even 'should not' run a HAMMER fs *without*
> PFS mounts.
> 
> ... or nullfs mounts.
> 
> or even  without softlinks.  Persih the htought. Or the
> confusion...
> 
> At all.
> 
> EG: 'none of the above'.
> 
> Mind - I see the rationale - even necessity - for their use in more
> than a few circumstances.
> 
> But I cannot seem to find the prohibitions against their 'non-use'.
> 
> What do you suppose breaks if I do not apply these in an initial
> setup, but rather leave them until specific needs arise, such as
> volume expansion, export, or mirroring?
> 
> I have in mind small drive(s) for /, /usr, /var/, /tmp, /home
> - perhaps not even hammerfs, those. Nothing there that was ever
> overly hard to backup, restore, of JF replace. My mailstore, for
> exampel, has never lived in any of those. Nor web pages. Nor
> Databases.
> 
> It is on separate, much larger, drive(s) for /data, /mail, /web, /pub 
> and such - where 'mission critical' clients live and play.
> 
> UFS(1), / FFS(1)  - not UFS2/FFS2 has made for less hassle when
> hardware goes pear-shaped or OS migration is afoot.
> 
> Enter (BF)HAMMER
> 
> But what concept am I missing here? Nice-to-have? Or absolute
> necessity?

PFS is the smallest unit of mirroring, and the unit to which you can
apply specific retainment policies. For example while you do not want
to retain much history for /tmp, you might want to do so for /home.
When it comes to mirroring, you clearly do not want to mirror changes to
PFS /tmp, while you want to mirror changes to PFS /home. If everything
would lie on a single huge filesystem "/", we could not decide what to
mirror and what not. That's the major design decision. 

You might ask, why not simply specify which directories to mirror 
and which to leave out (without considering PFS)? The issue here is,
that, AFAIK, mirroring works on a very low level, where only inode
numbers are available and not full pathnames, so something like:
  
  tar -cvzf /tmp/backup.tgz --reject="/tmp;/var/tmp"

would not work, or would be slow.

Another issue is locality. Metadata from one PFS lies more close
together and as such is faster to iterate.

Regards,

  Michael



Re: the 'why' of pseudofs

2009-02-17 Thread Matthew Dillon
There are several reasons for using PFSs.

* Shared allocation space.  You don't have to worry about blowing
  out small filesystems and having to resize them.

* Each PFS has its own inode space, allowing mirroring to be used
  to manage backups on a per-PFS basis.  Thus mirroring slaves can
  be conveniently created and destroyed, and masters can be used
  to differentiate what you do and do not want to back up.  e.g.
  I want to backup /home, I don't want to backup /usr/obj.

  In this respect there is actually a lot more to it... PFSs are
  the primary enabler for most of the future multi-master clustering
  work.  Even slaves are extremely inconvenient to do without PFSs
  to manage independant inode spaces.

* Each PFS can have its own history/snapshot retention policy.
  For example you want to retain history on /home but who cares
  about /tmp or /usr/obj ?  You might want to retain only a few
  days worth of snapshots for /var but hundreds of days for /home.

* Each PFS can be pruned / reblocked independantly of the others.
  For example /build on pkgbox is configured to spend a lot longer
  pruning and reblocking then /archive.

With regards to softlinks vs null mounts, null mounts are preferred
because softlinks are not always handled properly, or handled in
the expected way, by utilities.

An example of this would be, say, /usr/src.  If /usr/src is a softlink
then the /usr/obj paths generated would be the expanded softlink.
So instead of getting /usr/obj/usr/src/... you would instead get
/usr/obj/pfs/@@0x:0007/src/...

It can get messy very quickly when the filesystem space is glued 
together with softlinks instead of mounts.

-Matt



the 'why' of pseudofs

2009-02-17 Thread Bill Hacker

Folks,

Google was no help, and I have only the last 54,000 or so of the 
DragonFlyBSD newsgroup messages to hand on on the PowerBook, wherein a 
message-body search on pfs, PFS, pseudofs turned up only about 240 or so 
messages, or Mark One eyeball processing..


That now done, I find:

Several of these cover conception, gestation, birth, and education  - 
the 'what' or 'how' of pseudofs / PFS, so to speak.


ONE of which lists the pro /con vs PFS_NOT. And that one not really 
hard-edged.


NONE of which tell me with any degree of absolute-ish-ness, if you will..

.. that one cannot, or even 'should not' run a HAMMER fs *without* PFS 
mounts.


.. or nullfs mounts.

or even  without softlinks.  Persih the htought. Or the confusion...

At all.

EG: 'none of the above'.

Mind - I see the rationale - even necessity - for their use in more than 
a few circumstances.


But I cannot seem to find the prohibitions against their 'non-use'.

What do you suppose breaks if I do not apply these in an initial setup, 
but rather leave them until specific needs arise, such as volume 
expansion, export, or mirroring?


I have in mind small drive(s) for /, /usr, /var/, /tmp, /home
- perhaps not even hammerfs, those. Nothing there that was ever overly 
hard to backup, restore, of JF replace. My mailstore, for exampel, has 
never lived in any of those. Nor web pages. Nor Databases.


It is on separate, much larger, drive(s) for /data, /mail, /web, /pub 
and such - where 'mission critical' clients live and play.


UFS(1), / FFS(1)  - not UFS2/FFS2 has made for less hassle when hardware 
goes pear-shaped or OS migration is afoot.


Enter (BF)HAMMER

But what concept am I missing here? Nice-to-have? Or absolute necessity?

Regards,

Bill


Re: fdisk implementation [read this before deciding what to do]

2009-02-17 Thread Bill Hacker

Jost Tobias Springenberg wrote:

Is there any chance of this moving forward ?
As mentioned in my previous  post I really need this feature :). If you
need any help or the like I would be more than happy to help and get
this project on track again!
Otherwise I might go back to were I started off and finish a new fdisk
with less features myself.

Greetings,
Tobi



I need it also - even my laptop has all four *BSD on one HDD. One of the 
towers has those four, plus Solaris, Slackware, Minix, Syllable...


However ... I don't code 'C', and ASM is probably not in your plans...

But I sure can *test* on a wide range of MB, controllers, media, and 
'foreign' OS disk layouts.


The 'feature' I need most is an fdisk (and a disklabel) that at least do 
not harm what they cannot understand.


;-)

Bill


Re: fdisk implementation [read this before deciding what to do]

2009-02-17 Thread Jost Tobias Springenberg
Is there any chance of this moving forward ?
As mentioned in my previous  post I really need this feature :). If you
need any help or the like I would be more than happy to help and get
this project on track again!
Otherwise I might go back to were I started off and finish a new fdisk
with less features myself.

Greetings,
Tobi



Re: hammer history ls

2009-02-17 Thread Jost Tobias Springenberg

> Hmm.   The filename space is kinda overloaded there with those
> TID ranges. Do we even need TID ranges?  A person could simply sort
> the output and pick out what he wanted.  

Yes certainly all the info can be parsed out of the current output, 
I just consider it good practice to have it available directly as an
option. Parsing the output via sed / perl / whatsoever is just not that
nice to do all the time.

> Alternatively they could be
> specified as normal options prior to the directive. 
> 
> hammer --start-tid=0x... --end-tid=0x... history-count 
> 
> If you wanted to get really fancy you could also implement date
> ranges.
> 
> hammer --start-date=20-Sep-2008 --end-date=01-Jan-2009 ...

Agree on that one, if you decide that you want it in the repo I can
make the necessary changes tonight or tomoroow evening, should not be
such a big effort anyway, otherwise I might be too lazy ;).
Tobi



Re: hammer history ls

2009-02-17 Thread Bill Hacker

Steve O'Hara-Smith wrote:

On Tue, 17 Feb 2009 11:14:16 +0800
Bill Hacker  wrote:

All-numeral dating might be more human-language independent, and an 
input-format reminder built-in, as in:


--startDDMM=20092008 --endDDMM=01012009


For all numeric it's probably best to use ISO8601 formats like
mmdd, mmddThhmm, mmddThhmmss and even hhmmss.



Agree that. Also more consistent with other utils aleady in-place.

(wonder why  ...)

;-)

Bill