Hans Reiser wrote:
> The sponsor for it flaked, and we are focusing on our darpa and hp and
> applianceware contracts.
Thought so...
> Version 4 will be exciting.
Looking forward to that.
I am still trying to find time to implement my log based Squid object
store, but I have much of the same problems with the design of Squid as
your people had so it is not yet given a very high priority until the
Squid issues is resolved in a good manner (which is one of those long
term projects which must be allowed to take time to complete). However,
the design of version 1 of my store have now been decided upon and is
really simple for this kind of workloads (which defenitely is a good
thing as it should allow for a really simple implementation). The design
trades space utilization to gain speed, perhaps a bit to aggressive for
today (or atleast yesterday) but given the long term developments of
disk latency, disk space, disk I/O bandwidth and network bandwidth I
think it is an reasonable tradeoff which will only become more and more
important by time.
One odd thing about log structured filesystem noted while researching
prior work in that area is that is seems that none of the researchers
seems to have even attempted finding meta structures suitable for a log
structure. All available research seems to be based on structures
assuming metadata is at fixed locations, basically starting with a
traditional filesystem and then trying to make it fit into a log
structure.
For Squid it would become very interesting if in some time (lets say
about a year, maybe more) there is a good volatile permanent object
store similar to reiserfs-raw but with a slightly more flexible
application interface.
As expressed earlier my main concern with the interface used by
reierfs-raw is the inability to concurrently read an old copy of an
object where a new version is currently being stored (i.e. unlink+create
new while open). This is a requirement for Squid in it's current
versions (was only minor issue earlier). Preferably there should be some
kind of simple transaction/commit interface where the new object only
becomes available when completely stored, and still allowing for old
references to complete. I.e. sort of the equivalence of
create(temporary)
write():
rename(temporary, objectname)
in UNIX style filesystems, allowing the object to always be referenced
and references opened after the rename gets the new content while older
references keep the old content until done with it (all referenced
closed).
One nice features of a proxy workload is the rich set of locality hints
available (object name, referer object name, access times, client
identities) which is an important when looking into object clustering,
prefetching and indexing.
One model we are looking at for SMP scaling of Squid is to have most of
store logics in the filesystem, and then have multiple processes/threads
access the store concurrently without having to care much what the other
processes/threads do. In this model it is the store/filesystems
responsibility to keep track of the store and what happens therein, and
to maintain consistent views to each process/thread using it.
--
Henrik Nordstrom
Squid developer
