On Sat, Sep 13, 2014 at 10:39 AM, Richard Hipp wrote:
> I say that a filesystem is an eventually-consistent key/value database.
Yes!
> The keys are the filenames and the values are all big BLOBs, specifically
> the file content. Filesystems also have a hierarchical keyspace, which is
> an exten
On Fri, Sep 12, 2014 at 7:21 PM, Richard Hipp wrote:
> On Fri, Sep 12, 2014 at 8:07 PM, Simon Slavin wrote:
>> one thing that annoys me about SQLite is that it needs to make a
>> journal file which isn't part of the database file. Why ? Why can't it
>> just write the journal to the database f
On Fri, Sep 12, 2014 at 6:47 PM, James K. Lowden
wrote:
> On Fri, 12 Sep 2014 19:38:53 +0100
> Simon Slavin wrote:
>
>> I don't think it can be done by trying to build it on top of an
>> existing file system. I think we need a file system (volume format,
>> drivers, etc.) built from the ground u
On Sat, 13 Sep 2014 15:43:00 -0400
Richard Hipp wrote:
> There were often restrictions on the permitted values for block
> sizes. And you couldn't ask the operating system to tell you whether
> a file was text or binary or sequential or random-access or what its
> block-size was; you just had t
On Sat, Sep 13, 2014 at 2:53 PM, Howard Chu wrote:
> Scott Robison wrote:
>
>> On Sat, Sep 13, 2014 at 2:24 PM, Howard Chu wrote:
>>
>> Scott Robison wrote:
>>>
>>> A couple of academic thoughts.
>>> 1. If one wanted to embed the journal within the database, would it be
adequat
Scott Robison wrote:
On Sat, Sep 13, 2014 at 2:24 PM, Howard Chu wrote:
Scott Robison wrote:
A couple of academic thoughts.
1. If one wanted to embed the journal within the database, would it be
adequate to reserve a specific page as the "root" page of the journal,
then
allocate the rema
On Sat, Sep 13, 2014 at 2:24 PM, Howard Chu wrote:
> Scott Robison wrote:
>
>> A couple of academic thoughts.
>>
>
>> 1. If one wanted to embed the journal within the database, would it be
>> adequate to reserve a specific page as the "root" page of the journal,
>> then
>> allocate the remaining
Scott Robison wrote:
On Fri, Sep 12, 2014 at 6:21 PM, Richard Hipp wrote:
On Fri, Sep 12, 2014 at 8:07 PM, Simon Slavin
wrote:
one thing that annoys me about SQLite is that it needs to make a
journal file which isn't part of the database file. Why ? Why can't it
just write the journal
On 13 Sep 2014, at 9:16pm, Howard Chu wrote:
> Scott Robison wrote:
>
>> At the
>> very least (and I suspect more) Commodore 8-bit DOS (which was embedded
>> within a smart drive with its very own dedicated CPU & RAM) supported
>> (essentially) sequential byte stream files (no random seeking fo
Scott Robison wrote:
On Sat, Sep 13, 2014 at 1:43 PM, Richard Hipp wrote:
Decades ago, files came in all kinds of varieties and permutations.
Details varied from one OS to the next. But it was common to have a
distinction between text files and binary files (with different APIs for
accessing
On Sat, Sep 13, 2014 at 1:43 PM, Richard Hipp wrote:
> Decades ago, files came in all kinds of varieties and permutations.
> Details varied from one OS to the next. But it was common to have a
> distinction between text files and binary files (with different APIs for
> accessing each.) It was a
On 13 Sep 2014, at 8:43pm, Richard Hipp wrote:
> On Sat, Sep 13, 2014 at 2:46 PM, Simon Slavin wrote:
>
>> I would dispute that. [snip]
>
> Roger is correct.
>
> [description]
Thanks for the correction. Apologies to Roger for doubting him.
> You youngsters really have concept of the chao
On Sat, Sep 13, 2014 at 2:46 PM, Simon Slavin wrote:
>
>
> > Before Unix came along it was quite common for files to be structured,
> > managed by the operating system and to be record based with file apis
> > working that way. Unix turned files (and similar) into unstructured
> bags of
> > byte
On Fri, Sep 12, 2014 at 6:21 PM, Richard Hipp wrote:
> On Fri, Sep 12, 2014 at 8:07 PM, Simon Slavin
> wrote:
> >
> > one thing that annoys me about SQLite is that it needs to make a
> > journal file which isn't part of the database file. Why ? Why can't it
> > just write the journal to the
On 13 Sep 2014, at 7:01pm, Luuk wrote:
> On 13-9-2014 19:45, Simon Slavin wrote:
>
>> What I want is the entire volume as a database. It would be something like
>>
>> Key Example value
>> --- -
>> /robots.txt/path /
>>
On 13 Sep 2014, at 7:05pm, Roger Binns wrote:
> Ever hear of Windows and Transactional NTFS :-)
>
> http://msdn.microsoft.com/en-us/magazine/cc163388.aspx
>
> It turns out that adding transactions indiscriminately doesn't magically
> make everything better and lots of thought does need to be a
Scott Robison wrote:
On Sat, Sep 13, 2014 at 9:39 AM, Richard Hipp wrote:
I say that a filesystem is an eventually-consistent key/value database.
The keys are the filenames and the values are all big BLOBs, specifically
the file content. Filesystems also have a hierarchical keyspace, which is
On Sat, Sep 13, 2014 at 9:39 AM, Richard Hipp wrote:
> I say that a filesystem is an eventually-consistent key/value database.
> The keys are the filenames and the values are all big BLOBs, specifically
> the file content. Filesystems also have a hierarchical keyspace, which is
> an extension fr
On 12/09/14 17:07, Simon Slavin wrote:
> Programmers don't expect file services to support transactions because file
> services have never supported transactions.
Ever hear of Windows and Transactional NTFS :-)
http://msdn.microsoft.com/en-us/magazine/cc163388.aspx
It turns out that adding tra
On 13-9-2014 19:45, Simon Slavin wrote:
On 13 Sep 2014, at 6:16pm, Tim Streater wrote:
On 13 Sep 2014 at 16:39, Richard Hipp wrote:
I say that a filesystem is an eventually-consistent key/value database.
The keys are the filenames and the values are all big BLOBs, specifically
the file con
Simon Slavin wrote:
That's not what I meant. That's the file as a database. What I want is
the entire volume as a database.
That's exactly what I pointed you to before. The thesis is pretty enlightening
too.
http://www.fsl.cs.sunysb.edu/docs/kbdbfs-msthesis/
--
-- Howard Chu
CTO, Syma
On 13 Sep 2014, at 6:16pm, Tim Streater wrote:
> On 13 Sep 2014 at 16:39, Richard Hipp wrote:
>
>> I say that a filesystem is an eventually-consistent key/value database.
>> The keys are the filenames and the values are all big BLOBs, specifically
>> the file content. Filesystems also have a
On 13 Sep 2014 at 16:39, Richard Hipp wrote:
> I say that a filesystem is an eventually-consistent key/value database.
> The keys are the filenames and the values are all big BLOBs, specifically
> the file content. Filesystems also have a hierarchical keyspace, which is
> an extension from the
On Sat, Sep 13, 2014 at 11:01 AM, James K. Lowden
wrote:
> I used "database" because that was his word. Whether or not a
> filesystem is a database depends on the definition. I would say that
> the files are data, but the filesystem is not a DBMS, partly for
> reasons you mention. Do the files
On Sat, 13 Sep 2014 01:07:59 +0100
Simon Slavin wrote:
> > Implement a block-transaction store on the device
> > itself: no inodes, no directories, just writeable blocks managed in
> > transactions. Build your DBMS on that.
>
> That would be ... erm ... perhaps a new disk volume format. Wher
On 13 Sep 2014, at 1:21am, Richard Hipp wrote:
> On Fri, Sep 12, 2014 at 8:07 PM, Simon Slavin wrote:
>
>> one thing that annoys me about SQLite is that it needs to make a
>> journal file which isn't part of the database file. Why ? Why can't it
>> just write the journal to the database file
On Fri, Sep 12, 2014 at 8:07 PM, Simon Slavin wrote:
>
> one thing that annoys me about SQLite is that it needs to make a
> journal file which isn't part of the database file. Why ? Why can't it
> just write the journal to the database file it already has open ? This
> would reduce the probl
On 13 Sep 2014, at 12:47am, James K. Lowden wrote:
> I asked your question: why not add transactions to FFS?
>
> His answer: that's the province of a database.
A file system is a database. It accepts data from a user. It stores it away
for later retrieval. It allows it to be searched i
On 13 Sep 2014, at 12:47am, James K. Lowden wrote:
> The filesystem guys I've talked to all consider transactions to be a
> high-level construct hardly ever needed by most applications. They're
> interested in raw speed and a consistent file structure (metadata)
> after a crash. They feel that
On Fri, 12 Sep 2014 19:38:53 +0100
Simon Slavin wrote:
> I don't think it can be done by trying to build it on top of an
> existing file system. I think we need a file system (volume format,
> drivers, etc.) built from the ground up with
> atomicity/ACID/transactions in mind. Since the greatest
hi!
Thanks for the note, and the interest.
I'm cc'ing Thanu into this discussion -- he and a few other grad students
did all the heavy lifting and can share the details of what they've found.
As I recall, he has already spoken with some of the SQLite gang a bit
about the purported "problem" (and
On Fri, Sep 12, 2014 at 1:18 PM, Howard Chu wrote:
> Nico Williams wrote:
>> On ZFS datasets with sync disabled fsync() functions as osync(), as a
>> write
>> barrier without durability and without the associated penalty. The
>> obvious
>> problem is that really do need osync() and fsync(); just
On 12 Sep 2014, at 7:08pm, Nico Williams wrote:
> On ZFS datasets with sync disabled fsync() functions as osync(), as a write
> barrier without durability and without the associated penalty. The obvious
> problem is that really do need osync() and fsync(); just one or the other
> is not a reaso
Nico Williams wrote:
On ZFS datasets with sync disabled fsync() functions as osync(), as a write
barrier without durability and without the associated penalty. The obvious
problem is that really do need osync() and fsync(); just one or the other
is not a reasonable compromise.
Write barriers h
On ZFS datasets with sync disabled fsync() functions as osync(), as a write
barrier without durability and without the associated penalty. The obvious
problem is that really do need osync() and fsync(); just one or the other
is not a reasonable compromise.
Nico
--
Richard Hipp wrote:
On Thu, Sep 11, 2014 at 5:49 PM, Kees Nuyt wrote:
Hi all,
Today I bumped into a presentation about ordering and atomicity
of filesystems that might interest you.
https://www.youtube.com/watch?v=YvchhB1-Aws
The Application/Storage Interface: After All These Years, We're
Simon Slavin wrote:
On 12 Sep 2014, at 3:18am, Scott Robison wrote:
That was an interesting 90 minutes.
Indeed. Thanks to Kees for posting it. Though I was surprised he didn't
mention the term 'ACID' explicitly.
I'm still of the opinion that we need an actual transactional file system
w
On Thu, Sep 11, 2014 at 5:49 PM, Kees Nuyt wrote:
>
> Hi all,
>
> Today I bumped into a presentation about ordering and atomicity
> of filesystems that might interest you.
>
> https://www.youtube.com/watch?v=YvchhB1-Aws
>
> The Application/Storage Interface: After All These Years, We're
> Still D
On 12 Sep 2014, at 3:18am, Scott Robison wrote:
> That was an interesting 90 minutes.
Indeed. Thanks to Kees for posting it. Though I was surprised he didn't
mention the term 'ACID' explicitly.
I'm still of the opinion that we need an actual transactional file system with
equivalents to BE
On Thu, Sep 11, 2014 at 4:09 PM, Kees Nuyt wrote:
> On Thu, 11 Sep 2014 23:49:22 +0200, Kees Nuyt
> wrote:
>
> > Today I bumped into a presentation about ordering and atomicity
> > of filesystems that might interest you.
> >
> > https://www.youtube.com/watch?v=YvchhB1-Aws
>
> Compliments for sql
On Thu, 11 Sep 2014 23:49:22 +0200, Kees Nuyt
wrote:
> Today I bumped into a presentation about ordering and atomicity
> of filesystems that might interest you.
>
> https://www.youtube.com/watch?v=YvchhB1-Aws
Compliments for sqlite at 43:20 .. 43:59
__
Hi all,
Today I bumped into a presentation about ordering and atomicity
of filesystems that might interest you.
https://www.youtube.com/watch?v=YvchhB1-Aws
The Application/Storage Interface: After All These Years, We're
Still Doing It Wrong
Remzi Arpaci-Dusseau, University of Wisconsin—Madison
42 matches
Mail list logo