HBase is another application that needs write-append.
Every HBase update is written both to a RAM-based and file-system-based
log. On a period the RAM-based log is flushed to the filesystem. The
RAM-based log and its flushes are used fielding queries.
The sympathetic file-system-based log is meant to act as insurance
against machine crashes. On recovery, the file-system-based log is
supposed to be replayed to catch-up on all updates that happened after
the last RAM-based log flush and before machine crash.
Currently the file-system-based log is being written to HDFS though its
near-to-useless since the log is only seen after its closed.
Thanks,
St.Ack
P.S. Bdbje writing its logs to HDFS would be pretty cool.
Ali Salehi wrote:
Hi,
Thanks for your response.
Actually I want to use it with Berkeley Db java ed. [bdbj].
BDBJ only uses appends to create its db so I though using hadoop
would be a viable option considering the scalability requirements
I'm facing.
Cheers,
AliS
Hi Ali,
>From what I know, there are quite a few applications that need
append-to-exisiting-file feature. Since there is a demand for it, HDFS
will
probably implement it in the near future. I do not know of precise
time-frames yet.
Do you have an application that requires appending-writes to files. If so,
it will be helpful to understand the type and behaviour of this
application.
Thanks,
dhruba
-----Original Message-----
From: Ali Salehi [mailto:[EMAIL PROTECTED]
Sent: Thursday, May 31, 2007 11:46 AM
To: [email protected]
Subject: HDFS, appending-writes status
Hi,
While I was reading the HDFS docs from the wiki, I bump into this
sentence:
"There is a plan to support appending-writes to a file in future"
I would like to know, if possible, what is the time line for implementing
this future.
Cheers,
AliS
**************************************************************
Ali Salehi, LSIR - Distributed Information Systems Laboratory
EPFL-IC-IIF-LSIR, Bâtiment BC, Station 14, CH-1015 Lausanne, Switzerland.
http://lsirwww.epfl.ch/
email: [EMAIL PROTECTED]
Tel: +41-21-6936656 Fax: +41-21-6938115
**************************************************************
Ali Salehi, LSIR - Distributed Information Systems Laboratory
EPFL-IC-IIF-LSIR, Bâtiment BC, Station 14, CH-1015 Lausanne, Switzerland.
http://lsirwww.epfl.ch/
email: [EMAIL PROTECTED]
Tel: +41-21-6936656 Fax: +41-21-6938115
