I don’t see where I can pass the UPDATE_LENGTH flag to hflush. Should it be 
there? David

Best Regards,

David R Robison
Senior Systems Engineer

From: Harsh J [mailto:ha...@cloudera.com]
Sent: Wednesday, August 9, 2017 3:01 PM
To: David Robison <david.robi...@psgglobal.net>; user@hadoop.apache.org
Subject: Re: Forcing a file to update its length

I don't think it'd be safe for a reader to force an update of length at the 
replica locations directly. Only the writer would be perfectly aware of the DNs 
in use for the replicas and their states, and the precise count of bytes 
entirely flushed out of the local buffer. Thereby only the writer is in a 
confident position to ask the NameNode to update the visible length during a 

Perhaps your app can perform the update length at the more "occassional" 
lengths of time (or other such stimuli), instead of at every write?

P.s. Unless you need strict persistence to disk, you may also choose to use 
hflush with the UPDATE_LENGTH flag.
On Wed, 9 Aug 2017 at 18:07 David Robison 
<david.robi...@psgglobal.net<mailto:david.robi...@psgglobal.net>> wrote:
I understand that, when writing to a file, I can force it to update its length 
on the namenode by using the following command:


Is there a way to force the update without having to open a DFSOutputStream? 
Can I do this from the FileSystem class or some other Java class? The reason 
for this is that I am mostly writing to HDFS and only occasionally reading. 
However, when I go to read, I am most often reading the most recent data 
written (reading the end of the file not the beginning). If I could force the 
length update at the time of reading that would save time by not having to make 
sure I update the length every time I write to the file (which is about once 
per second).

Thanks, David

David R Robison
Senior Systems Engineer
O. +1 512 247 3700<tel:0512%20247%203700>
M. +1 757 286 0022<tel:+1%20757-286-0022>

Prometheus Security Group Global, Inc.
3019 Alvin Devane Boulevard
Building 4, Suite 450
Austin, TX 78741

Reply via email to