[
https://issues.apache.org/jira/browse/HADOOP-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HADOOP-5438:
---------------------------------
Attachment: Hadoop-5438(2009-04-06).patch
Hadoop-5438(2009-04-06).patch added in javadoc and apache license for
CreateFlag.
Summary:
1. added an Enum Type called CreateFlag for specifiying the semantic of create()
2. currently CreateFlag only includes: OVERWRITE, APPEND and CREATE
3. Semantics:
(1)CREATE: create the file if it does not exist, throw an exception if it
already existes
(2)OVERWRITE: overwrite a file if it already exists and create it if it does
not exist
(3)APPEND: append to the file if it existes, throw an exception if it does not
exists
4. combine OVERWRITE with either CREATE or APPEND does the same as only use
OVERWRITE
5. combine CREATE and APPEND has the semantic: create a file if it does not
exist, and append to it if it already existes
6. users can comibine mode with: EnumSet.of(CreateFlag, CreateFlag...).
7. Because the current RPC system can not serialize EnumSet. This issue depends
on H-5596, which adds support for serializing EnumSet
Thanks,dhruba borthakur, Konstantin Shvachko and Jakob Homan. All your
suggestions are really helpful !
> Merge FileSystem.create and FileSystem.append
> ---------------------------------------------
>
> Key: HADOOP-5438
> URL: https://issues.apache.org/jira/browse/HADOOP-5438
> Project: Hadoop Core
> Issue Type: Improvement
> Reporter: He Yongqiang
> Attachments: Hadoop-5438(2009-04-06).patch,
> Hadoop-5438-2009-03-30.patch, Hadoop-5438-2009-03-31-2.patch,
> Hadoop-5438-2009-03-31.patch
>
>
> Currently, when a user wants to modify a file, the user first calls exists()
> to know if this file is already there. And then uses create() or append()
> according to whether the file exists or not.
> the code looks like:
> {code}
> FSDataOutputStream out_1 = null;
> if (fs.exists(path_1))
> out_1 = fs.append(path_1);
> else
> out_1 = fs.create(path_1);
> {code}
> . On the performace side,It involes two RPCs. On the easy-of-use side, it is
> not very convient in contrast to the traditional open interface.
> It will more complicate if there is a overwrite parameter specified. I donot
> know whether there is a bug about 'overwrite' in 0.19, some times it takes a
> long time for overwrite creates to reture. So i make the write file code with
> overwrite param works like:
> {code}
> boolean exists = fs.exists(name);
> if (overwrite) {
> if (exists)
> fs.delete(name, true);
> this.out = fs.create(name, overwrite, bufferSize, replication,
> blockSize, progress);
> this.currentRowID = 0;
> } else {
> if (!exists)
> this.out = fs.create(name, overwrite, bufferSize,
> replication, blockSize, progress);
> else
> this.out = fs.append(name, bufferSize, progress);
> {code}
> Some code statements there are really redundant and not needed, especialy
> with the delete(). But without deleting first, the overwrite takes a long
> time to reture.
> BTW, i will create another issue about the overwrite problem. If it is not a
> bug at all or a duplicate, someone please close it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
