On Wed, Nov 25, 2015 at 04:54PM, Ivan V. wrote: > Hi, Konstantin, > "TRASH" (the name comes from > org.apache.ignite.internal.processors.igfs.IgfsFileInfo#TRASH_ID Java > constant) notion is only applicable to primary (IGFS) file system. This is > a special "folder" that does not have file system path. When IGFS is > running over a secondary Fs, TRASH also exists in the primary IGFS, but > does not exist in the secondary Fs. > In secondary Fs deletion is performed just through the ordinary Fs API. So, > we *do not* employ any assumption regarding the TRASH existence and > behavior in the secondary Fs. > > As Dmitriy mentioned above, TRASH in primary Fs is needed for performance > reasons: with it we delete file with only 1 transaction in Meta cache: we > do not do any transactions in Data cache. > (Similar technique is frequently applied frequently in real Fs deletion, > like mv foo /tmp/ && rm -r /tmp/foo/ .)
I understand. However, I am wary about the potentially funny and inconsistent cases when an _optional_ secondary file system is plugged in. Cos > Currently we have fix of https://issues.apache.org/jira/browse/IGNITE-1679 > that makes PURGE events enabled for all files. > I still not quite realize how this functionality will be used by customers, > but now it is repaired: once merged, you will be able to use it. > > On Tue, Nov 24, 2015 at 2:52 AM, Dmitriy Setrakyan <dsetrak...@apache.org> > wrote: > > > Cos, > > > > The main reason soft delete was added is performance. Without soft-delete, > > the delete operation would have to wait until a file is fully deleted from > > a folder, which may take time. > > > > As far as secondary FS handling it, IGFS does not require a secondary FS, > > so we should account for cases when IGFS is running stand-alone. > > > > Thoughts? > > > > D. > > > > On Mon, Nov 23, 2015 at 11:00 PM, Konstantin Boudnik <c...@apache.org> > > wrote: > > > > > Let me ask a different question: what's the point of having the concept > > of > > > TRASH? > > > > > > Here's an example why I think the 'soft' delete would only complicate > > > thing. > > > Suppose IGFS is sitting on top of HDFS and both have 'Trash' enabled. > > Now, > > > the file is getting soft-deleted from IGFS and is moved to TRASH folder. > > > But > > > in HDFS it is also a move to a place which doesn't have any special > > meaning > > > for HDFS. > > > > > > Even worst, if IFGS TRASH is linked to HDFS .Trash. HDFS has it's own > > > policy > > > on how to clean that up, which is likely to be different from that on > > IGFS. > > > Often enough, HDFS .Trash is simply disabled. This discrepancy is going > > to > > > create a situation when a file should still be in TRASH, but the > > secondary > > > FS > > > has already purged it. > > > > > > And what if yet another secondary file system like S3 has yet another > > > policy > > > around their own trash, which they don't even have, I believe? > > > > > > Where I am going with this is pretty straight forward: let's drop the > > > soft-delete support and let the secondary FS to deal with it. If there's > > no > > > secondary FS configured - the content of deleted file will have to > > > retrieved > > > by other means. > > > > > > Thoughts? > > > Cos > > > > > > On Fri, Nov 20, 2015 at 07:15PM, Ivan V. wrote: > > > > Hi, dev, > > > > need opinions on the question discussed in > > > > https://issues.apache.org/jira/browse/IGNITE-1679 (IGFS: Purge event > > is > > > > inconsistent). > > > > In short: in Igfs we have "soft" delete that moves the deleted file or > > > > folder to special "TRASH" folder. > > > > Special async worker walks inside TRASH and removes the items > > > permanently. > > > > When an item is completely removed, an event of type > > > > org.apache.ignite.events.EventType#EVT_IGFS_FILE_PURGED is fired. > > > > But such events are now fired only for files, and only in case if such > > > file > > > > was deleted itself, but not a part of a folder sub-tree. It's quite > > > obvious > > > > that such behavior is not quite consistent, so we should either get rid > > > of > > > > PURGE events at all, or make them consistent. > > > > In the latter case it would be good to have answer to the question: > > what > > > > are real use cases when we may need the purge events ? (Now they seem > > to > > > > be used in tests only). > > > > If we don't have such real use cases, are there any objections to get > > rid > > > > of the purge events at all? > > > > Thanks in advance. > > > > >
signature.asc
Description: Digital signature
