Re: work with files and directories

2018-05-16 Thread Dmitriy Govorukhin
One more thing, If we implement this component and will aggregate information about all files, we can create failure handler which provided detail information about node persistence structure. It will be helpful for debugging node crash. On Wed, May 16, 2018 at 12:46 PM, Dmitry Pavlov wrote: > A

Re: work with files and directories

2018-05-16 Thread Dmitry Pavlov
Andrey Gura expressed in words what I also thought. I agree if we extract common code to some kind of file utils. In the same time creating common file operation framework is not possible because of different nature of WAL, Page Store and other components using files. ср, 16 мая 2018 г. в 11:27,

Re: work with files and directories

2018-05-16 Thread Dmitriy Govorukhin
Andrey, My point is, it will be very cool if there is some component who will know about persistence folder and files, and we can move all generic code, for work with files, to this component. On Wed, May 16, 2018 at 12:59 AM, Pavel Kovalenko wrote: > Andrey, > > I think the main idea of that

Re: work with files and directories

2018-05-15 Thread Pavel Kovalenko
Andrey, I think the main idea of that not about how to unify writing content to file. It's about creating peristence files and folders. For other persistence components (WAL, Checkpoint, etc.) it should be like FileFactory object with methods like "public File getOrCreatePartitionFile(...), public

Re: work with files and directories

2018-05-15 Thread Andrey Gura
Hi, I understand you idea but it just increases dependencies of different component from one that is in general bad practice. We have different components where each one can use different approach for file management. For example page store and WAL have different file IO implementations due to pe

Re: work with files and directories

2018-05-15 Thread Dmitry Pavlov
Hi Maxim, FileWriteAheadLogManager & FsyncModeFileWriteAheadLogManager were intentionally copy-pasted in hope we will soon delete FsyncManager. But it is still shows this tool works good. Probably we could integrate this tool to our processes. Sincerely, Dmitriy Pavlov вт, 15 мая 2018 г. в

Re: work with files and directories

2018-05-15 Thread Maxim Muzafarov
+1 also for something like "resource manager". Recently, I've found for myself sonarcloud.io tool for code analisys. It's free for open source project and I've made Ignite project initial run [1]. I've prepeared analisys for mysefl and found a lot of duplicated code blocks [1]. Of course it's not

Re: work with files and directories

2018-05-15 Thread Pavel Kovalenko
+1 to this approach, It can be also very helpful in failover scenarios when something wrong happened with disk. In this case we're reducing the number of points of failure. 2018-05-15 18:37 GMT+03:00 Dmitriy Govorukhin : > Hi Ignite, > > Do we have a general approach to work with a file and dir

work with files and directories

2018-05-15 Thread Dmitriy Govorukhin
Hi Ignite, Do we have a general approach to work with a file and directories? I see many duplication logic for write through .tmp file. For example, GridCacheDatabaseSharedManager.writeCheckpointEntry(); GridCacheDatabaseSharedManage.nodeStart(); FileWriteAheadLogManager.FileArchiver.archiveSegm