Re: VIM and NTFS streams
Alex Jakushev wrote: On 1/31/08, Bram Moolenaar [EMAIL PROTECTED] wrote: Try setting the 'backupcopy' option to yes. This worked, but then it raises another question. Previous value of the 'backupcopy' option was auto, which means yes or no, which works best. Why didn't it choose yes, if no fails? Vim doesn't detect the situation that you are editing an info stream directly. Also, in my configuration, backup option is no. Theoretically, I should not bother with all this stuff at all? I use Vim7.1 last official release. The 'writebackup' option matters too. I thought Vim did copy the streams, but I suppose this doesn't work when you edit one specific stream. If you want to look at it: function copy_infostreams() in src/os_win32.c This may take a while for me to set up everything... But i have an idea that the problem is with file names. When you open some specific stream, vim considers the stream name as filename, and filename as subfolder. The stream name is separated from file name with a colon (c:\path\foo.txt:bar), and the same colon is used to separate drive letter. Maybe this causes some confusion? Detecting the colon should be simple. When not using NTFS the file name would be illegal. I'll add a todo item for this. But it would be nice if someone can make a patch for it. Also, VIM converts dots in file name to underscores sometimes, when streams are specified. I haven't seen this. Perhaps it's because the long file name is converted to a 8.3 file name? Can you give a reproducable example? NTFS streams are mostly restricted to MS-Windows applications, and rarely used. I don't think it's a good idea to support them in Vim directly. I understand it, but vim has specific windows functionality anyway. But of course it is easy for me to suggest this and that :) I always put more effort in functionality that works everywhere. And to keep programs portable rare features should be avoided, especially when they don't add something essential for the end user. MS thinks otherwise: They want programs to only run on MS-Windows. Portability means they might lose customers. Some Linux developers also go in this direction, they don't care about MS-Windows users. I care for everybody :-). -- The process for understanding customers primarily involves sitting around with other marketing people and talking about what you would to if you were dumb enough to be a customer. (Scott Adams - The Dilbert principle) /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\download, build and distribute -- http://www.A-A-P.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org/// --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: VIM and NTFS streams
krischik wrote: I fear you missed something - since NTFS v3.0 named forks are supported. They are called Alternate data streams. Oops, yep, I have a lot of stuff to catch up with. -- Alexei Alexandrov --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: VIM and NTFS streams
On 5 Feb., 00:12, Ben Schmidt [EMAIL PROTECTED] wrote: And that sad story told, I do agree with the other posters who don't think this has much use in Vim. I think to have some kind of consistency, probably an OS or other fairly central component needs to lead the way, not a text editor. I never meant that - I was just countering the claim that named forks are useless. And I only used Vim as an example of where Vim would have been simpler to implement - on an operating system which properly supports type attributes. Sad that everybody else only sees only hurdles and risks of change and not the chances and the risk of no-change. And yes: there is a rist of no-change! The risk of an ever more complex software world with ever more complex solutions which could be done much easier if one would have done it right from the beginning. And the risk of ever more exploits of those complex solutions. Martin --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: VIM and NTFS streams
On 4 Feb., 23:09, Matt Wozniski [EMAIL PROTECTED] wrote: On Feb 4, 2008 4:29 PM, krischik wrote: On 4 Feb., 21:10, Matt Wozniski wrote: While this would be nice, it would require support code from every application you have. It may only be 6 lines, but 6 lines * 5000 binaries is much more code than is in vim for line ending detection. But the other 5000 applications need detection as well. I did mention webserver and browser getting detection wrong. konquror/nautilus/ explorer needs detection and far more complex then vim. And sure OS/2 needed detection. But there you could always overwrite faulty detected value by manualy correcting the EA's. I'm not sure that most other apps do need detection. wget, for instance, doesn't have to care what line endings the data it saves has. But, for what you want, it would have to detect and save it, even though it doesn't use it. No, a EA aware wget would request the EA's already attached to the file from an EA aware ftp server - the same way it request file permissions today when used with --preserve-permissions. Shell redirection has no idea what's going through it, so it has no way to possibly say what Content-type it is... or dd... That's true. or tar extracting files that were created on a filesystem that didn't track extended attributes... Well, you might like to ask the authors - xattr and acl support has been added to tar last year: http://www.redhatmagazine.com/2007/07/02/tips-from-an-rhce-tar-vs-star-the-battle-of-xattrs/ The issue isn't waiting for it to catch on, it's that until it's universally available it only increases the amount of code and maintenance burden. You are proving my point: backward compatibility has hobbled software development. Things like marking the content encoding might be more useful, but still, not good to work with... If, for example, you had a file test.txt encoded in UTF-16 and with extended attributes marking it as such, and your locale is set to use UTF-8, what would you expect the result of cp test.txt test2.txt to be? I would use cp --archive test.txt test2.txt in which case the EA's are copied as well. At least on SuSE Linux 9.2 onwards. And of course the file would still be UTF-16 - after all it's copy not convert. Without meaning offence: You should read up a little on the subject as you knowledge is not up to date. What about cat text.txt text2.txt? If you expect those two commands to have the same effect, I don't see how it can be done without changes to cp (mark attrs on dest), cat (mark attrs on stdout), and the kernel itself (allow extended attributes on streams). I do not expect that cat and cp behave the same. For the simple reason that they never have behaved the same: cp has options like -- preserve and --archive - cat has not. Even today using cat to copy a file will mean that you loose the all the meta informations attached. a shell that knows how to get/set these attributes (and which it needs to set), Done: see setfattr and getfattr. and a copy of the coreutils that are aware of the changes (so that cp file1 file2 creates file2 with the attrs of file1, not the defaults), Done: GNU cp will do that if --archive is used. On a quick glance, I see absolutely no way to do this properly without support from the OS. While I countered most of your arguments - you are absolutely right here. Only I have once used an operating system (OS/2) which did exactly that - hence my frustration. Martin --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: VIM and NTFS streams
krischik wrote: [...] I would use cp --archive test.txt test2.txt in which case the EA's are copied as well. At least on SuSE Linux 9.2 onwards. And of course the file would still be UTF-16 - after all it's copy not convert. Without meaning offence: You should read up a little on the subject as you knowledge is not up to date. [...] Done: GNU cp will do that if --archive is used. [...] Hmm... According to info cp (on my openSUSE 10.3 system), -a or --archive is equivalent to -dpPR, which means: -d copy symlinks as symlinks and preserve hardlinks between sources in the copies -p preserve attributes. If not specifying which attributes, the default is: mode,ownership,timestamps (xattrs must be specified explicitly to be included) -P copy symlinks as symlinks (sic) -R copy directories recursively Best regards, Tony. -- She said, `I know you ... you cannot sing'. I said, `That's nothing, you should hear me play piano.' -- Morrisey --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: VIM and NTFS streams
On 5 Feb., 11:37, Tony Mechelynck [EMAIL PROTECTED] wrote: krischik wrote: [...] I would use cp --archive test.txt test2.txt in which case the EA's are copied as well. At least on SuSE Linux 9.2 onwards. And of course the file would still be UTF-16 - after all it's copy not convert. Without meaning offence: You should read up a little on the subject as you knowledge is not up to date. [...] Done: GNU cp will do that if --archive is used. [...] Hmm... According to info cp (on my openSUSE 10.3 system), -a or --archive is equivalent to -dpPR, which means: -d copy symlinks as symlinks and preserve hardlinks between sources in the copies -p preserve attributes. If not specifying which attributes, the default is: mode,ownership,timestamps (xattrs must be specified explicitly to be included) -P copy symlinks as symlinks (sic) -R copy directories recursively Well I did not read the manual but copied a file with xattr (Samba creates them at mass) and saw what happened. But that was on a fairly old SuSE 9.2. I check SuSE 10.3 when I am home. Martin --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: VIM and NTFS streams
On Tue, 5 Feb 2008 02:25:26 -0800 (PST) krischik [EMAIL PROTECTED] wrote: Sad that everybody else only sees only hurdles and risks of change and not the chances and the risk of no-change. And yes: there is a rist of no-change! I have to agree with krischik here. At University, I once got stuck in a problem with some of the other mathematicians, who were all of the opinion that Yes, I'll have tea if you're having tea. Problem was, nobody actually had tea. It took one of the lawyers to stand up and say Alright then, I'll have tea. Then all the mathmos jumped ship. An amusing story perhaps, but it proves the point. In all things like this, someone needs to jump first. Does it really matter who that is, as long as someone does? It's been a bone of contention of mine for years, the way modified keys work in terminals. Five years ago XTerm gained ways to express generic modified keys (e.g. Ctrl-Shift-Left). Only recently did any application, such as Vim, start to understand those. It's always been a you jump, I'll jump situation - why should the terminal send sequences nobody would understand, or why should any application look for sequences nobody would send? Someone has to go first. Back to the subject of EAs - someone already has gone first. The GNU fileutils already support EAs. As does tar. I also know that the lighttpd webserver uses them by default, only falling back on filename-based detection if the file doesn't have a Content-Type EA. It'd be lovely if Vim could set those for it. -- Paul LeoNerd Evans [EMAIL PROTECTED] ICQ# 4135350 | Registered Linux# 179460 http://www.leonerd.org.uk/ signature.asc Description: PGP signature
Re: VIM and NTFS streams
On 5 Feb., 12:55, Paul LeoNerd Evans [EMAIL PROTECTED] wrote: I also know that the lighttpd webserver uses them by default, only falling back on filename-based detection if the file doesn't have a Content-Type EA. Funny as you say - on my research for my little rants here (yes I do research before I rant) I found out that Apache too uses EA's when to store and detect file types. And then there is Samba which uses user.DOSATTRIB to store dos attributes. It'd be lovely if Vim could set those for it. That would be a nice indeed. - Bram - are you still with us? Martin --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: VIM and NTFS streams
krischik wrote: Open a console and type man 5 attr? Or look here: http://linux.die.net/man/5/attr No it has nothing to do with Linux. FreeBSD, Solaris and Mac OS X support extended attributes as well. See: http://en.wikipedia.org/wiki/Extended_file_attributes Quote Wikipedia: In Linux, the ext2, ext3, ext4, JFS, ReiserFS and XFS filesystems support extended attributes (abbreviated xattr) if the libattr feature is enabled in the kernel configuration. Oh, thanks - this is a good learning for me. It's a shame I never heard of this before. On the other hand use of extended attributes could solve a problem with 5 lines of code where solving the same problem without could cost you 50. Determine file types, text file line endings and text file encoding come to my mind here. Ask Bram how many line of code he needed in Vim to determine these three informations. With consequent use of xattribs it would have been 6 lines: [...] And best of all: you know before you open the file. AFAIK Bram need to close and reopen files in unfortunate combinations. Such attributes would be useful, yes. But it's backward compatibility that is the root of all evil and you also mentioned this in a sibling post. So I'm not sure these things will become popular unless there is a brand new OS appears which doesn't have to be compatible with anything and it wins the market. Also, the named attributes story doesn't have to do much with numbered NTFS data streams, does it? It might be similar but numbers are too far from string identifier convenience. Which brings me back to my original thought - the design and implementation of data streams in NTFS are useless. -- Alexei Alexandrov --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: VIM and NTFS streams
On 4 Feb., 15:18, krischik [EMAIL PROTECTED] wrote: And it is precisely those simple abstractions (files/directories) available on all modern systems which bring those problems. Only: It's not modern systems - it's between 1 and 2 decades old systems. Truly modern operating systems support extended attributes. One of the reasons I feel so strongly about this issue is the fact that I used OS/2 before where storing the file type in the extended attributes was standart. And it worked a lot better then extensions (Dos) or scanning the first 256 bytes for magic patterns (Unix). There was no guessing involved: The first filetype (the .TYPE xattrib in OS/ 2 is a list) was the primary filetype the rest where possible alternative representations (like a html file could also be represented as plain text). And then there was the .ICON xattrib - how I miss the .ICON xattrib - I could drag and drop an icon to every file and from thereon the file would be represented by that icon. Now, nostalgia besides: The point is that those simple abstractions lead to primitive solutions. And indeed OS/2 with it's advanced file system offered a user experience which even Vista won't offer you. KDE offers some of it - for the price of littering you filesystem with .directory files. But KDE still can't do what OS/2 could do - mainly because they would need use extended attributes to do so - and the KDE developers follow the same line of thought you do - which in turn holds them back. Martin PS: On FAT the extended attributes where stored in a file called EA DATA. SF - which shows that even older file systems can be taught new tricks. --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: VIM and NTFS streams
On 4 Feb., 21:10, Matt Wozniski [EMAIL PROTECTED] wrote: On Feb 4, 2008 9:18 AM, krischik wrote: On the other hand use of extended attributes could solve a problem with 5 lines of code where solving the same problem without could cost you 50. Determine file types, text file line endings and text file encoding come to my mind here. Ask Bram how many line of code he needed in Vim to determine these three informations. With consequent use of xattribs it would have been 6 lines: char mime_type[64]; char mime_encoding[64]; char line_ending[64]; getxattr (filename, Content-Type,mime_type, sizeof mime_type); getxattr (filename, Line-Ending, line_ending, sizeof line_ending); getxattr (filename, Content-Encoding,mime_encoding, sizeof mime_encoding); While this would be nice, it would require support code from every application you have. It may only be 6 lines, but 6 lines * 5000 binaries is much more code than is in vim for line ending detection. But the other 5000 applications need detection as well. I did mention webserver and browser getting detection wrong. konquror/nautilus/ explorer needs detection and far more complex then vim. And sure OS/2 needed detection. But there you could always overwrite faulty detected value by manualy correcting the EA's. After all, vim can't use this information unless something put it there, which requires that everything that can create a file - from touch to wget to shell redirection - needs to be able to put that attribute into the file. And, AFAIK, there's no way for those wget or shell redirection to even know what type of line ending the data that they wrote out had. Which means they'd need to detect it. Just one example: The OS/2 version of ZIP would pack extended attributes. Once it catches on then more application will support it. But I know that this won't happen. We went down the worse is better way for far to long. Which would require that just about every program out there that is transcribing a data stream from one spot to another, rather than authoring it itself, would need to duplicate the sort of EOL detection code in vim in itself. And vim would still need to continue supporting the old way since it's designed to run on filesystems that still don't support extended attributes - or it would need to come up with a way to fake extended attributes on those FS types. No one is saying that extended attributes can't do useful things - just that without being in any way standardized, it's unreasonable to expect that any attributes will ever be set by anyone but you. Ahh, indeed, there is the problem and the reason why computing has not advanced as much in the last 15 years as one would have expected seeing the 15 years before: Backward compatibilty has hobbled us. Martin --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: VIM and NTFS streams
On Feb 4, 2008 9:18 AM, krischik wrote: On the other hand use of extended attributes could solve a problem with 5 lines of code where solving the same problem without could cost you 50. Determine file types, text file line endings and text file encoding come to my mind here. Ask Bram how many line of code he needed in Vim to determine these three informations. With consequent use of xattribs it would have been 6 lines: char mime_type[64]; char mime_encoding[64]; char line_ending[64]; getxattr (filename, Content-Type,mime_type, sizeof mime_type); getxattr (filename, Line-Ending, line_ending, sizeof line_ending); getxattr (filename, Content-Encoding,mime_encoding, sizeof mime_encoding); While this would be nice, it would require support code from every application you have. It may only be 6 lines, but 6 lines * 5000 binaries is much more code than is in vim for line ending detection. After all, vim can't use this information unless something put it there, which requires that everything that can create a file - from touch to wget to shell redirection - needs to be able to put that attribute into the file. And, AFAIK, there's no way for those wget or shell redirection to even know what type of line ending the data that they wrote out had. Which means they'd need to detect it. Which would require that just about every program out there that is transcribing a data stream from one spot to another, rather than authoring it itself, would need to duplicate the sort of EOL detection code in vim in itself. And vim would still need to continue supporting the old way since it's designed to run on filesystems that still don't support extended attributes - or it would need to come up with a way to fake extended attributes on those FS types. No one is saying that extended attributes can't do useful things - just that without being in any way standardized, it's unreasonable to expect that any attributes will ever be set by anyone but you. That being said, if vim had a convenient way to see if extended attributes support was available on the filesystem, it might be worth caching things like the fileformat in the extended attributes, so it only needed to be computed once... Though that might open up a whole new can of worms, since someone could easily change the line endings with another tool between two vim runs, confusing the n00b user with a bunch of ^M's show up everywhere... all in all, this is one place where manual detection seems the best idea. For things like content encoding, it might be more useful - but again, since every other tool doesn't create it, vim would still need to fall back on autodetection if it couldn't find an attribute for it. ~Matt --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: VIM and NTFS streams
On Feb 4, 2008 4:29 PM, krischik wrote: On 4 Feb., 21:10, Matt Wozniski wrote: While this would be nice, it would require support code from every application you have. It may only be 6 lines, but 6 lines * 5000 binaries is much more code than is in vim for line ending detection. But the other 5000 applications need detection as well. I did mention webserver and browser getting detection wrong. konquror/nautilus/ explorer needs detection and far more complex then vim. And sure OS/2 needed detection. But there you could always overwrite faulty detected value by manualy correcting the EA's. I'm not sure that most other apps do need detection. wget, for instance, doesn't have to care what line endings the data it saves has. But, for what you want, it would have to detect and save it, even though it doesn't use it. Shell redirection has no idea what's going through it, so it has no way to possibly say what Content-type it is... or dd... or tar extracting files that were created on a filesystem that didn't track extended attributes... After all, vim can't use this information unless something put it there, which requires that everything that can create a file - from touch to wget to shell redirection - needs to be able to put that attribute into the file. And, AFAIK, there's no way for those wget or shell redirection to even know what type of line ending the data that they wrote out had. Which means they'd need to detect it. Just one example: The OS/2 version of ZIP would pack extended attributes. Once it catches on then more application will support it. But I know that this won't happen. We went down the worse is better way for far to long. The issue isn't waiting for it to catch on, it's that until it's universally available it only increases the amount of code and maintenance burden. And, there's no way that it would ever be universally available for things like line endings style, because most applications just don't know or need to care, so have no reason to set that attribute. Things like marking the content encoding might be more useful, but still, not good to work with... If, for example, you had a file test.txt encoded in UTF-16 and with extended attributes marking it as such, and your locale is set to use UTF-8, what would you expect the result of cp test.txt test2.txt to be? What about cat text.txt text2.txt? If you expect those two commands to have the same effect, I don't see how it can be done without changes to cp (mark attrs on dest), cat (mark attrs on stdout), and the kernel itself (allow extended attributes on streams). Which would require that just about every program out there that is transcribing a data stream from one spot to another, rather than authoring it itself, would need to duplicate the sort of EOL detection code in vim in itself. And vim would still need to continue supporting the old way since it's designed to run on filesystems that still don't support extended attributes - or it would need to come up with a way to fake extended attributes on those FS types. No one is saying that extended attributes can't do useful things - just that without being in any way standardized, it's unreasonable to expect that any attributes will ever be set by anyone but you. Ahh, indeed, there is the problem and the reason why computing has not advanced as much in the last 15 years as one would have expected seeing the 15 years before: Backward compatibilty has hobbled us. The problem with this idea isn't the limitations inherent in backwards compatibility - if it were worthwhile to do, it's no problem to have code that uses attributes if possible and the old, backwards compatible method if not. The problem is that creating this standard and tying it properly into a POSIX OS would require a lot of low-level changes. If we can agree on the requirements - that there must be some number of predefined attributes that are set on every regular file (and what about special files?), that the operating system must handle initializing these attributes correctly when a file is created, that streams would also need to have attributes, and that copying, extracting, and in general moving around data will just work, can you see how difficult this would be to implement? To fully implement this, the very least that's required is an operating system that only supports filesystems that have some way of storing extended attributes, a kernel that's able to initialize each of those attributes on every file created, a shell that knows how to get/set these attributes (and which it needs to set), and a copy of the coreutils that are aware of the changes (so that cp file1 file2 creates file2 with the attrs of file1, not the defaults), at the very least? And that work is just the baseline - the point where other software can start counting on a base set of attributes that will always exist, and trusting that they're correct. On a quick
Re: VIM and NTFS streams
Ben Schmidt wrote: Ahh, indeed, there is the problem and the reason why computing has not advanced as much in the last 15 years as one would have expected seeing the 15 years before: Backward compatibilty has hobbled us. Indeed, I think it has. The Mac OS used to use resource forks and type attributes which were beautiful. Now we have descended to the level of extensions and magic numbers like everyone else. The filesystem supports resource forks and named forks, but nobody's really using them yet--indeed, if anything, at the moment they add confusion because some parts of the OS detect file types that way rather than by extension, when available, so it's hard to know what's going on. And that sad story told, I do agree with the other posters who don't think this has much use in Vim. I think to have some kind of consistency, probably an OS or other fairly central component needs to lead the way, not a text editor. Ben. Send instant messages to your online friends http://au.messenger.yahoo.com --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
RE: VIM and NTFS streams
Alexei Alexandrov wrote: 2 notes here: 1. I think NTFS streams is useless feature. I've never seen any practical example of the usefulness of this. For my own purposes, I agree. However, some users and some applications will find uses for streams (same can be said of any feature). It would be much more useful if more filesystems supported it. 2. I'm not sure there is any API to enumerate streams inside a given file. FindFirstStreamW() FindNextStreamW() --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
Re: VIM and NTFS streams
Alexei Alexandrov schrieb: Alex wrote: Hi all, I understand that these bugs and feature requests are nice to have, but anyway... VIM is not working very well with NTFS streams. When you edit file, say, test.txt:aaa, it complains about backup during save (E510). If you edit file test.txt:my.txt, VIM opens and edits file test_txt (but with correct stream ). Just fooled around with this (I don't know much about NTFS streams ...). Things to try out when writing the stream: (1) simply add a bang: :w! (2) unset 'writebackup' :h 'wb :set nowb :w (3) set 'backupcopy' to yes (default is auto) :h 'bkc :set wb bkc=yes :w ... To me, (3) looks ok, (2) and (1) less ok ... I'd set these options only temporarily, when needed. There must be some explicit support for streams, e.g. the swap file is created as another stream. -- Andy --~--~-~--~~~---~--~~ You received this message from the vim_dev maillist. For more information, visit http://www.vim.org/maillist.php -~--~~~~--~~--~--~---
