0644
--- a/libntfs-3g/compress.c
+++ b/libntfs-3g/compress.c
@@ -6,6 +6,7 @@
* Copyright (c) 2004-2006 Szabolcs Szakacsits
* Copyright (c) 2005 Yura Pakhuchiy
* Copyright (c) 2009-2013 Jean-Pierre Andre
+ * Copyright (c) 2014 Eric Biggers
*
* This program/include file is free soft
Also, results from tests I did copying a file to compressed directory on a
NTFS-3g mount, with time elapsed and compressed sizes shown:
silesia_corpus.tar (211,957,760 bytes)
Current 43.318s 111,230,976 bytes
Proposed12.903s 111,751,168 bytes
canterbury_corpus.tar
On Sun, Aug 10, 2014 at 10:45:16AM +0200, Jean-Pierre André wrote:
>
> You have defined the hash table on static data, and I do
> not want to enter into the meanings of static data in
> shared objects in various operating systems (allowed or
> not, shared by threads or not...). I prefer to have it
On Sun, Aug 10, 2014 at 11:18:49AM +0200, Jean-Pierre André wrote:
> Hi,
>
> Did you compare with the Microsoft implementation ?
>
> I have only checked the biggest file in IE7 update for WinXP
> (WINDOWS/ie7updates/KB963027-IE7/ieframe.dll) with
> cluster size 4096 :
>
> Original size 6066688
>
On Sun, Aug 10, 2014 at 04:51:44PM +0200, Jean-Pierre André wrote:
>
> I gather the crc polynomial you propose has proved
> efficient in other compression algorithm, and is
> supposed to be stable. Then a simpler way is to
> compute the table once for all, and encode it as an
> array of constants.
On Sun, Aug 10, 2014 at 05:29:53PM +0200, Jean-Pierre André wrote:
>
> For a better way you would have to identify which is the dll
> which compresses , and submit compression tasks with some
> control over the durations.
>
RtlCompressBuffer() in ntdll.dll can do LZNT1 compression, which I think
2005 Yura Pakhuchiy
* Copyright (c) 2009-2014 Jean-Pierre Andre
+ * Copyright (c) 2014 Eric Biggers
*
* This program/include file is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as published
@@ -21,17 +22,6 @@
* along
ht (c) 2014 Eric Biggers
*
* This program/include file is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as published
@@ -21,17 +22,6 @@
* along with this program (in the main directory of the NTFS-3G
* distribution in the
* Copyright (c) 2014 Eric Biggers
*
* This program/include file is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as published
@@ -21,17 +22,6 @@
* along with this program (in the main directory of the NTFS-3G
* distribution in t
In ntfs_comp_set(), NTFS-3g will write uninitialized memory if the
compressed data partially fills the last cluster. Fix by zeroing the
memory.
Error reported from valgrind:
Syscall param pwrite64(buf) points to uninitialised byte(s)
at 0x3C7520FA63: __pwrite_nocancel (in /usr/lib64/libpthrea
ntfs_valid_sid() required that the subauthority count be between 1 and 8
inclusively. However, Windows permits more than 8 subauthorities as well
as 0 subauthorities:
- The install.wim file for the latest Windows 10 build contains a file
whose DACL contains a SID with 10 subauthorities.
Hello,
I've made an experimental fork of ntfs-3g that supports reading the "System
Compressed" files that are / will be supported by Windows 10. This feature
allows rarely-modified files to be stored using XPRESS or LZX compression, with
stronger compression than the LZNT1 compression built into
> Where can I find more information about this feature,
> and how can I create such files on Windows 10 ?
>
> Glancing at your code, I do not see anything related
> to (sparse) cluster allocation. Does that mean these
> files are not seekable and must be read/written
> sequen
Hi,
I had a report of a file's data disappearing when an NTFS volume was archived
using wimlib, which uses libntfs-3g to read from NTFS volumes. What seems to
have happened is that libntfs-3g reported two unnamed data streams for a file:
one nonempty and one empty, and wimlib happened to store th
ng...
>
> 6) Rough tests on a Sparc CPU
> A few quick tests of read(), lseek() and stat() ran fine, no
> endianness or alignment issue met.
>
> Finally, a question : is the decompressing code reversible
> and reusable for compressing, or is some mirror code required
> for cre
Hi,
"WOF compression" is as good as the other names. It still seems slightly
wrong
because WOF (the "Windows Overlay Filesystem Filter") is a more general
feature,
and this is actually the *second* compression technology that Microsoft has
built on top of it (the first was "WIMBoot"). For now, I
hen I
have time.
On Tue, Sep 22, 2015 at 10:49 PM, Eric Biggers wrote:
> Hi,
>
> "WOF compression" is as good as the other names. It still seems slightly
> wrong
> because WOF (the "Windows Overlay Filesystem Filter") is a more general
> feature,
> and t
Hi,
The return value of ntfs_attr_pwrite() is documented as follows:
> On success, return the number of successfully written bytes. If this number
> is lower than @count this means that an error was encountered during the
> write so that the write is partial. 0 means nothing was written (also ret
writes in practice.
What if the existing ntfs_attr_pwrite() was simply moved to an internal
function, and ntfs_attr_pwrite() was written as a retry loop around the internal
function?
Eric
On Sat, Oct 31, 2015 at 07:06:01PM +0100, Jean-Pierre André wrote:
> Hi Eric,
>
> Eric Biggers wrote
Hi,
I also noticed that libntfs-3g does not permit certain operations on a reparse
point file that is not a symlink or junction:
- delete the file
- create a hard link to the file
- set the short (DOS) name of the file
Is there any particular reason for these restrictions
Hi,
During a randomized test I discovered that ntfs_create() can fail with ENOSPC
when adding a file to a directory that has many named data streams,
independently of the size of the volume. I am still investigating the problem,
but I thought I'd post some reproducing code. The problem is very s
On Mon, Nov 02, 2015 at 12:53:18PM +0100, Jean-Pierre André wrote:
> This is simply because it is felt dangerous to
> meddle with objets whose purpose and structure
> are not known.
At the NTFS level, the high-level structure of a reparse point file is known:
it's a file with a reparse point attri
Windows requires non-Microsoft reparse points (identified by having bit
31 of the reparse tag clear) to have a 16-byte GUID following the regular
reparse point header. This GUID is not, and cannot, be included in the
"reparse data length" field.
The corrected validation logic allows applications
On Wed, Nov 04, 2015 at 08:10:56AM +0100, Jean-Pierre André wrote:
> Hi Eric,
>
> Attached is the patch (simpler than I first thought).
>
> Jean-Pierre
Thanks. I tested the patch and it made the ENOSPC problem go away.
I'm currently trying to track down a corruption problem that seems to trigg
[Resending with only the script attached, since the original apparently didn't
go through]
Hi,
I finally have more information, and a potential solution, for the apparent NTFS
corruption bug I've been encountering during randomized tests. The bug, as I've
been experiencing it, results in an unre
other code cleanups and added a short subsection to the
ntfs-3g man page.
Eric
On Tue, Sep 22, 2015 at 10:54:10PM -0500, Eric Biggers wrote:
> I've pushed changes to my repository that address a few things you brought
> up:
>
> - compiler warnings addressed
> - decompression m
On Sat, Nov 21, 2015 at 09:56:24AM +0100, Jean-Pierre André wrote:
> I have a question on the use of system compression on
> Windows 10 : I have two computers running Windows 10.
> Both have the same disk size and same partition layout
> (64GB for Win10, about 32GB used), but one uses system
> comp
On Mon, Nov 23, 2015 at 08:49:44AM +0100, Jean-Pierre André wrote:
> See proposal on
> http://jp-andre.pagesperso-orange.fr/systcomp.patches.gz
Hi,
Great work! It looks cleaner than I had expected. I have a few questions and
comments:
Should plugin_operations_t really be declared in reparse.h?
On Fri, Nov 27, 2015 at 11:36:12AM +0100, Jean-Pierre André wrote:
> I suspect there is a problem with the LZX mode :
> I have compressed /Windows/System32/aclui.dll using
> compact.exe, and the decoding of the last compression
> block is wrong (mostly zeroes from offset 0x540670).
> My original fi
On Wed, Nov 25, 2015 at 10:32:06AM +0100, Jean-Pierre André wrote:
> Agreed. It should be plugin.h, but where should this be
> located ("src/plugin.h" ?)
There are a few options I can think of:
1.) Make it include/ntfs-3g/plugin.h and install it with the library headers
2.) Make it src/plugin.h a
Hi,
On Fri, Dec 04, 2015 at 10:03:41AM +0100, Jean-Pierre André wrote:
> Hi Eric,
>
> Please see http://jp-andre.pagesperso-orange.fr/systcomp.tar.gz
> with (most of) your comments taken into account.
It generally looks good. I did a basic test of the system compression plugin
(reading files on
On Fri, Dec 04, 2015 at 10:03:41AM +0100, Jean-Pierre André wrote:
> For creating a new compressed file, the procedure would be :
> - create a new void file
> - "truncate" it to the desired size (hence a void sparse file)
> - set reparse data for the desired compression mode
> - feed the data seque
On Wed, Dec 09, 2015 at 12:15:26PM +0100, Jean-Pierre André wrote:
> Maybe, but ENODATA is defined by Posix as optional...
ENODATA would be consistent with ntfs_get_ntfs_reparse_data(), which also uses
ENODATA. include/ntfs-3g/compat.h already defines ENODATA to ENOENT if it isn't
already defined
Hi,
Please consider the following patch to simplify how reparse plugin support is
configured. I had suggested this approach before and I do think it is a more
maintainable approach. So here is a concrete patch:
By making symlinks and junctions always handled through internal plugins,
we can eli
is to have the plugin distributed separately,
maintained by me, and seeing how many people are interested. It could always be
moved into the NTFS-3G source tree in a future release, if desired.
Thanks!
Eric
On Fri, May 06, 2016 at 10:19:24AM +0200, Jean-Pierre André wrote:
> Hi Eric,
>
>
Although ntfs_log_trace() is defined to a no-op in non-DEBUG builds,
ntfs_attr_name_get() is not. This function performs a string conversion
and a memory allocation, so it is nice to have the call to it compiled
out when not needed.
Signed-off-by: Eric Biggers
---
libntfs-3g/dir.c | 4
1
errno.
Signed-off-by: Eric Biggers
---
libntfs-3g/dir.c | 7 +++
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/libntfs-3g/dir.c b/libntfs-3g/dir.c
index ce69eaf..bd049d2 100644
--- a/libntfs-3g/dir.c
+++ b/libntfs-3g/dir.c
@@ -1904,12 +1904,11 @@ int ntfs_delete(ntfs_volume
On Wed, Jun 22, 2016 at 12:12:13PM +0200, Jean-Pierre André wrote:
>
> FYI Following a recent patch, translations of wide chars
> to utf-8 should never fail any more, even when they are
> invalid Unicode ones.
Yes, I probably would not have found this bug after that patch.
I have been planning t
The other NTFS-3G headers use the "_NTFS_" prefix.
Signed-off-by: Eric Biggers
---
include/ntfs-3g/plugin.h | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/include/ntfs-3g/plugin.h b/include/ntfs-3g/plugin.h
index bd90213..f4bb7b2 100644
--- a/include/ntfs-3
Signed-off-by: Eric Biggers
---
include/ntfs-3g/plugin.h | 4
1 file changed, 4 insertions(+)
diff --git a/include/ntfs-3g/plugin.h b/include/ntfs-3g/plugin.h
index f4bb7b2..d32e149 100644
--- a/include/ntfs-3g/plugin.h
+++ b/include/ntfs-3g/plugin.h
@@ -30,7 +30,11 @@
#ifndef
Hello,
I have made the NTFS-3G system compression plugin available in a new repository
at https://github.com/ebiggers/ntfs-3g-system-compression.
I also made a few small updates and updated the build system to use autotools.
With libntfs-3g installed including headers, the plugin can be built and
On Linux, the functions in the dlopen API are in libdl. However, on
FreeBSD, libdl doesn't exist and the functions are instead in libc.
Signed-off-by: Eric Biggers
---
configure.ac| 10 ++
src/Makefile.am | 4 ++--
2 files changed, 12 insertions(+), 2 deletions(-)
diff --
7;c2' in IGNORE_CASE mode
Signed-off-by: Eric Biggers
---
include/ntfs-3g/layout.h | 33 -
libntfs-3g/unistr.c | 28 +++-
2 files changed, 31 insertions(+), 30 deletions(-)
diff --git a/include/ntfs-3g/layout.h b/include/ntfs-3g/l
dealing with the XATTR_CREATE and XATTR_REPLACE
flags. These flags are defined in , so they must be
redefined on platforms without extended attributes support.
Signed-off-by: Eric Biggers
---
include/ntfs-3g/xattrs.h | 27 ---
libntfs-3g/dir.c | 9 +
libntfs
Signed-off-by: Eric Biggers
---
libntfs-3g/xattrs.c | 4
1 file changed, 4 deletions(-)
diff --git a/libntfs-3g/xattrs.c b/libntfs-3g/xattrs.c
index f17e4ca..2b7e709 100644
--- a/libntfs-3g/xattrs.c
+++ b/libntfs-3g/xattrs.c
@@ -81,10 +81,6 @@ struct LE_POSIX_ACL {
#endif
#endif
Signed-off-by: Eric Biggers
---
ntfsprogs/ntfscmp.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/ntfsprogs/ntfscmp.c b/ntfsprogs/ntfscmp.c
index 555e401..cabc9c0 100644
--- a/ntfsprogs/ntfscmp.c
+++ b/ntfsprogs/ntfscmp.c
@@ -547,7 +547,7 @@ static void cmp_index_allocation
n a ntfs_volume.
Some memory leaks in error paths of ntfs_open_secure() are also fixed.
Signed-off-by: Eric Biggers
---
include/ntfs-3g/security.h | 4 ++-
libntfs-3g/security.c | 87 --
libntfs-3g/volume.c| 7
src/lowntfs-3g.c
ix the validation and move it into a helper function, as it was done
identically in the read and write paths.
Signed-off-by: Eric Biggers
---
libntfs-3g/mst.c | 45 +++--
1 file changed, 27 insertions(+), 18 deletions(-)
diff --git a/libntfs-3g/mst.c b/lib
ord.
Fix the validation and move it into a helper function, as it was done
identically in the read and write paths.
[v2: Say "logical sector" instead of "sector", and document NTFS_BLOCK_SIZE.]
Signed-off-by: Eric Biggers
---
incl
On Wed, Jul 27, 2016 at 12:20:24PM +0200, Jean-Pierre André wrote:
>
> Can you disambiguate the word "sector" here ? This is not
> a physical sector, but an ntfs logical sector whose size
> is NTFS_BLOCK_SIZE (512 bytes). This might not have been
> known to the original developer, and it would be
GCC 6 reports a warning in ntfs_allowed_dir_access() because the code has
misleading indentation. Fix by adding braces. There is no actual change
in behavior because of the '(allowed == 2)' condition.
Signed-off-by: Eric Biggers
---
src/ntfs-3g.c | 2 ++
1 file changed, 2 insertion
T rather than a ntfs_volume.
Some memory leaks in error paths of ntfs_open_secure() are also fixed.
[v2: call ntfs_close_secure() earlier, check for error, and other cleanups]
Signed-off-by: Eric Biggers
---
include/ntfs-3g/security.h | 4 +-
libntfs-3g/secur
valid_reparse_data() would read past the end of the reparse point buffer
if it was passed a malformed reparse point that had the tag for a
mountpoint or a symlink but had a data buffer smaller than expected.
Fix this by validating the buffer size.
Signed-off-by: Eric Biggers
---
libntfs-3g
utf16_to_utf8_size() was not guaranteed to fail with ENAMETOOLONG if the
computed length was greater than @outs_len. This could cause a buffer
overrun in ntfs_utf16_to_utf8(). This was a bug introduced by the
patches to allow broken Unicode. Fix it.
Signed-off-by: Eric Biggers
---
libntfs-3g
he maximum length. This was a longstanding bug. Fix it by leaving
space for a terminating null.
Signed-off-by: Eric Biggers
---
libntfs-3g/unistr.c | 14 ++
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/libntfs-3g/unistr.c b/libntfs-3g/unistr.c
index 190dbd8..e70e316 10
ntfs_file_value_upcase() is not called from anywhere in NTFS-3G, seems
unlikely to be used by third-party programs, and can be replaced with
calling ntfs_name_upcase() directly. So remove it.
Signed-off-by: Eric Biggers
---
include/ntfs-3g/unistr.h | 3 ---
libntfs-3g/unistr.c | 17
Since ntfs_ucstombs() returns the output string length, there is no need
to call strlen().
Signed-off-by: Eric Biggers
---
libntfs-3g/dir.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/libntfs-3g/dir.c b/libntfs-3g/dir.c
index f818147..fdc87fa 100644
--- a/libntfs-3g
NTFS_BUG() was broken because it relied on dereferencing a NULL pointer.
This is undefined behavior, and gcc was compiling out the statement.
Crashing in library code is also unfriendly in general.
There were only two users. Make them just use regular error handling.
Signed-off-by: Eric Biggers
ACEs can be found in the public
documentation on MSDN.
Signed-off-by: Eric Biggers
---
include/ntfs-3g/layout.h | 90 +---
1 file changed, 62 insertions(+), 28 deletions(-)
diff --git a/include/ntfs-3g/layout.h b/include/ntfs-3g/layout.h
index 98380de
use object ACEs are rarely used.)
- Only validate the SID for recognized ACE types. The placement or
presence of the SID should not be assumed for future ACE types.
Signed-off-by: Eric Biggers
---
libntfs-3g/acls.c | 114 +++---
1 file changed
Now that the size of the reparse point attribute is no longer used by
the FUSE drivers to populate st_size for symlinks and junctions, it no
longer needs to be returned by ntfs_make_symlink().
Signed-off-by: Eric Biggers
---
include/ntfs-3g/reparse.h | 4 ++--
libntfs-3g/reparse.c | 4
NTFS-3G used several different conventions for setting st_size of
symlinks. Make it use the standard POSIX convention of setting st_size
to the length of the link target without a terminating null.
Signed-off-by: Eric Biggers
---
src/lowntfs-3g.c | 35 +++
src
On Wed, Sep 21, 2016 at 10:47:57AM +0200, Jean-Pierre André wrote:
> Hi Eric,
>
> There has been a recent request for ntfs-3g to return
> the st_size for symlinks as the size of the target
> path (as described in the stat manual), so the target
> is now useful
>
> Regards
>
Well, the previo
Hmm, I think I messed up the "object" ACEs slightly (assuming they ever actually
exist). According to
https://msdn.microsoft.com/en-us/library/windows/desktop/aa374857(v=vs.85).aspx,
the SID is actually at a variable offset.
Another problem is that NTFS-3G interprets ACLs in other places, so if t
On Wed, Sep 21, 2016 at 12:01:23PM +0200, Jean-Pierre André wrote:
>
> I have never met an object ACE and they might be irrelevant
> for a file system which only deals with files and directories.
>
> Is there a point in ntfs-3g accepting ACE types controlling
> entities which are not emulated on
Hi Jean-Pierre,
Sorry for the late response.
On Mon, Sep 26, 2016 at 01:52:36PM +0200, Jean-Pierre André wrote:
> > 1. "Object" ACEs are mentioned as only being used for Active Directory
> > objects
> > [source: Windows Internals 6th edition]. On Windows, trying to use
> > SetFileSecuri
;ll send a patch for that too.
Thanks,
Eric
On Wed, Sep 14, 2016 at 11:39:07PM -0700, Eric Biggers wrote:
> utf16_to_utf8_size() was not guaranteed to fail with ENAMETOOLONG if the
> computed length was greater than @outs_len. This could cause a buffer
> overrun in ntfs_utf16_to_utf8(
Signed-off-by: Eric Biggers
---
src/lowntfs-3g.c | 14 +++---
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/src/lowntfs-3g.c b/src/lowntfs-3g.c
index a91d123..9d933d2 100644
--- a/src/lowntfs-3g.c
+++ b/src/lowntfs-3g.c
@@ -1493,15 +1493,15 @@ close
t;Is there an
> environment which defines uintptr_t different from unsigned
> long ?"
>
> As doing something about it in "configure" would be
> overkilling, I will probably take the proposal for next
> version.
>
> Jean-Pierre
>
> Eric Biggers wrote:
> &g
On Sat, Oct 29, 2016 at 09:45:57AM +0200, Jean-Pierre André wrote:
>
> I am waiting for a green light from Tuxera for merging them
> into the git.
Is there any particular reason you need their permission to do so?
It's difficult for people to contribute to NTFS-3G, or help review or test the
lat
On Sat, Oct 29, 2016 at 12:07:17PM +0200, Jean-Pierre André wrote:
> Eric Biggers wrote:
> > On Sat, Oct 29, 2016 at 09:45:57AM +0200, Jean-Pierre André wrote:
> > >
> > > I am waiting for a green light from Tuxera for merging them
> > > into the git.
> &g
This is no longer used.
Signed-off-by: Eric Biggers
---
src/lowntfs-3g.c | 3 ---
src/ntfs-3g.c| 3 ---
2 files changed, 6 deletions(-)
diff --git a/src/lowntfs-3g.c b/src/lowntfs-3g.c
index 9d933d2..59fbcd0 100644
--- a/src/lowntfs-3g.c
+++ b/src/lowntfs-3g.c
@@ -4227,7 +4227,6 @@ int
On Sat, Oct 29, 2016 at 11:21:37AM +0200, Jean-Pierre André wrote:
> Hi again,
>
> Eric Biggers wrote:
> > Hi Jean-Pierre,
> >
> > Sorry for the late response.
>
> No problem. I also did not do much about it.
>
> The intent of ntfs_valid_descr() was
continue to forbid creating
such names on a filesystem FUSE-mounted with the windows_name option.
Additionally, continue to forbid a trailing a dot or space in DOS names;
this matches the Windows behavior.
Signed-off-by: Eric Biggers
---
include/ntfs-3g/layout.h | 11 ---
include/ntfs-3g
[v2: keep using 'long' instead of switching to 'uintptr_t']
Signed-off-by: Eric Biggers
---
src/lowntfs-3g.c | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/lowntfs-3g.c b/src/lowntfs-3g.c
index b05492e..3f09840 100644
--- a/src/lowntfs-3g
ory".
This bug was found using the wlfuzz program from wimlib.
Signed-off-by: Eric Biggers
---
libntfs-3g/attrib.c | 4
1 file changed, 4 insertions(+)
diff --git a/libntfs-3g/attrib.c b/libntfs-3g/attrib.c
index a5a6549a..1cc3ef64 100644
--- a/libntfs-3g/attrib.c
+++ b/libntfs-3g/at
76 matches
Mail list logo
