[Bug 1647222] Re: [MIR] xdelta3

[Seth Arnold]

I reviewed xdelta3 version 3.0.11-dfsg-1ubuntu1 as checked into zesty.
This should not be considered a full security audit but rather a quick
gauge of maintainability.

There is one CVE in our database; the patch mingled together the
functional change with twenty times more whitespace changes and a test
for the functional change. This isn't ideal but at least the patch was
labeled to fix a buffer overflow.

xdelta3 provides a binary-patch style of interface that is immensely
flexible. It can either be used as very manual style of compression
tool -- one of the inputs can be a dictionary of 'phrases' that would
be discovered automatically by other compression tools -- or it could be
used as a way to distribute a small set of changes against a fixed file.
It does so by building a program to interpret to emit the final results.

No dependencies.

xdelta3 is written in C, and heavily exploits pointer arithmetic, memcpy()
(but oddly enough not memmove() despite comments indicating that there may
be overlaps), and array indexing operations. When a native Rust version is
available please consider switching to it. (The existing VCDIFF crates on
crates.io are bindings for Google's VCDIFF implementation open-vcdiff.)

- xdelta3 provides a command line interface and header files that can be
  used to bake xdelta3 into other software (not via library use but by
  including the headers directly).
- No build deps
- No daemons
- No pre/post inst/rm scripts
- No init scripts
- No dbus services
- No setuid executables
- /usr/bin/xdelta3 binary
- No sudo fragments
- No udev rules
- The test suite is built into the executable and distributed in the
  binary. We should DISABLE these tests as they are very low quality.
- No cron jobs
- One common warning in the build logs "invalid suffix on literal"

- Subprocesses are spawned to support optional compression; execlp() is
  used directly, and while the execution itself looks safe, the
  compressors it executes can have their execution modified via
  environment variables. This is probably not a security issue but could
  be a cause of unreliable behaviour or support trouble. I strongly
  recommend executing xdelta3 with a whitelist of a handful of needed
  environment variables.

- Subprocesses are also spawned in the test suite. This code may be
  suitable for tests but is unacceptable in the executable or in the
  address space of anything that may choose to embed xdelta3. Ideally we'd
  build two executables in the package: one to run tests, and one to ship
  to users in binary packages, and the test binary would not be packaged
  in any binary packages. If this is too much work please just disable
  the tests at build time.

- Memory management has some potential integer multiplication overflows.

- Memory management is very complicated. Bugs have been found before and
  probably bugs still exist. We should fuzz xdelta3 extensively before we
  rely upon it. I'd love to see a re-write in Rust.

- Files are written to, under control of the command line, using simple
  fopen(3) calls.

- Logging is extensive, looked safe.

- The XDELTA environment variable handling is very complicated; I don't
  understand what the code does with this variable. It could be used to
  adjust the execution of xdelta3 in ways that are probably not security
  issues but may be unreliable or unexpected.

- No privileged operations
- No encryption
- Inputs moderately sanitized; fuzzing may be productive
- No privileged portions of code
- No temporary files
- No WebKit
- No JavaScript
- No PolicyKit
- Clean cppcheck


There are eight patches in debian/patches but only three patches in
debian/patches/series. This should be fixed.

Due to unique code layout decisions, the self-test code appears built into
the xdelta3 executable. The self-test code quality is very poor. (This is
common to most test code but most test code doesn't get included into the
shipped binaries.) We should _disable_ build-time tests so that the
low-quality code is not available for abuse.

__xd3_alloc_func() has an unchecked integer multiplication overflow. I
do not know if this is reachable via untrusted inputs but calloc()
should be used instead.

main_alloc() has an unchecked integer multiplication overflow. I do not
know if this is reachable via untrusted inputs but calloc() should be
used instead.

setup_environment() has an unchecked integer multiplication overflow.
It's controlled by the size of the XDELTA environment variable and thus
may not be possible to actually trip in practice. calloc() could be used
instead.

External compression just executes e.g. gzip from the filesystem. This
gzip will interpret the GZIP environment variable which may have
unintended unexpected consequences. We should ensure to set exactly the
minimum set of environment variables that xdelta3 needs to run properly.
While it's hard to say the environment variables of a program shouldn't be
completely trusted this has the potential to be extremely 

[Bug 1647222] Re: [MIR] xdelta3

[Thomi Richards]

Thanks Matthias!

On Tue, Jan 10, 2017 at 1:43 AM, Matthias Klose  wrote:

> Override component to main
> xdelta3 3.0.11-dfsg-1ubuntu1 in zesty: universe/utils -> main
> xdelta3 3.0.11-dfsg-1ubuntu1 in zesty amd64: universe/utils/optional/100%
> -> main
> xdelta3 3.0.11-dfsg-1ubuntu1 in zesty arm64: universe/utils/optional/100%
> -> main
> xdelta3 3.0.11-dfsg-1ubuntu1 in zesty armhf: universe/utils/optional/100%
> -> main
> xdelta3 3.0.11-dfsg-1ubuntu1 in zesty i386: universe/utils/optional/100%
> -> main
> xdelta3 3.0.11-dfsg-1ubuntu1 in zesty powerpc:
> universe/utils/optional/100% -> main
> xdelta3 3.0.11-dfsg-1ubuntu1 in zesty ppc64el:
> universe/utils/optional/100% -> main
> xdelta3 3.0.11-dfsg-1ubuntu1 in zesty s390x: universe/utils/optional/100%
> -> main
> 8 publications overridden.
>
>
> ** Changed in: xdelta3 (Ubuntu)
>Status: New => Fix Released
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1647222
>
> Title:
>   [MIR] xdelta3
>
> Status in xdelta3 package in Ubuntu:
>   Fix Released
>
> Bug description:
>   MIR for xdelta3
>
>
>   This is a request to include the xdelta3 package in Ubuntu main.
>
>   See below for point-for-point discussion of the items listed at:
>   https://wiki.ubuntu.com/UbuntuMainInclusionRequirements
>
>   [Availability]
>
>   Ubuntu Zesty contains xdelta 3.0.11-dfsg-1 in universe.
>
>   [Rationale]
>
>   xdelta3 is required for the 'download delta' feature in snapd.  This
> allows
>   users to save a considerable amount of bandwidth when downloading
> updates for
>   installed snap packages. The code has all landed in snapd behind a
> feature flag,
>   but cannot be turned on by default until xdelta3 is in main, so snapd
> can depend
>   on xdelta3.
>
>   [Security]
>
>   There was one CVE files against xdelta3 that I could find:
>   http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-9765
>
>   The xdelta3 package installs a single binary (/usr/bin/xdelta3) which is
> not
>   suid or sgid.
>
>   [Quality assurance]
>
>   - The xdelta3 package requires no configuration after installation.
>
>   - As far as I can tell, the package asks no debconf questions of any
>   priority.
>
>   - There are 90 open issues in the upstream bugtracker:
> https://github.com/jmacd/xdelta/issues
>
>   - I've scanned the issue list, and while a few issues may impact Ubuntu
> users
> using xdelta3, none of them seem serious enough to warrant exclusion
> from main
> in my opinion (but what do I know - that's for someone else to
> determine).
>
>   - The debian bug tracker contains security bug:
>
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814067
>
> However this is fixed in the upstream release that's in zesty, and I
> can see a
> distropatch in the version that's in Xenial (I'm assuming it's been
> fixed in
> yakkety as well).
>
>   - The debian package is maintained by 'A Mennucc1', see:
> https://packages.qa.debian.org/x/xdelta3.html
>
>   - The xdelta3 packages does not require any exotic hardware.
>
>   - I'm honestly not sure if the upstream test suite is run during the
> package
> build. I see no explicit test runs in debian/rules, but there is a
> 'check'
> make target, so perhaps that's invoked by default?
>
>   - The package contains a debain/watch file.
>
>   [UI Standards]
>
>   The xdelta3 package ships command line utilities, so I think it's except
> from
>   the requirements of this section.
>
>   [Dependencies]
>
>   The two dependencies of xdelta3 (libc6 and liblzma5) are both already
>   in main.
>
>   [Standards Compliance]
>
>   Since xdelta3 is already in debian, I can only assume that it conforms
> to the
>   related standards.
>
>   [Maintenance]
>
>   I think xdelta3 is relatively stable software, and the debian
> maintenance seems
>   adequate to me to minimise the amount of work we need to do to keep this
> package
>   in main.
>
>   [Background Information]
>
>   The xdelta3 package description contains a basic useful description of
> the
>   purpose of the package. The motivation behind this MIR is described in
> the
>   'rationale' section of this bug report.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/xdelta3/+bug/
> 1647222/+subscriptions
>


-- 
Thomi Richards
thomi.richa...@canonical.com


** Bug watch added: Debian Bug tracker #814067
   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814067

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2014-9765

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1647222

Title:
  [MIR] xdelta3

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/xdelta3/+bug/1647222/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com

[Bug 1647222] Re: [MIR] xdelta3

[Matthias Klose]

Override component to main
xdelta3 3.0.11-dfsg-1ubuntu1 in zesty: universe/utils -> main
xdelta3 3.0.11-dfsg-1ubuntu1 in zesty amd64: universe/utils/optional/100% -> 
main
xdelta3 3.0.11-dfsg-1ubuntu1 in zesty arm64: universe/utils/optional/100% -> 
main
xdelta3 3.0.11-dfsg-1ubuntu1 in zesty armhf: universe/utils/optional/100% -> 
main
xdelta3 3.0.11-dfsg-1ubuntu1 in zesty i386: universe/utils/optional/100% -> main
xdelta3 3.0.11-dfsg-1ubuntu1 in zesty powerpc: universe/utils/optional/100% -> 
main
xdelta3 3.0.11-dfsg-1ubuntu1 in zesty ppc64el: universe/utils/optional/100% -> 
main
xdelta3 3.0.11-dfsg-1ubuntu1 in zesty s390x: universe/utils/optional/100% -> 
main
8 publications overridden.


** Changed in: xdelta3 (Ubuntu)
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1647222

Title:
  [MIR] xdelta3

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/xdelta3/+bug/1647222/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1647222] Re: [MIR] xdelta3

[Thomi Richards]

Hi Matthias,


snappy-dev has been subscribed now.

Thanks,

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1647222

Title:
  [MIR] xdelta3

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/xdelta3/+bug/1647222/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1647222] Re: [MIR] xdelta3

[Matthias Klose]

then somebody (the team admin) has to subscribe the team; I can't do
that myself.


** Changed in: xdelta3 (Ubuntu)
 Assignee: Matthias Klose (doko) => (unassigned)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1647222

Title:
  [MIR] xdelta3

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/xdelta3/+bug/1647222/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1647222] Re: [MIR] xdelta3

[Michael Terry]

** Changed in: xdelta3 (Ubuntu)
 Assignee: (unassigned) => Matthias Klose (doko)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1647222

Title:
  [MIR] xdelta3

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/xdelta3/+bug/1647222/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1647222] Re: [MIR] xdelta3

[Thomi Richards]

Hi,

After speaking with Gustavo, we agreed that the snappy-dev team can be
the bug subscribers for this package.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1647222

Title:
  [MIR] xdelta3

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/xdelta3/+bug/1647222/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1647222] Re: [MIR] xdelta3

[William Grant]

I've fixed xdelta3 in zesty-proposed to run the test suite. It's
currently stuck behind a few thousand autopkgtests, but should migrate
soonish.

** Changed in: xdelta3 (Ubuntu)
   Status: Incomplete => New

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1647222

Title:
  [MIR] xdelta3

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/xdelta3/+bug/1647222/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1647222] Re: [MIR] xdelta3

[Matthias Klose]

- please run the tests during the build

** Changed in: xdelta3 (Ubuntu)
   Status: Confirmed => Incomplete

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1647222

Title:
  [MIR] xdelta3

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/xdelta3/+bug/1647222/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1647222] Re: [MIR] xdelta3

[Matthias Klose]

is "desktop packages" the correct bug subscriber for snap related
packages?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1647222

Title:
  [MIR] xdelta3

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/xdelta3/+bug/1647222/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1647222] Re: [MIR] xdelta3

[Launchpad Bug Tracker]

Status changed to 'Confirmed' because the bug affects multiple users.

** Changed in: xdelta3 (Ubuntu)
   Status: New => Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1647222

Title:
  [MIR] xdelta3

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/xdelta3/+bug/1647222/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs