Re: [Reproducible-builds] Bug#826711: diffoscope: please add argument completion
Hi,
On Sun, Jun 19, 2016 at 8:16 AM, Paul Wise wrote:
> On Sun, 2016-06-19 at 00:32 +0530, Satyam Zode wrote:
>
>> I made the changes as you mentioned, please find an attached patches.
>
> Personally, I would suggest that the changes you have separated into
> separate patches per file are a single logical change and as such
> should all be made in the same commit. Some discussion of this "logical
> change" concept from the OpenStack community is here:
>
> https://wiki.openstack.org/wiki/GitCommitMessages
>
Thanks :) Good resource for learning!
>> I have removed this temporarily! we already had discussion regarding
>> this on IRC. Looking forward to your response!
>
> I would suggest leaving the range completers in, even with the enforced
> sorting by bash itself, I think they are useful. For the upper end of
> the completion range, I think go with double the current maximum.
Here is range completion using updated patch:
satyam@satyamz:~/debian/satymaz-guest/copy/diffoscope/bin$ ./diffoscope -
--css --help
--max-diff-block-lines --text
--debug--html
--max-diff-input-lines --version
--debugger --html-dir
--max-report-size
--fuzzy-threshold --jquery --new-file
-h --list-tools
--separate-file-diff-size
satyam@satyamz:~/debian/satymaz-guest/copy/diffoscope/bin$
./diffoscope --max-report-size
0100 120 140 160 180 20 200
40 60 80
satyam@satyamz:~/debian/satymaz-guest/copy/diffoscope/bin$
./diffoscope --max-diff-block-lines
0 10 15 20 25 30 35 40 45 5 50
satyam@satyamz:~/debian/satymaz-guest/copy/diffoscope/bin$
./diffoscope --max-diff-input-lines
0 10 2 3 4 500055000 65000 75000
85000 95000
1 15000 25000 35000 45000 5 6 7 8 9
satyam@satyamz:~/debian/satymaz-guest/copy/diffoscope/bin$
./diffoscope --separate-file-diff-size
0 10 12 14 16 18 2 20 4
6 8
satyam@satyamz:~/debian/satymaz-guest/copy/diffoscope/bin$
./diffoscope --fuzzy-threshold
0100 120 140 160 180 20 200 220 240 260 280 300 320
340 360 380 40 400 60 80
>
> Some further comments on the patches below:
>
>> +dh_auto_build:
>> + debian/diffoscope.bash-completion
>> + debian/diffoscope.1
>
> This will cause the build to fail, those three lines need to all be on
> one single line with spaces in between. Did gmail wrap it for you?
>
> dh_auto_build: debian/diffoscope.bash-completion debian/diffoscope.1
>
>> +++ b/debian/clean
>> @@ -0,0 +1,3 @@
>> +rm -f debian/diffoscope.1
>> +dh_clean -O--buildsystem=pybuild
>
> The first two lines of the debian/clean file need fixes. See the manual
> page for dh_clean, but the debian/clean file is simply a list of files
> (one per line) to be removed by dh_clean from `debian/rules clean`.
> The full debian/clean file for diffoscope should look like this:
>
> debian/diffoscope.1
> debian/diffoscope.bash-completion
>
>> +elif '_ARGCOMPLETE' not in os.environ:
>
> This will trigger the error in the circumstance where you don't have
> argcomplete installed and you aren't asking for completion. This means
> it will give an error when running it from the command-line if
> argcomplete isn't installed. I think we want an error when you don't
> have argcomplete installed but you are asking for completion.
> To fix this, remove the "not" from this line.
>
>> +print('ERROR: Argument completion requested but Python argcomplete
>> module not installed', file=sys.stderr)
>
> In another place in the diffoscope codebase, a critical error is
> reported using logger.critical before exiting with an error.
> I'm not sure if print, logger.critical or logger.error should be used
> for this. I guess Lunar or other folks can advise you about this.
>
> logger.critical('Console is unable to print Unicode characters. Set e.g.
> LC_CTYPE=en_US.UTF-8')
>
Here, I used logger.error for displaying error and if error occurs
diffoscope returns.
Thanks!
Satyam Zode
From 849db5cbf37905973e5171239e5222a673117536 Mon Sep 17 00:00:00 2001
From: Satyam Zode <satyamz...@gmail.com>
Date: Sun, 19 Jun 2016 15:14:42 +0530
Subject: [PATCH 2/2] Add dependencies for argument completion
Add new rules for bash-completion script.
Add new file debian/clean.
---
debian/clean | 2 ++
debian/control | 2 ++
debian/rules | 13 +++--
3 files changed, 11 insertions(+), 6 deletions(-)
create mode 100644 debian/clean
diff --git a/debian/clean b/debian/clean
new file mode 100644
index 000..81040f5
--- /dev/null
+++ b/debian/clean
@
Re: [Reproducible-builds] Bug#826711: diffoscope: please add argument completion
Hi, all ! Pabs, Thanks for suggestions :) I made the changes as you mentioned, please find an attached patches. On Sun, Jun 12, 2016 at 1:31 PM, Paul Wise <p...@debian.org> wrote: > On Sun, 2016-06-12 at 11:50 +0530, Satyam Zode wrote: > > > Tags: patch > > That only works in mails to sub...@bugs.debian.org, for mails to the > bug report address, you want this instead (-1 means the current bug): > > Control: tags -1 + patch > > > I have updated the patch as per suggestions please review it. > > Review below... > > > override_dh_clean: > > + debian/diffoscope.bash-completion > > This causes the package to fail to build. I think override_dh_clean > should be removed and both the bash completion script and > debian/diffoscope.1 added to debian/clean. Moving the existing things > to the debian/clean file should be done in a separate commit. > > > dh_python3 --recommends=python-debian --recommends=rpm-python > --recommends=tlsh --recommends=guestfs > > This line needs to add --recommends=argcomplete > > > +override_dh_auto_build: > > + register-python-argcomplete diffoscope > > debian/diffoscope.bash-completion > > This disables the upstream build system, you should call dh_auto_build > after calling register-python-argcomplete, I think this is a better way > to do it than what is there now: > > override_dh_auto_build: > debian/diffoscope.bash-completion debian/diffoscope.1 > > debian/diffoscope.bash-completion: > register-python-argcomplete diffoscope > $@ > I made above changes and here is build log : http://paste.debian.net/740041/ I am not sure if anything else is needed or not in debian/* files. > > You also need to add generated files to .gitignore so they don't get > accidentally committed to the repository. > > > +* ``python-argcomplete`` is used for argument completion. > > + Available on Debian and Fedora as > > + ``python-argcomplete``. > > + ``python-argcomplete`` is also available on `PyPI ` as > > + ``argcomplete``. > > I think I would write that like this: > > * ``argcomplete`` is used for argument completion. > Available on Debian as ``python3-argcomplete``. > Available on Fedora as ``python-argcomplete``. > Available on `PyPI <https://pypi.python.org/pypi/argcomplete/>`_. > > I have made these changes too. > > -default=Config.general.max_report_size) > > > + > default=Config.general.max_report_size).completer=RangeCompleter(0, > > +Config.general.max_report_size, 20) > > I think this prevents people from completing values above the > default max_report_size? Same for the other RangeCompleters. > > Interestingly, in the range completers, 100 appears to sort before > 80, so it is doing string sorting not numeric sorting. > I have removed this temporarily! we already had discussion regarding this on IRC. Looking forward to your response! Thanks! Satyam Zode From 43f3ed25ab3138cf353c3a34438cc7b8c8418e83 Mon Sep 17 00:00:00 2001 From: Satyam Zode <satyamz...@gmail.com> Date: Sat, 18 Jun 2016 22:12:58 +0530 Subject: [PATCH 4/4] Added new rules for bash-completion script --- debian/rules | 15 +-- 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/debian/rules b/debian/rules index b15a73b..a4bdf79 100755 --- a/debian/rules +++ b/debian/rules @@ -8,10 +8,10 @@ ifneq ($(VERSION_dch),$(VERSION_py)) endif %: - dh $@ --with python3 --buildsystem=pybuild + dh $@ --with python3 --with bash-completion --buildsystem=pybuild override_dh_python3: - dh_python3 --recommends=python-debian --recommends=rpm-python --recommends=tlsh --recommends=guestfs + dh_python3 --recommends=python-debian --recommends=rpm-python --recommends=tlsh --recommends=guestfs --recommends=argcomplete override_dh_gencontrol: TOOLS="$$(bin/diffoscope --list-tools=debian | tail -n 1 | \ @@ -23,13 +23,16 @@ override_dh_gencontrol: debian/diffoscope.1: debian/diffoscope.1.rst rst2man $< $@ +debian/diffoscope.bash-completion: + register-python-argcomplete diffoscope > $@ + +dh_auto_build: + debian/diffoscope.bash-completion + debian/diffoscope.1 + override_dh_installman: debian/diffoscope.1 dh_installman -O--buildsystem=pybuild -override_dh_clean: - rm -f debian/diffoscope.1 - dh_clean -O--buildsystem=pybuild - diffoscope/presenters/icon.py: favicon.png (echo '# Generated from favicon.png'; \ echo 'FAVICON_BASE64 = """'; \ -- 2.1.4 From d7192fb92a4fca051bee2b13548878923c34995e Mon Sep 17 00:00:00 2001 From: Satyam Zode <satyamz...@gmail.com> Date: Sat, 18 Jun 2016 22:05:19 +0530 Subject: [PATCH 3/4] Added debian/clean file --- debian/clean | 3 +++
Re: [Reproducible-builds] [diffoscope] Support for --ignore-profile flag in diffoscope
Hi, all ! I have been working on " hiding differences " options. I have tried to add some possible use cases, which I came across after researching different differences as suggested in above discussion,many thanks to all for their views. Here is wiki page https://wiki.debian.org/ReproducibleBuilds/HideProfilesSpecification Since, we have a couple of use cases. I think, it's a proper time to discuss and start thinking about implementation of these use cases as well as categorize the CLI interface for the options. Most importantly, we need to prioritize given use cases and decide which one to support and which one to ignore. Here, we also need to design these hiding options in such a way that user shall understand what he/she is hiding and how it'll affect the output. Please feel free to edit the wiki page to add new things. :) Looking forward to suggestions. Thanking you! Satyam Zode ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#826711: diffoscope: please add argument completion
Hi, all !
Paul Wise :
>
> Package: diffoscope
> Severity: wishlist
> Tags: newcomer
>
> Please add argument completion. Using the Python 3 argcomplete module
> makes this relatively simple. This project can be used as an example:
>
> https://anonscm.debian.org/cgit/collab-maint/check-all-the-things.git/
Please find an attached patch which enables argument completion for diffoscope.
Looking forward to review and feedback.
Thanks!
Satyam Zode
From 50dffe772b38355eb673994fdea49d9d74be7cac Mon Sep 17 00:00:00 2001
From: Satyam Zode <satyamz...@gmail.com>
Date: Sat, 11 Jun 2016 16:26:28 +0530
Subject: [PATCH] Added support for argument completion. Closes #826711
---
bin/diffoscope | 1 +
debian/control | 1 +
diffoscope/__main__.py | 7 +++
3 files changed, 9 insertions(+)
diff --git a/bin/diffoscope b/bin/diffoscope
index 2393807..2422b70 100755
--- a/bin/diffoscope
+++ b/bin/diffoscope
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# PYTHON_ARGCOMPLETE_OK
# -*- coding: utf-8 -*-
#
# diffoscope: in-depth comparison of files, archives, and directories
diff --git a/debian/control b/debian/control
index 19619d7..733d3be 100644
--- a/debian/control
+++ b/debian/control
@@ -20,6 +20,7 @@ Build-Depends:
python3-rpm,
python3-setuptools,
python3-tlsh (>= 3.4.1),
+ python3-argcomplete,
vim-common,
Standards-Version: 3.9.8
Homepage: https://diffoscope.org/
diff --git a/diffoscope/__main__.py b/diffoscope/__main__.py
index ac7913c..ab3739f 100644
--- a/diffoscope/__main__.py
+++ b/diffoscope/__main__.py
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+# PYTHON_ARGCOMPLETE_OK
# -*- coding: utf-8 -*-
#
# diffoscope: in-depth comparison of files, archives, and directories
@@ -30,6 +31,10 @@ try:
import tlsh
except ImportError:
tlsh = None
+try:
+import argcomplete
+except ImportError:
+argcomplete = None
from diffoscope import logger, VERSION, set_locale, clean_all_temp_files
import diffoscope.comparators
from diffoscope.config import Config
@@ -89,6 +94,8 @@ def create_parser():
parser.add_argument('file2', help='second file to compare')
if not tlsh:
parser.epilog = 'File renaming detection based on fuzzy-matching is currently disabled. It can be enabled by installing the “tlsh” module available at https://github.com/trendmicro/tlsh'
+if argcomplete:
+argcomplete.autocomplete(parser)
return parser
--
1.9.1
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Support for --ignore-profile flag in diffoscope
Hi, all ! I am Satyam Zode, I am GSoC student intern (http://satyamz.github.io /blog/2016/05/08/google-summer-of-code-2016-with-debian -reproducible-builds-introduction/). I am trying to understand the problem "Allow users to ignore arbitrary differences" ( --ignore-profiles flag) in which diffoscope users will be able to ignore arbitrary differences. This problem is also described in diffoscope wishlist (https://reproducible-builds.org/events/athens2015/ diffoscope-wishlist/). I have started thinking about the solution of this problem and for that, I want to know as a diffoscope user, what kind of stuff would you like to ignore ? Such as any irrelevant differences which are just making noise and useless. Currently, I am looking at pkg-diff (provided by OBS) code because it's able to ignore some kind of stuff (Lunar has suggested me to have a look at pkg-diff to get an idea about this problem) . I kindly request you all to express your views and expectations regarding this particular problem :) Thanking you! *Satyam Zode* ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#818856: Bug#818856: diffoscope: crashes on broken symlinks
Hi Reiner!
I did some trivial changes. Please find an attached patch.
This patch closes #818856 and #796262.
Thanks!
Satyam Zode
From 85c062d32b84ed6d8fcf5a80f0d2a2fbb4e43e54 Mon Sep 17 00:00:00 2001
From: Satyam Zode <satyamz...@gmail.com>
Date: Thu, 7 Apr 2016 01:15:04 +0530
Subject: [PATCH] Fixed issue related to diffoscope symlinks crashing (Closes:
#818856,#796262)
---
diffoscope/comparators/binary.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/diffoscope/comparators/binary.py b/diffoscope/comparators/binary.py
index 9663214..2b61538 100644
--- a/diffoscope/comparators/binary.py
+++ b/diffoscope/comparators/binary.py
@@ -182,8 +182,8 @@ class File(object, metaclass=ABCMeta):
def has_same_content_as(self, other):
logger.debug('%s has_same_content %s', self, other)
# try comparing small files directly first
-my_size = os.path.getsize(self.path)
-other_size = os.path.getsize(other.path)
+my_size = os.lstat(self.path).st_size
+other_size = os.lstat(other.path).st_size
if my_size == other_size and my_size <= SMALL_FILE_THRESHOLD:
if open(self.path, 'rb').read() == open(other.path, 'rb').read():
return True
--
2.1.4
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#818856: diffoscope: crashes on broken symlinks
Hi Reiner!
Thank you for reviewing patch. I have made all the changes you
mentioned above. Please find an attachment :-)
Cheers,
Satyam Zode
From 2e440a527e4689a6f4321695801b5cd04ff8642a Mon Sep 17 00:00:00 2001
From: Satyam Zode <satyamz...@gmail.com>
Date: Thu, 7 Apr 2016 00:31:42 +0530
Subject: [PATCH 1/2] Fixed issue related to diffoscope symlinks crashing
---
diffoscope/comparators/binary.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/diffoscope/comparators/binary.py b/diffoscope/comparators/binary.py
index 9663214..2b61538 100644
--- a/diffoscope/comparators/binary.py
+++ b/diffoscope/comparators/binary.py
@@ -182,8 +182,8 @@ class File(object, metaclass=ABCMeta):
def has_same_content_as(self, other):
logger.debug('%s has_same_content %s', self, other)
# try comparing small files directly first
-my_size = os.path.getsize(self.path)
-other_size = os.path.getsize(other.path)
+my_size = os.lstat(self.path).st_size
+other_size = os.lstat(other.path).st_size
if my_size == other_size and my_size <= SMALL_FILE_THRESHOLD:
if open(self.path, 'rb').read() == open(other.path, 'rb').read():
return True
--
2.1.4
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#818856: diffoscope: crashes on broken symlinks
Package: diffoscope Version: 51 Severity: normal Hi, Pabs! Please try an attached patch. Hopefully, it'll solve this problem. :) Thanks! Satyam Zode PICT, Pune On Sun, Apr 3, 2016 at 2:02 AM, Satyam Zode <satyamz...@gmail.com> wrote: > Hi, Pabs! > > Please try an attached patch! Hopefully, it'll solve this problem. :) > > Thank you! > Satyam > Satyam Zode > PICT, Pune > > > On Mon, Mar 21, 2016 at 8:02 AM, Paul Wise <p...@debian.org> wrote: >> Package: diffoscope >> Version: 51 >> Severity: normal >> >> I found a way to crash diffoscope with broken symlinks: >> >> pabs@chianamo ~ $ mkdir foo bar >> pabs@chianamo ~ $ touch foo/baz >> pabs@chianamo ~ $ ln -s asdasfahguwbagpiuasbdifgsabdf bar/baz >> pabs@chianamo ~ $ diffoscope foo bar >> Traceback (most recent call last): >> File "/usr/lib/python3/dist-packages/diffoscope/__main__.py", line 177, in >> main >> sys.exit(run_diffoscope(parsed_args)) >> File "/usr/lib/python3/dist-packages/diffoscope/__main__.py", line 148, in >> run_diffoscope >> parsed_args.file1, parsed_args.file2) >> File "/usr/lib/python3/dist-packages/diffoscope/comparators/__init__.py", >> line 92, in compare_root_paths >> return compare_directories(path1, path2) >> File "/usr/lib/python3/dist-packages/diffoscope/comparators/directory.py", >> line 104, in compare_directories >> return FilesystemDirectory(path1).compare(FilesystemDirectory(path2)) >> File "/usr/lib/python3/dist-packages/diffoscope/comparators/directory.py", >> line 157, in compare >> my_file, other_file, source=name) >> File "/usr/lib/python3/dist-packages/diffoscope/comparators/__init__.py", >> line 102, in compare_files >> if file1.has_same_content_as(file2): >> File "/usr/lib/python3/dist-packages/diffoscope/__init__.py", line 143, in >> tool_check >> return original_function(*args, **kwargs) >> File "/usr/lib/python3/dist-packages/diffoscope/comparators/binary.py", >> line 186, in has_same_content_as >> other_size = os.path.getsize(other.path) >> File "/usr/lib/python3.5/genericpath.py", line 50, in getsize >> return os.stat(filename).st_size >> FileNotFoundError: [Errno 2] No such file or directory: 'bar/baz' >> >> -- System Information: >> Debian Release: stretch/sid >> APT prefers testing >> APT policy: (900, 'testing'), (860, 'testing-proposed-updates'), (850, >> 'buildd-testing-proposed-updates'), (800, 'unstable'), (790, >> 'buildd-unstable'), (700, 'experimental'), (690, 'buildd-experimental'), >> (500, 'unstable-debug'), (1, 'experimental-debug') >> Architecture: amd64 (x86_64) >> >> Kernel: Linux 4.4.0-1-amd64 (SMP w/4 CPU cores) >> Locale: LANG=en_AU.utf8, LC_CTYPE=en_AU.utf8 (charmap=UTF-8) >> Shell: /bin/sh linked to /bin/dash >> Init: systemd (via /run/systemd/system) >> >> Versions of packages diffoscope depends on: >> ii python3-libarchive-c 2.1-3 >> ii python3-magic 1:5.25-2 >> ii python3-pkg-resources 18.8-1 >> pn python3:any >> >> Versions of packages diffoscope recommends: >> ii acl 2.2.52-3 >> ii binutils-multiarch 2.26-5 >> ii bzip2 1.0.6-8 >> ii caca-utils 0.99.beta19-2+b1 >> ii colord 1.2.12-1 >> ii cpio2.11+dfsg-5 >> pn default-jdk | java-sdk >> ii enjarify20151118-1 >> ii fontforge-extras0.3-4 >> pn fp-utils >> ii genisoimage 9:1.1.11-3 >> ii gettext 0.19.7-2 >> ii ghc 7.10.3-7 >> ii ghostscript 9.18~dfsg-4 >> ii gnupg 1.4.20-4 >> pn mono-utils >> ii pdftk 2.02-3 >> ii poppler-utils 0.38.0-2 >> ii python3-debian 0.1.27 >> pn python3-guestfs >> ii python3-rpm 4.12.0.1+dfsg1-3+b2 >> ii python3-tlsh3.4.4+20151206-1+b1 >> ii rpm2cpio4.12.0.1+dfsg1-3+b2 >> ii sng 1.1.0-1 >> ii sqlite3 3.11.1-1 >> ii squashfs-tools 1:4.3-3 >> ii unzip 6.0-20 >> ii vim-common 2:7.4.963-1+b2 >> ii xz-utils5.1.1alpha+20120614-2.1 >> >> Versions of packages diffoscope suggests: >> ii libjs-jquery 1.11.3+dfsg-4 >&g
Re: [Reproducible-builds] [GSoC 2016] : Application review
Hi all!
Jérémy Bobbio:
> Satyam Zode:
>> So this summer I intend to work on
>> 1) Improvements to diffoscope:
>> 1.1) Allow users to ignore arbitrary differences (Addition of
>> ignore-profiles flag).
>> 1.2) Perform fuzzy-matching across archives.
>> 1.3) Finish parallel processing part.
>> Above points are mentioned on GSoC wiki. And also there are more
>> features mentioned in whishlist
>> (https://reproducible-builds.org/events/athens2015/diffoscope-wishlist/)
>> I will try to cover some of those too.
>
> Sounds good. Be aware that the first part will require some design work.
> Finding the right UI regarding ignores might require input from the
> community. I'd recommend to split this into two parts
> (design+experiment+survey / implementation+documentation). You probably
> will want to work on other things in parellel with the discussions.
>
I have started thinking on design will soon let you all know. It'd be
great if you show me some directions towards designing the proposed
features.
(I will create new thread for this).
> Could you try to come up with rough estimations on how much time all of
> the above would require?
>
Yes! But first, I need some time for that (maybe upcoming 24 hours)
because I want to think more about features proposed above(I don't
know how much complex those features could be!).
Moving to the interesting part :)
> Fixing #818856 should not be too hard. If you could submit a patch that
> would make more confident that you could do all the above.
>
Thank you so much for giving me this task :-) . Now I know how
diffoscope can be tested and build from code. (Before this I was only
familiar with codebase and diffoscope functionalities )
I have fixed this issue. I also have fixed link in the documentation.
Please find an attachment.
Here is the output which I have got:
satyam@satyamz:~/Debian/experiment/diffoscope/bin$ mkdir foo bar
satyam@satyamz:~/Debian/experiment/diffoscope/bin$ touch foo/baz
satyam@satyamz:~/Debian/experiment/diffoscope/bin$ ln -s somefile bar/baz
satyam@satyamz:~/Debian/experiment/diffoscope/bin$ ./diffoscope foo bar
--- foo
+++ bar
├── stat {}
│ @@ -1,8 +1,8 @@
│
│Size: 4096 Blocks: 8 IO Block: 4096 directory
│ Links: 2
│ Access: (0755/drwxr-xr-x) Uid: ( 1000/ satyam) Gid: ( 1000/ satyam)
│
│ -Modify: 2016-03-21 17:22:54.004395232 +
│ +Modify: 2016-03-21 17:23:20.516394537 +
│
│ Birth: -
│ --- foo/baz
├── +++ bar/baz
│ @@ -0,0 +1,2 @@
│ +000: 6465 7374 696e 6174 696f 6e3a 2073 6f6d destination: som
│ +010: 6566 696c 650a efile.
│ ├── stat {}
│ │ @@ -1,8 +1,8 @@
│ │
│ │ - Size: 0 Blocks: 0 IO Block: 4096 regular empty file
│ │ + Size: 8 Blocks: 0 IO Block: 4096 symbolic link
│ │ Links: 1
│ │ -Access: (0644/-rw-r--r--) Uid: ( 1000/ satyam) Gid: ( 1000/ satyam)
│ │ +Access: (0777/lrwxrwxrwx) Uid: ( 1000/ satyam) Gid: ( 1000/ satyam)
│ │
│ │ -Modify: 2016-03-21 17:22:54.004395232 +
│ │ +Modify: 2016-03-21 17:23:20.516394537 +
│ │
│ │ Birth: -
│ ╵
╵
satyam@satyamz:~/Debian/experiment/diffoscope/bin$
> Most of the improvements we could think of have indeed been implemented
> since. :)
Nice :)
I also want to know, whether I will be able to edit application or not
after a deadline?
Thanks again!
Satyam Zode
From 3e9aea18767099dffe62c14e7215aed54347a10f Mon Sep 17 00:00:00 2001
From: Satyam Zode <satyamz...@gmail.com>
Date: Mon, 21 Mar 2016 23:12:55 +0530
Subject: [PATCH 1/2] fixed issue related to diffoscope symlinks crashing
---
diffoscope/comparators/binary.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/diffoscope/comparators/binary.py b/diffoscope/comparators/binary.py
index 9663214..5622a9c 100644
--- a/diffoscope/comparators/binary.py
+++ b/diffoscope/comparators/binary.py
@@ -183,7 +183,7 @@ class File(object, metaclass=ABCMeta):
logger.debug('%s has_same_content %s', self, other)
# try comparing small files directly first
my_size = os.path.getsize(self.path)
-other_size = os.path.getsize(other.path)
+other_size = os.lstat(other.path).st_size
if my_size == other_size and my_size <= SMALL_FILE_THRESHOLD:
if open(self.path, 'rb').read() == open(other.path, 'rb').read():
return True
--
2.1.4
From 79809c35a402f1e28f1c3f7c94985274172c0415 Mon Sep 17 00:00:00 2001
From: Satyam Zode <satyamz...@gmail.com>
Date: Mon, 21 Mar 2016 23:25:27 +0530
Subject: [PATCH 2/2] Fixed link in documentation
---
README.rst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/README.rst b/README.rst
index fe23a37..5288e52 100644
--- a/README.rst
+++ b/README.rst
@@ -83,7 +83,7 @@ system against the diffoscope package:
Join the users and developers mail
Re: [Reproducible-builds] [GSoC 2016] : Application review
Hi, everyone! Jérémy Bobbio: > Thanks for your application. I much appreciated that it's done before > the deadline. I also like you being clear of your other commitments. > > I think it would have been fine application for last summer, but we've > made significant progress on several fronts since, therefore I'm not > convinced that there's much work left in the tasks your propose. I think > it would be better to aim with more precise tasks, e.g. toolchain > software you'll be improving to solve classes of issues, or at leas an > outline the features you intend to add to strip-nondeterminism or > diffoscope. > > (If you feel you're part of the reproducible builds team and disagree > with my comments, please say so!) Thanks Lunar for this valuable feedback. Yes, I am agree with you. After reading the reasons( which you mentioned) and as a part of reproducible builds team I don't think the proposed work(don't need whole summer to work on) by me will help much to reproducible builds effort too. But I think there are some issues which still needs to be fixed. There are some issues in which not even a single package have patch. I will try to look into those and will try to search for solutions. So this summer I intend to work on 1) Improvements to diffoscope: 1.1) Allow users to ignore arbitrary differences (Addition of ignore-profiles flag). 1.2) Perform fuzzy-matching across archives. 1.3) Finish parallel processing part. Above points are mentioned on GSoC wiki. And also there are more features mentioned in whishlist (https://reproducible-builds.org/events/athens2015/diffoscope-wishlist/) I will try to cover some of those too. I guess Better/smarter ELF diffing is underdevelopment (I have checked git logs and diffoscope for same) 2) Improving reproducibility of Debian packages: In this section I will be fixing Debian packages and will try to find the solutions to the issues which do not have solution yet. I am trying to enlist such issues. > If you look at packages identified as leaving timestamps in gzip > headers, you'll see that most of them already have patches, and the ones > who don't are affected with other issues > https://tests.reproducible-builds.org/issues/unstable/timestamps_in_gzip_headers_issue.html > These other issues probably deter maintainers' motivation to fix the > problems with gzip timestamps. > > Almost all packages with varying mtimes in data.tar or control.tar have > patches or have been fixed through toolchain improvements: > https://tests.reproducible-builds.org/issues/unstable/varying_mtimes_in_data_tar_gz_or_control_tar_gz_issue.html > > It feels quite suboptimal to highlight user and groups in tarballs as > separate issues as I think all are affected by other tarball related > issues. They should be fixed at the same time: > https://tests.reproducible-builds.org/issues/unstable/users_and_groups_in_tarball_issue.html > > Regarding timestamps due to C pre-processor macros, Dhole is waiting > for GCC patch window to open again—which will be in April, IIRC. > So unless you intend to work on adding support for SOURCE_DATE_EPOCH in > clang, I'm not sure there's much work left on this issue. I believe that > fixing the 400+ packages individually should not be undertaken if > we can avoid it. > https://gcc.gnu.org/ml/gcc-patches/2015-12/msg01402.html > https://wiki.debian.org/ReproducibleBuilds/TimestampsFromCPPMacros > > Emmanuel Bourg has been working and fixing almost all Java-related > issues in the course of the past year. I expect he'll probably work on > this fixing locale related javadoc issue in a near future. I guess you > could coordinate with him to write the necessary patches, though. > https://tests.reproducible-builds.org/issues/unstable/locale_in_documentation_generated_by_javadoc_issue.html > A big thanks to you, because I really didn't know about many of the above things. Its good to know that people are already working on this part. :-) > These quick evaluations leave me the feeling that your proposed schedule > is currently not adequate with actual needs of the reproducible builds > effort. > > This probably means that progress can be made on making more visible > areas that actually require work… Please let me know what you think about the work which I have proposed now. I will frame timeline accordingly. PS: I really feel that I am part of reproducible builds and I want to strengthen the bond by spending my summer working with reproducible builds ;-) ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#818111: debian-keyring: please make the build reproducible (locale, fileordering)
Source: debian-keyring Version: 2016.01.20 Severity: wishlist Tags: patch User: reproducible-builds@lists.alioth.debian.org Usertags: fileordering X-Debbugs-Cc: reproducible-builds@lists.alioth.debian.org Hi! While working on the “reproducible builds” effort [1], we have noticed that debian-keyring could not be built reproducibly. The attached patch fix the order of files in md5sums. Once applied, debian-keyring can be built reproducibly in our current experimental framework. [1]: https://wiki.debian.org/ReproducibleBuilds Thank you! Satyam Zode PICT, Pune diff -Nru debian-keyring-2015.04.10/debian/changelog debian-keyring-2015.04.10.0~reproducible1/debian/changelog --- debian-keyring-2015.04.10/debian/changelog 2015-04-10 14:53:32.0 +0530 +++ debian-keyring-2015.04.10.0~reproducible1/debian/changelog 2016-03-13 23:59:02.0 +0530 @@ -1,3 +1,9 @@ +debian-keyring (2015.04.10.0~reproducible1) UNRELEASED; urgency=medium + + * Made debian-keyring reproducible + + -- Satyam Zode <satyamz...@gmail.com> Sun, 13 Mar 2016 23:57:03 +0530 + debian-keyring (2015.04.10) unstable; urgency=medium [ Gunnar Wolf ] diff -Nru debian-keyring-2015.04.10/debian/rules debian-keyring-2015.04.10.0~reproducible1/debian/rules --- debian-keyring-2015.04.10/debian/rules 2015-03-31 04:23:17.0 +0530 +++ debian-keyring-2015.04.10.0~reproducible1/debian/rules 2016-03-14 00:01:43.0 +0530 @@ -53,7 +53,8 @@ chown -R root.root debian/tmp chmod -R go=rX debian/tmp - cd debian/tmp && find . -type f ! -regex '.*DEBIAN/.*' -printf '%P\0' | xargs -r0 md5sum > DEBIAN/md5sums + cd debian/tmp && find . -type f ! -regex '.*DEBIAN/.*' -printf '%P\0' | \ + LC_ALL=C sort -z | xargs -r0 md5sum > DEBIAN/md5sums find debian/tmp -depth -newermt '$(BUILD_DATE)' -print0 | \ xargs -0r touch --no-dereference --date='$(BUILD_DATE)' ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
