git-dpm: ERROR: 'upstream' does not contain previously recorded revision

[Ludovic Rousseau]

Hello,

I tried to update my package pykcs11 since the repository moved from SVN to
git.
http://anonscm.debian.org/cgit/python-modules/packages/pykcs11.git/

I imported a new upstream .orig.tar.gz version as descibed in
https://wiki.debian.org/Python/GitPackaging#New_upstream_release
and could build and upload the package without problem.

Now the command "git-dpm status" fails with:
$ git-dpm status
git-dpm: ERROR: 'upstream' does not contain previously recorded revision
'8234fc2de2b89d65661233b357f922494590b5aa'!

and also
$ git-dpm tag
git-dpm: ERROR: 'upstream' differs from recorded one!

I think I missed something.
I don't want to add more errors/problems in the git repository. I am very
new to git-dpm process and I don't want to make the repository (more)
unusable.

Can someone tell me what I missed and fix the error?
Or describe in detail how to fix the problem?

Thanks

-- 
 Dr. Ludovic Rousseau

Re: pybuild sphinxdoc and extensions

[Piotr Ożarowski]

[Piotr Ożarowski, 2015-10-23]
>   override_dh_auto_build:
>   dh_auto_build
>   PYTHONPATH=`pybuild --build -i python3 -s custom --build-args 'echo 
> {build_dir}'`\
>   $(MAKE) -C doc html

last one, I promise:

  override_dh_auto_build:
dh_auto_build
pybuild --build -i python3 -s custom --build-args 'make -C {dir}/doc 
html'
-- 
evil schizophrenic general Piotr who wants to have the last word in the 
discussion

Re: pybuild sphinxdoc and extensions

[Dimitri John Ledkov]

i knew it! there are three identical twins of Piotr. No wonder they
get so much done!

On 23 October 2015 at 15:22, Piotr Ożarowski  wrote:
> [Piotr Ożarowski, 2015-10-23]
>> [Piotr Ożarowski, 2015-10-23]
>> >   override_dh_auto_build:
>> > dh_auto_build
>> > PYTHONPATH=$(CURDIR)/.pybuild/build_cpython3/ $(MAKE) -C doc html
>> >
>> > no matter which Python 3.X version is the default one
>> > (it still hardcodes ".pybuild" which I don't like, though)
>>
>> which leads back to `pybuild --interpreter python3 --echo {build_dir}`
>> (i.e. new --echo parameter which I suggested last year)
>
> - LOL, you fool, you implemented it already!
>
>   override_dh_auto_build:
> dh_auto_build
> PYTHONPATH=`pybuild --build -i python3 -s custom --build-args 'echo 
> {build_dir}'`\
> $(MAKE) -C doc html
>
> - oh, you're right, Piotr
> - yeah I'm right! You should listed to me more!
> - bite me!
> --
> evil schizophrenic general Piotr
>



-- 
Regards,

Dimitri.

Re: pybuild sphinxdoc and extensions

[Piotr Ożarowski]

[Piotr Ożarowski, 2015-10-23]
>   override_dh_auto_install:
>   dh_auto_install
>   PYTHONPATH=$(CURDIR)/debian/python3-kwant/usr/lib/python3/dist-packages 
> \
>   $(MAKE) -C doc html

you might even need to do it later: dh_python3 is not called yet, so
some files might not be moved to /usr/lib/python3/dist-packages yet.
If you decide to override_dh_python3, remember to clean __pycache__
later.

/me thinks he has to provide a simpler way, so that build stage can used...
How about creating .pybuild/build_python3 symlink to whatever is the
default Python3 version? This would make something like this possible:

  override_dh_auto_build:
dh_auto_build
PYTHONPATH=$(CURDIR)/.pybuild/build_cpython3/ $(MAKE) -C doc html

no matter which Python 3.X version is the default one
(it still hardcodes ".pybuild" which I don't like, though)
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

Re: pybuild sphinxdoc and extensions

[Piotr Ożarowski]

[Piotr Ożarowski, 2015-10-23]
>   override_dh_auto_build:
>   dh_auto_build
>   PYTHONPATH=$(CURDIR)/.pybuild/build_cpython3/ $(MAKE) -C doc html
> 
> no matter which Python 3.X version is the default one
> (it still hardcodes ".pybuild" which I don't like, though)

which leads back to `pybuild --interpreter python3 --echo {build_dir}`
(i.e. new --echo parameter which I suggested last year)
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

Re: pybuild sphinxdoc and extensions

[Lennart Sorensen]

On Fri, Oct 23, 2015 at 04:41:56PM +0100, Dimitri John Ledkov wrote:
> i knew it! there are three identical twins of Piotr. No wonder they
> get so much done!

If it is three, wouldn't that be identical triplets?

-- 
Len Sorensen

Re: Bug#802839: django-celery: python 3 tests not invoked and break

[Robert Collins]

I'd probably shut that warning up using the warnings module API rather
than weaking the test more broadly. Also - track down and file a bug
on the leak source.

Re: Bug#802839: django-celery: python 3 tests not invoked and break

[Brian May]

Brian May  writes:

> ==
> FAIL: test_discovery_with_broken (djcelery.tests.test_discovery.TestDiscovery)
> --
> Traceback (most recent call last):
>   File 
> "/home/brian/tree/debian/python-modules/django-celery/tests/../djcelery/tests/test_discovery.py",
>  line 45, in test_discovery_with_broken
> self.assertEqual(log, [])
> AssertionError: Lists differ: [ 0x7f2b03474c88>] != []
>
> First list contains 1 additional elements.
> First extra element 0:
> {message : ResourceWarning("unclosed file <_io.TextIOWrapper 
> name='/home/brian/tree/debian/python-modules/django-celery/tests/someapp/tasks.py'
>  mode='r' encoding='utf-8'>",), category : 'ResourceWarning', filename : 
> '/home/brian/tree/debian/python-modules/django-celery/tests/../djcelery/loaders.py',
>  lineno : 204, line : None}
>
> - []
> + []
>
> --
> Ran 66 tests in 0.340s
>
> FAILED (errors=2, failures=1)

That includes a debug message I put in. Oops. Showing the warning object
that caused the problem.

It seems that Python3 is generating an unexpected error ("unclosed
file") that is upsetting the test because it expects 0 warnings.

Any ideas how to fix this? It doesn't appear to be this project that is
producing the warning, maybe that comes from Django.

Or should I just remove the Assert?

The code in question that appears to be producing the warning:

def autodiscover():
"""Include tasks for all applications in ``INSTALLED_APPS``."""
global _RACE_PROTECTION

if _RACE_PROTECTION:
return
_RACE_PROTECTION = True
try:
if django.VERSION < (1, 7):
return filter(None, [find_related_module(app, 'tasks')
 for app in settings.INSTALLED_APPS])
else:
from django.apps import apps
return filter(None, [find_related_module(app.name, 'tasks')
 for app in apps.get_app_configs()])
finally:
_RACE_PROTECTION = False


This is in git, in the expected python-modules location.
-- 
Brian May 

Re: pybuild sphinxdoc and extensions

[Piotr Ożarowski]

[Piotr Ożarowski, 2015-10-23]
> [Piotr Ożarowski, 2015-10-23]
> >   override_dh_auto_build:
> > dh_auto_build
> > PYTHONPATH=$(CURDIR)/.pybuild/build_cpython3/ $(MAKE) -C doc html
> > 
> > no matter which Python 3.X version is the default one
> > (it still hardcodes ".pybuild" which I don't like, though)
> 
> which leads back to `pybuild --interpreter python3 --echo {build_dir}`
> (i.e. new --echo parameter which I suggested last year)

- LOL, you fool, you implemented it already!

  override_dh_auto_build:
dh_auto_build
PYTHONPATH=`pybuild --build -i python3 -s custom --build-args 'echo 
{build_dir}'`\
$(MAKE) -C doc html

- oh, you're right, Piotr
- yeah I'm right! You should listed to me more!
- bite me!
-- 
evil schizophrenic general Piotr

Re: git-dpm: ERROR: 'upstream' does not contain previously recorded revision

[Brian May]

Ludovic Rousseau  writes:

> I tried to update my package pykcs11 since the repository moved from SVN to
> git.
> http://anonscm.debian.org/cgit/python-modules/packages/pykcs11.git/
>
> I imported a new upstream .orig.tar.gz version as descibed in
> https://wiki.debian.org/Python/GitPackaging#New_upstream_release
> and could build and upload the package without problem.
>
> Now the command "git-dpm status" fails with:
> $ git-dpm status
> git-dpm: ERROR: 'upstream' does not contain previously recorded revision
> '8234fc2de2b89d65661233b357f922494590b5aa'!

At a guess, it looks like you didn't push the upstream or the
prestine-tar branches to git.debian.org. git-dpm requires all branches
to be pushed, not just master.

git push --all

i.e. pykcs11_1.3.0.orig.tar.gz isn't in prestine-tar, and the upstream
branch should point to the latest upstream import
'8234fc2de2b89d65661233b357f922494590b5aa' but doesn't.

However this shouldn't have caused you errors with your working
directory unless you deleted your working directory and cloned again
from git.debian.org.

> and also
> $ git-dpm tag
> git-dpm: ERROR: 'upstream' differs from recorded one!

This is a known bug with git-dpm. See
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=801548.

Until it is fixed, the best work around seems to be to create the tags
manually, at least until you import a new upstream version. Then the
problem will go away.
-- 
Brian May 

Re: pybuild sphinxdoc and extensions

[Christoph Groth]

I found this thread while trying to update the Debian packaging 
for a Python library that I maintain.  The library has Sphinx 
documentation that is quite complex to build:  First, the library 
needs to be built (it contains C extensions), then figures are 
generated by scripts, and finally Sphinx is run.  This is all done 
in a Makefile.


Piotr Ożarowski wrote:


[PICCA Frederic-Emmanuel, 2014-04-06]
> so I would like to know how to change this snipset to use the
> {build_dir} for the PYTHONPATH.

you cannot really use pybuild for this, it will invoke your 
command
after or before each interpreter/version call. If you hardcode 
pybuild's

internal paths on the other hand (.pybuild/something in case of
distutils build plugin), it will break once the internal path 
changes...


I suggest to move building docs to install target and add
debian/foo/usr/lib/python3/dist-packages) to PYTHONPATH. I also 
build
directly into debian/foo/usr/share/doc/foo/html so I feel a 
little less

guilty for doing it in install step ;)


With “install target”, do you mean dh_installdocs?  In our 
debian/rules, we have now


# Make documentation only if needed.
ifneq "$(shell dh_listpackages | grep -- -doc)" ""
override_dh_installdocs:
	cd doc && PYTHONPATH=$(abspath 
	debian/python-kwant/usr/lib/python2.7/dist-packages) 
	$(MAKE) html

dh_installdocs
endif

This seems to work, but is this what you meant?  Also, I have two 
questions:


• How to get rid of the explicit mention of python2.7?

• Is the “ifneq”-clause the proper way to ensure that 
 documentation is only built for binary-indep?


I have one further more general question with regard to Debian 
packaging with pybuild.  Pybuild deletes the *.egg-info directory 
but that directory is included in the tarball as made by “setup.py 
sdist”.  (I am aware that some people consider that directory to 
be generated data, but this is debatable [1], and it’s standard 
practice to include it in the tarball anyway.)  This has the 
consequence that after running “gbp buildpackage”, some files are 
missing and thus “gbp buildpackage” cannot be re-run without “git 
checkout .”.  This situation is unsatisfactory.  Is there a 
recommended solution?


The complete Debian packaging of our library is available in a git 
repository [2].  I hope that one day (rather soon) it could be 
added to Debian proper.


Thanks for any hints,
Christoph

[1] https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2092#comment:16
[2] http://git.kwant-project.org/debian-kwant/


signature.asc
Description: PGP signature

Sorry for posting the same message twice

[Christoph Groth]

Hello again,

Sorry for posting the same message twice.  I believed that it got
silently dropped as I did not see it show up on gmane (this was because
I replied to an old thread).  So I subscribed to
whitel...@lists.debian.org and resent it.

Christoph


signature.asc
Description: PGP signature

Re: pybuild sphinxdoc and extensions

[Piotr Ożarowski]

[please don't CC me on a mailing list if you don't want to end up in SPAM]

[Christoph Groth, 2015-10-22]
> With “install target”, do you mean dh_installdocs?  In our debian/rules, we

I meat override_dh_auto_install or {install,binary}-{arch,indep}

> have now
> 
> # Make documentation only if needed.
> ifneq "$(shell dh_listpackages | grep -- -doc)" ""
> override_dh_installdocs:
>   cd doc && PYTHONPATH=$(abspath
>   debian/python-kwant/usr/lib/python2.7/dist-packages)$(MAKE) html
>   dh_installdocs
> endif

that should work too, but I'd suggest to use:

  override_dh_auto_install:
dh_auto_install
PYTHONPATH=$(CURDIR)/debian/python3-kwant/usr/lib/python3/dist-packages 
\
$(MAKE) -C doc html

> • How to get rid of the explicit mention of python2.7?

no need, we will not support 2.6 or 2.8 so it's OK to hardcode 2.7
and for Python 3.X use above example

> • Is the “ifneq”-clause the proper way to ensure that  documentation is only
> built for binary-indep?

I'd use:
  ifeq (,$(filter nodoc,$(DEB_BUILD_OPTIONS)))
...
  endif

in override_dh_auto_install-indep target, but neither nodoc or nodocs is
official (vide: #759186)

> I have one further more general question with regard to Debian packaging
> with pybuild.  Pybuild deletes the *.egg-info directory but that directory
> is included in the tarball as made by “setup.py sdist”.  (I am aware that
> some people consider that directory to be generated data, but this is
> debatable [1], and it’s standard practice to include it in the tarball

I'm lazy and now that DPMT keeps full upstream sources in git and I have
to add .gitignore to all packages I touch, I will most probably remove
the feature of deleting evil egg-info and just log an info about adding
"*.egg-info" to debian/clean or "extend-diff-ignore="^[^/]+\.egg-info/"
to debian/source/options"
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645