Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[Luca Boccassi]

On Tue, 29 Nov 2016 22:10:47 +0100 Julian Taylor  wrote:
> On 29.11.2016 15:10, Tobias Hansen wrote:
> > On Thu, 3 Nov 2016 20:25:23 +0100 Julian Taylor
> >  wrote:
> >> On 03.11.2016 20:17, Sandro Tosi wrote:
>  Looks like your last upload fixed the build on most
architectures. However the
>  package still failed to build on arm64.
> 
>  https://buildd.debian.org/status/package.php?p=pyzmq
> >>>
> >>> Hey Julian, do you have time to look at the failures in the
> >>> https://buildd.debian.org/status/package.php?p=pyzmq ? it looks
like
> >>> upstream released 16.0.0 so it might worth a shot packaging it to
see
> >>> if that solves them. thanks!!
> >>>
> >>
> >> I have packaged it though the tests are also unreliable on x86,
still
> >> needs to be looked at.
> >>
> > 
> > Could it be that zeromq3 4.2.0 (which is now in unstable) helps?
> > 
> > It would be nice if this could be fixed before the soft freeze on
> > January 5. We are trying to get sagemath (currently not uploaded
yet) to
> > migrate to testing before that date and it could become a problem
to
> > have different versions of pyzmq (and zeromq3) in unstable and
testing.
> > 
> 
> 
> No luck still hangs sometimes and I still had no time to properly
look
> into it :/
> Also the decision to not provide aligned even 4 byte aligned payload
> buffers in zmq 4.2 as indicated in #844479 is going to be a minefield
in
> our arches without unaligned access ...

Hi,

Note that there's a new option in libzmq 4.2.5 (still in DRAFT so not
available without rebuilding for now) to disable the zero-copy receive
that was added in 4.2.0:

https://github.com/zeromq/libzmq/blob/master/doc/zmq_ctx_set.txt#L130

I'm not sure in which use case payloads would need to be aligned, but
for such programs you'll be able to set it - if you have already
encountered issues you can rebuild libzmq (or get the packages from our
OBS repos: https://software.opensuse.org/download.html?project=network%
3Amessaging%3Azeromq%3Arelease-draft=libzmq3-dev ) and try it
out.

-- 
Kind regards,
Luca Boccassi

signature.asc
Description: This is a digitally signed message part

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[Julian Taylor]

On 29.11.2016 15:10, Tobias Hansen wrote:
> On Thu, 3 Nov 2016 20:25:23 +0100 Julian Taylor
>  wrote:
>> On 03.11.2016 20:17, Sandro Tosi wrote:
 Looks like your last upload fixed the build on most architectures. However 
 the
 package still failed to build on arm64.

 https://buildd.debian.org/status/package.php?p=pyzmq
>>>
>>> Hey Julian, do you have time to look at the failures in the
>>> https://buildd.debian.org/status/package.php?p=pyzmq ? it looks like
>>> upstream released 16.0.0 so it might worth a shot packaging it to see
>>> if that solves them. thanks!!
>>>
>>
>> I have packaged it though the tests are also unreliable on x86, still
>> needs to be looked at.
>>
> 
> Could it be that zeromq3 4.2.0 (which is now in unstable) helps?
> 
> It would be nice if this could be fixed before the soft freeze on
> January 5. We are trying to get sagemath (currently not uploaded yet) to
> migrate to testing before that date and it could become a problem to
> have different versions of pyzmq (and zeromq3) in unstable and testing.
> 


No luck still hangs sometimes and I still had no time to properly look
into it :/
Also the decision to not provide aligned even 4 byte aligned payload
buffers in zmq 4.2 as indicated in #844479 is going to be a minefield in
our arches without unaligned access ...

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[Tobias Hansen]

On Thu, 3 Nov 2016 20:25:23 +0100 Julian Taylor
 wrote:
> On 03.11.2016 20:17, Sandro Tosi wrote:
> >> Looks like your last upload fixed the build on most architectures. However 
> >> the
> >> package still failed to build on arm64.
> >>
> >> https://buildd.debian.org/status/package.php?p=pyzmq
> > 
> > Hey Julian, do you have time to look at the failures in the
> > https://buildd.debian.org/status/package.php?p=pyzmq ? it looks like
> > upstream released 16.0.0 so it might worth a shot packaging it to see
> > if that solves them. thanks!!
> > 
> 
> I have packaged it though the tests are also unreliable on x86, still
> needs to be looked at.
> 

Could it be that zeromq3 4.2.0 (which is now in unstable) helps?

It would be nice if this could be fixed before the soft freeze on
January 5. We are trying to get sagemath (currently not uploaded yet) to
migrate to testing before that date and it could become a problem to
have different versions of pyzmq (and zeromq3) in unstable and testing.

Best,
Tobias

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[Julian Taylor]

On 03.11.2016 20:17, Sandro Tosi wrote:
>> Looks like your last upload fixed the build on most architectures. However 
>> the
>> package still failed to build on arm64.
>>
>> https://buildd.debian.org/status/package.php?p=pyzmq
> 
> Hey Julian, do you have time to look at the failures in the
> https://buildd.debian.org/status/package.php?p=pyzmq ? it looks like
> upstream released 16.0.0 so it might worth a shot packaging it to see
> if that solves them. thanks!!
> 

I have packaged it though the tests are also unreliable on x86, still
needs to be looked at.

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[Sandro Tosi]

> Looks like your last upload fixed the build on most architectures. However the
> package still failed to build on arm64.
>
> https://buildd.debian.org/status/package.php?p=pyzmq

Hey Julian, do you have time to look at the failures in the
https://buildd.debian.org/status/package.php?p=pyzmq ? it looks like
upstream released 16.0.0 so it might worth a shot packaging it to see
if that solves them. thanks!!

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[Emilio Pozuelo Monfort]

On 20/04/16 00:33, Julian Taylor wrote:
> On 13.04.2016 15:39, Julian Taylor wrote:
>> On 04/13/2016 03:24 PM, Emilio Pozuelo Monfort wrote:
>>> On 29/03/16 19:10, László Böszörményi (GCS) wrote:
 Hi Julian,

 On Sun, Mar 20, 2016 at 12:27 PM, Julian Taylor
  wrote:
> On 20.03.2016 11:11, László Böszörményi (GCS) wrote:
>>   Sorry, my life was chaotic. Yes and no, checked it. First, there's a
>> new upstream version of pyzmq (15.2) for two months. It seems to be
>> security related according to the release log[1]:
>> - FIX: unicode/bytes bug in password prompt in zmq.ssh on Python 3
>> - workaround overflow bug in libzmq preventing receiving messages
>> larger than MAX_INT
 [...]
> The latest version does unfortunately not fix the problem. It is also
> not security related, it just could not send large data due to bugs in 
> zmq.
> I guess a good approach would be to bisect zmq to see what they changed
> to cause it.
   Just a friendly ping; could you test the latest Git master if that
 fixes the hang and/or could you ask upstream for advice? Should I do
 it? It seems it constantly get fixes that may be relevant.
>>>
>>> The package is team maintained, so I would say go ahead and do a team 
>>> upload.
>>>
>>
>> I see nothing in git that might be related and I'm not a fan of just
>> doing a guess upload but if you like go ahead.
>> Note that you have to disable the new large send test as it will fail on
>> 32 bit and possibly swap on 64 bit.
>> I still need to get porterbox access before I can look at it, but if
>> someone can bisect libzmq before that happens for me would be great.
>>
> 
> here is what I have been able to figure out on powerpc, not much 
> unfortunately.
> https://github.com/zeromq/pyzmq/issues/838
> 

Looks like your last upload fixed the build on most architectures. However the
package still failed to build on arm64.

https://buildd.debian.org/status/package.php?p=pyzmq

Cheers,
Emilio

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[Julian Taylor]

On 13.04.2016 15:39, Julian Taylor wrote:

On 04/13/2016 03:24 PM, Emilio Pozuelo Monfort wrote:

On 29/03/16 19:10, László Böszörményi (GCS) wrote:

Hi Julian,

On Sun, Mar 20, 2016 at 12:27 PM, Julian Taylor
 wrote:

On 20.03.2016 11:11, László Böszörményi (GCS) wrote:

  Sorry, my life was chaotic. Yes and no, checked it. First, there's a
new upstream version of pyzmq (15.2) for two months. It seems to be
security related according to the release log[1]:
- FIX: unicode/bytes bug in password prompt in zmq.ssh on Python 3
- workaround overflow bug in libzmq preventing receiving messages
larger than MAX_INT

[...]

The latest version does unfortunately not fix the problem. It is also
not security related, it just could not send large data due to bugs in zmq.
I guess a good approach would be to bisect zmq to see what they changed
to cause it.

  Just a friendly ping; could you test the latest Git master if that
fixes the hang and/or could you ask upstream for advice? Should I do
it? It seems it constantly get fixes that may be relevant.


The package is team maintained, so I would say go ahead and do a team upload.



I see nothing in git that might be related and I'm not a fan of just
doing a guess upload but if you like go ahead.
Note that you have to disable the new large send test as it will fail on
32 bit and possibly swap on 64 bit.
I still need to get porterbox access before I can look at it, but if
someone can bisect libzmq before that happens for me would be great.



here is what I have been able to figure out on powerpc, not much 
unfortunately.

https://github.com/zeromq/pyzmq/issues/838

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[Emilio Pozuelo Monfort]

On 13/04/16 15:39, Julian Taylor wrote:
> On 04/13/2016 03:24 PM, Emilio Pozuelo Monfort wrote:
>> On 29/03/16 19:10, László Böszörményi (GCS) wrote:
>>> Hi Julian,
>>>
>>> On Sun, Mar 20, 2016 at 12:27 PM, Julian Taylor
>>>  wrote:
 On 20.03.2016 11:11, László Böszörményi (GCS) wrote:
>  Sorry, my life was chaotic. Yes and no, checked it. First, there's a
> new upstream version of pyzmq (15.2) for two months. It seems to be
> security related according to the release log[1]:
> - FIX: unicode/bytes bug in password prompt in zmq.ssh on Python 3
> - workaround overflow bug in libzmq preventing receiving messages
> larger than MAX_INT
>>> [...]
 The latest version does unfortunately not fix the problem. It is also
 not security related, it just could not send large data due to bugs in zmq.
 I guess a good approach would be to bisect zmq to see what they changed
 to cause it.
>>>  Just a friendly ping; could you test the latest Git master if that
>>> fixes the hang and/or could you ask upstream for advice? Should I do
>>> it? It seems it constantly get fixes that may be relevant.
>>
>> The package is team maintained, so I would say go ahead and do a team upload.
>>
> 
> I see nothing in git that might be related and I'm not a fan of just
> doing a guess upload but if you like go ahead.
> Note that you have to disable the new large send test as it will fail on
> 32 bit and possibly swap on 64 bit.
> I still need to get porterbox access before I can look at it, but if
> someone can bisect libzmq before that happens for me would be great.

Did you already request access per https://dsa.debian.org/doc/guest-account/ ?

Cheers,
Emilio

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[Julian Taylor]

On 04/13/2016 03:24 PM, Emilio Pozuelo Monfort wrote:
> On 29/03/16 19:10, László Böszörményi (GCS) wrote:
>> Hi Julian,
>>
>> On Sun, Mar 20, 2016 at 12:27 PM, Julian Taylor
>>  wrote:
>>> On 20.03.2016 11:11, László Böszörményi (GCS) wrote:
  Sorry, my life was chaotic. Yes and no, checked it. First, there's a
 new upstream version of pyzmq (15.2) for two months. It seems to be
 security related according to the release log[1]:
 - FIX: unicode/bytes bug in password prompt in zmq.ssh on Python 3
 - workaround overflow bug in libzmq preventing receiving messages
 larger than MAX_INT
>> [...]
>>> The latest version does unfortunately not fix the problem. It is also
>>> not security related, it just could not send large data due to bugs in zmq.
>>> I guess a good approach would be to bisect zmq to see what they changed
>>> to cause it.
>>  Just a friendly ping; could you test the latest Git master if that
>> fixes the hang and/or could you ask upstream for advice? Should I do
>> it? It seems it constantly get fixes that may be relevant.
> 
> The package is team maintained, so I would say go ahead and do a team upload.
> 

I see nothing in git that might be related and I'm not a fan of just
doing a guess upload but if you like go ahead.
Note that you have to disable the new large send test as it will fail on
32 bit and possibly swap on 64 bit.
I still need to get porterbox access before I can look at it, but if
someone can bisect libzmq before that happens for me would be great.

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[Emilio Pozuelo Monfort]

On 29/03/16 19:10, László Böszörményi (GCS) wrote:
> Hi Julian,
> 
> On Sun, Mar 20, 2016 at 12:27 PM, Julian Taylor
>  wrote:
>> On 20.03.2016 11:11, László Böszörményi (GCS) wrote:
>>>  Sorry, my life was chaotic. Yes and no, checked it. First, there's a
>>> new upstream version of pyzmq (15.2) for two months. It seems to be
>>> security related according to the release log[1]:
>>> - FIX: unicode/bytes bug in password prompt in zmq.ssh on Python 3
>>> - workaround overflow bug in libzmq preventing receiving messages
>>> larger than MAX_INT
> [...]
>> The latest version does unfortunately not fix the problem. It is also
>> not security related, it just could not send large data due to bugs in zmq.
>> I guess a good approach would be to bisect zmq to see what they changed
>> to cause it.
>  Just a friendly ping; could you test the latest Git master if that
> fixes the hang and/or could you ask upstream for advice? Should I do
> it? It seems it constantly get fixes that may be relevant.

The package is team maintained, so I would say go ahead and do a team upload.

Cheers,
Emilio

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[GCS]

Hi Julian,

On Sun, Mar 20, 2016 at 12:27 PM, Julian Taylor
 wrote:
> On 20.03.2016 11:11, László Böszörményi (GCS) wrote:
>>  Sorry, my life was chaotic. Yes and no, checked it. First, there's a
>> new upstream version of pyzmq (15.2) for two months. It seems to be
>> security related according to the release log[1]:
>> - FIX: unicode/bytes bug in password prompt in zmq.ssh on Python 3
>> - workaround overflow bug in libzmq preventing receiving messages
>> larger than MAX_INT
[...]
> The latest version does unfortunately not fix the problem. It is also
> not security related, it just could not send large data due to bugs in zmq.
> I guess a good approach would be to bisect zmq to see what they changed
> to cause it.
 Just a friendly ping; could you test the latest Git master if that
fixes the hang and/or could you ask upstream for advice? Should I do
it? It seems it constantly get fixes that may be relevant.

Cheers,
Laszlo/GCS

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[GCS]

On Sun, Mar 20, 2016 at 12:27 PM, Julian Taylor
 wrote:
> On 20.03.2016 11:11, László Böszörményi (GCS) wrote:
>> On Sun, Mar 20, 2016 at 10:56 AM, Emilio Pozuelo Monfort
>>  wrote:
>>> Got a chance to look at this?
>>  Sorry, my life was chaotic. Yes and no, checked it. First, there's a
>> new upstream version of pyzmq (15.2) for two months. It seems to be
>> security related according to the release log[1]:
[...]
> The latest version does unfortunately not fix the problem. It is also
> not security related, it just could not send large data due to bugs in zmq.
> I guess a good approach would be to bisect zmq to see what they changed
> to cause it.
 Still any reason not to update it for Sid? Can you try Git master
branch for building? That contains several fixes.
While the same company develops ZeroMQ and the Python bindings as
well, do you have contact with their Python team? It may worth to ask
them, they may know what's the cause of the hang is.

Regards,
Laszlo/GCS

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[Julian Taylor]

On 20.03.2016 11:11, László Böszörményi (GCS) wrote:
> On Sun, Mar 20, 2016 at 10:56 AM, Emilio Pozuelo Monfort
>  wrote:
>> On Tue, 15 Mar 2016 23:37:26 +0100
>> =?UTF-8?B?TMOhc3psw7MgQsO2c3rDtnJtw6lueWkgKEdDUyk=?=  wrote:
>>> On Tue, Mar 15, 2016 at 11:24 PM, Julian Taylor
>>>  wrote:
 On 15.03.2016 22:48, László Böszörményi (GCS) wrote:
>>>  While I was checking the failing build logs, I see:
>>> build/temp.linux-armv7l-2.7/scratch/tmp/timer_createOfMQXG.o: In
>>> function `main':
>>> timer_createOfMQXG.c:(.text+0x14): undefined reference to `timer_create'
>>> collect2: error: ld returned 1 exit status
>>>
>>> It's not just the implicit declaration, but a linker error later. I
>>> can be wrong, but it seems it _may_ cause the test hang as there's no
>>> timer to look for / to wait its expiration. Sorry if it happens in
>>> normal logs as well; will check it tomorrow morning as it's almost
>>> midnight here. :(
>  Checked, it happens in normal build logs as well.
> 
>> Got a chance to look at this?
>  Sorry, my life was chaotic. Yes and no, checked it. First, there's a
> new upstream version of pyzmq (15.2) for two months. It seems to be
> security related according to the release log[1]:
> - FIX: unicode/bytes bug in password prompt in zmq.ssh on Python 3
> - workaround overflow bug in libzmq preventing receiving messages
> larger than MAX_INT
> 
> My computer is old and has problems; even the archive version hangs
> during self-tests. Still, it may help if Julian update the package to
> the latest version. With my Python Modules team member hat on, should
> I do it myself?
> Just for the record, upstream Git tree has even more fixes that not
> yet released as a new version.
> 

The latest version does unfortunately not fix the problem. It is also
not security related, it just could not send large data due to bugs in zmq.
I guess a good approach would be to bisect zmq to see what they changed
to cause it.

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[GCS]

On Sun, Mar 20, 2016 at 10:56 AM, Emilio Pozuelo Monfort
 wrote:
> On Tue, 15 Mar 2016 23:37:26 +0100
> =?UTF-8?B?TMOhc3psw7MgQsO2c3rDtnJtw6lueWkgKEdDUyk=?=  wrote:
>> On Tue, Mar 15, 2016 at 11:24 PM, Julian Taylor
>>  wrote:
>> > On 15.03.2016 22:48, László Böszörményi (GCS) wrote:
>>  While I was checking the failing build logs, I see:
>> build/temp.linux-armv7l-2.7/scratch/tmp/timer_createOfMQXG.o: In
>> function `main':
>> timer_createOfMQXG.c:(.text+0x14): undefined reference to `timer_create'
>> collect2: error: ld returned 1 exit status
>>
>> It's not just the implicit declaration, but a linker error later. I
>> can be wrong, but it seems it _may_ cause the test hang as there's no
>> timer to look for / to wait its expiration. Sorry if it happens in
>> normal logs as well; will check it tomorrow morning as it's almost
>> midnight here. :(
 Checked, it happens in normal build logs as well.

> Got a chance to look at this?
 Sorry, my life was chaotic. Yes and no, checked it. First, there's a
new upstream version of pyzmq (15.2) for two months. It seems to be
security related according to the release log[1]:
- FIX: unicode/bytes bug in password prompt in zmq.ssh on Python 3
- workaround overflow bug in libzmq preventing receiving messages
larger than MAX_INT

My computer is old and has problems; even the archive version hangs
during self-tests. Still, it may help if Julian update the package to
the latest version. With my Python Modules team member hat on, should
I do it myself?
Just for the record, upstream Git tree has even more fixes that not
yet released as a new version.

Regards,
Laszlo/GCS
[1] https://pyzmq.readthedocs.org/en/latest/changelog.html

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[Emilio Pozuelo Monfort]

On Tue, 15 Mar 2016 23:37:26 +0100 
=?UTF-8?B?TMOhc3psw7MgQsO2c3rDtnJtw6lueWkgKEdDUyk=?=  wrote:

On Tue, Mar 15, 2016 at 11:24 PM, Julian Taylor
 wrote:
> On 15.03.2016 22:48, László Böszörményi (GCS) wrote:
>>  This is known to pyzmq upstream[1] for about three years. It happens
>> on Ubuntu, Red Hat, CentOS and with ZeroMQ 4.0.x as well. While
>> timer_create() [4] is in librt, the standard GNU C library, it seems
>> pyzmq miss to link with it on the failing architectures.
[...]
> Are you saying linking against -lrt will fix the tests deadlocking?
>
> I don't quite see the relation between the hanging tests and the linked
> issues.
> The implicit declaration errors have always been there I think but only
> in a configure check so harmless (until now?).
 While I was checking the failing build logs, I see:
build/temp.linux-armv7l-2.7/scratch/tmp/timer_createOfMQXG.o: In
function `main':
timer_createOfMQXG.c:(.text+0x14): undefined reference to `timer_create'
collect2: error: ld returned 1 exit status

It's not just the implicit declaration, but a linker error later. I
can be wrong, but it seems it _may_ cause the test hang as there's no
timer to look for / to wait its expiration. Sorry if it happens in
normal logs as well; will check it tomorrow morning as it's almost
midnight here. :(


Got a chance to look at this?

Emilio

Bug#818265: Bug#818187: Bug#818265: Bug#818187: zeromq3 migrated without any transition being done

[GCS]

On Tue, Mar 15, 2016 at 11:24 PM, Julian Taylor
 wrote:
> On 15.03.2016 22:48, László Böszörményi (GCS) wrote:
>>  This is known to pyzmq upstream[1] for about three years. It happens
>> on Ubuntu, Red Hat, CentOS and with ZeroMQ 4.0.x as well. While
>> timer_create() [4] is in librt, the standard GNU C library, it seems
>> pyzmq miss to link with it on the failing architectures.
[...]
> Are you saying linking against -lrt will fix the tests deadlocking?
>
> I don't quite see the relation between the hanging tests and the linked
> issues.
> The implicit declaration errors have always been there I think but only
> in a configure check so harmless (until now?).
 While I was checking the failing build logs, I see:
build/temp.linux-armv7l-2.7/scratch/tmp/timer_createOfMQXG.o: In
function `main':
timer_createOfMQXG.c:(.text+0x14): undefined reference to `timer_create'
collect2: error: ld returned 1 exit status

It's not just the implicit declaration, but a linker error later. I
can be wrong, but it seems it _may_ cause the test hang as there's no
timer to look for / to wait its expiration. Sorry if it happens in
normal logs as well; will check it tomorrow morning as it's almost
midnight here. :(

Laszlo/GCS