Re: Streamlining the use of Salsa CI on team packages

2019-09-16 Thread Hans-Christoph Steiner



Louis-Philippe Véronneau:
> On 19-09-05 01 h 40, Louis-Philippe Véronneau wrote:
>> Hello folks!
>>
>> I'd like to propose we start using Salsa CI for all the team packages. I
>> think using a good CI for all our packages will help us find packaging
>> bugs and fix errors before uploads :)
>>
>> I also think that when possible, we should be using the same CI jobs for
>> our packages. The Salsa CI Team's default pipeline [1] is a good common
>> ground, as currently it:
>>
>> * builds the package
>> * runs piuparts
>> * runs autopkgtest
>> * runs lintian
>> * runs reprotest
>> * and does more!
>>
>> I don't think a failing CI should be a blocker for an upload, but I
>> think it's a good red flag and should be taken in account.

Sounds good.

>> I know the Ruby team also decided to use debian/salsa-ci.yml instead of
>> debian/gitlab-ci.yml [2]. I guess we should also do the same.

This is still an open question:
https://salsa.debian.org/salsa-ci-team/pipeline/issues/86

Debian has a bad habit of customizing things that do not need to be
customizing.  That raises the barrier for contributors ever higher, in a
"death by a thousand papercuts" kind of way.  I think we should stick to
the standard file name for GitLab CI.


>> Thoughts? If we decide to go ahead with this, I guess we should modify
>> the policy accordingly and contact the Salsa Team to see if adding this
>> additional load is OK with them.

I think people should add these manually as they see a need.  Then only
if the Salsa Team says they really have the capacity, add it to all
packages.


>> [1] https://salsa.debian.org/salsa-ci-team/pipeline#basic-use
>> [2] https://salsa.debian.org/salsa-ci-team/pipeline/issues/86#note_106245
> These are the steps I see going forward with this:
> 
> --
> 1. Agree on a default pipeline we should be using on the DPMT & PAPT
> packages.
> 
> 2. Draft a proposed modification to the DPMT and the PAPT policies
> 
> 3. Adopt that proposed modification once we reach consensus
> 
> 4. Wait for the "All Clear" from the Salsa Team
> 
> 5. Commit the previously agreed upon CI pipeline to all the DPMT & PAPT
> packages, while making sure the CI isn't ran on that push ("-o ci.skip")
> --
> 
> For step 1, I proposed we use the "Salsa Pipeline" [1], as I feel it is
> the most mature solution, has more contributors and has more features
> (including reprotest and piuparts). This option seems to have had the
> most support so far.

I just checked out salsa-pipelines' piuparts support, it should take me
a couple hours to port it, if its needed.

I think reprotest and piuparts are going to be too heavy for gitlab-ci,
unless their use is manually triggered, or only on releases.  For
example, this salsa-pipeline piuparts-only job timed out after 2 hours
without having completed:
https://salsa.debian.org/debian/dpdk/-/jobs/221036
The reprotest failed after 30 minutes:
https://salsa.debian.org/debian/dpdk/-/jobs/221032

For ruby-fog-aws, these were much shorter:
https://salsa.debian.org/ruby-team/ruby-fog-aws/pipelines/63361
piuparts: ~5 minutes
reprotest: ~10 minutes

The whole libcloud job (build, package, lintian, autopkgtest) takes <10
minutes:
https://salsa.debian.org/python-team/modules/libcloud/-/jobs/248526

The whole androguard took ~13 minutes:
https://salsa.debian.org/python-team/modules/androguard/pipelines

Seems like there needs to be some load testing before pushing heavy
processes like reprotest and piuparts.


> Hans-Christoph Steiner proposed we use "ci-image-git-buidpackage"
> instead [2]. The code behind it is simpler and the way it's built makes
> it possible for maintainers to modify the CI for their packages.
> 
> For step 2, so far people seemed to agree that:
> 
> * having a standardised CI pipeline is a good idea
> * the CI should be used as a tool to help us improve our packages, and
> not be required to pass
> 
> Please contribute to this discussion if you care about this issue! I'll
> try to draft something more concrete in the next few days.
> 
> [1] https://salsa.debian.org/salsa-ci-team/pipeline
> [2] https://salsa.debian.org/salsa-ci-team/ci-image-git-buildpackage

Thanks for keeping this moving!

.hc



Re: What is the process to update the DPMT and PAPT policies?

2019-09-16 Thread Louis-Philippe Véronneau
On 19-09-16 08 h 00, Ondrej Novy wrote:
> Hi,
> 
> po 16. 9. 2019 v 9:55 odesílatel Mattia Rizzolo  napsal:
> 
>> I wonder, I think the historical reasons for papt and dpmt to be separated
>> don't quite stand anymore.  Perhaps the only difference left is the actual
>> technical difference between an application and a module as described in
>> the python policy (rather in the team one).
>>
> 
> I don't know historical reason but I agree we could/should merge DPMT and
> PAPT policies together.

Yes! I too agree this would make a lot of sense.

-- 
  ⢀⣴⠾⠻⢶⣦⠀
  ⣾⠁⢠⠒⠀⣿⡁  Louis-Philippe Véronneau
  ⢿⡄⠘⠷⠚⠋   po...@debian.org / veronneau.org
  ⠈⠳⣄



signature.asc
Description: OpenPGP digital signature


Re: What is the process to update the DPMT and PAPT policies?

2019-09-16 Thread Ondrej Novy
Hi,

po 16. 9. 2019 v 9:55 odesílatel Mattia Rizzolo  napsal:

> I wonder, I think the historical reasons for papt and dpmt to be separated
> don't quite stand anymore.  Perhaps the only difference left is the actual
> technical difference between an application and a module as described in
> the python policy (rather in the team one).
>

I don't know historical reason but I agree we could/should merge DPMT and
PAPT policies together.

-- 
Best regards
 Ondřej Nový


Re: What is the process to update the DPMT and PAPT policies?

2019-09-16 Thread Ondrej Novy
Hi,

po 16. 9. 2019 v 1:59 odesílatel Louis-Philippe Véronneau 
napsal:

> What is the process to update the DPMT and PAPT policies?


we don't have process/policy to update policy.

So I will propose one and let's add this process to policy itself:

   - anyone can submit merge request
   - with MR created, send email to ML
   - give seven? days grace period to discuss proposal
   - to accept MR we needed at least one? ack from DPMT admins

Comments? :)

-- 
Best regards
 Ondřej Nový


Re: [Help] Re: Bug#939181: cycle: Python2 removal in sid/bullseye

2019-09-16 Thread Michael Kesper
Hi Andreas,

On 16.09.19 11:38, Andreas Tille wrote:
> Hi Peter,
> 
> On Sun, Sep 15, 2019 at 02:47:50PM +0100, peter green wrote:
 tmp = rt.encrypt('Cycle{}'.format(pickle.dumps(objSave)))
>>>
>>> Thanks to this hint
>> This hint was *wrong*, it will introduce garbage into the string and the 
>> "rotor" code is clearly designed to work with byte strings, not unicode 
>> strings.
>>
>> Change it to
>>
>> "tmp=rt.encrypt( b'Cycle'+pickle.dumps(objSave) )"

Oops, got that backwards. Thanks for spotting this.
 
> Thanks a lot for your patience.  Unfortunately this is not
> yet the final solution:
> 
> ...
> Traceback (most recent call last):
>   File "/usr/bin/cycle", line 83, in OnCloseWindow
> Save_Cycle(cycle.name, cycle.passwd, cycle.file)
>   File "/usr/share/cycle/save_load.py", line 46, in Save_Cycle
> tmp=rt.encrypt( b'Cycle'+pickle.dumps(objSave) )
>   File "/usr/share/cycle/p_rotor.py", line 63, in encrypt
> return self.cryptmore(buf, 0)
>   File "/usr/share/cycle/p_rotor.py", line 88, in cryptmore
> c = rotors[i][c ^ pos[i]]
> TypeError: unsupported operand type(s) for ^: 'int' and 'float'

That code could be patched by using int() around potentially float
numbers. But honestly, it should better be ripped out and use
real encryption. The docstring tells so:
The rotor module has been removed from the Python 2.4
distribution because

  the rotor module uses an insecure algorithm and is deprecated.
  ==

Best wishes
Michael



signature.asc
Description: OpenPGP digital signature


Re: [Help] Re: Bug#939181: cycle: Python2 removal in sid/bullseye

2019-09-16 Thread Andreas Tille
Hi Peter,

On Sun, Sep 15, 2019 at 02:47:50PM +0100, peter green wrote:
> > > tmp = rt.encrypt('Cycle{}'.format(pickle.dumps(objSave)))
> > 
> > Thanks to this hint
> This hint was *wrong*, it will introduce garbage into the string and the 
> "rotor" code is clearly designed to work with byte strings, not unicode 
> strings.
> 
> Change it to
> 
> "tmp=rt.encrypt( b'Cycle'+pickle.dumps(objSave) )"

Thanks a lot for your patience.  Unfortunately this is not
yet the final solution:

...
Traceback (most recent call last):
  File "/usr/bin/cycle", line 83, in OnCloseWindow
Save_Cycle(cycle.name, cycle.passwd, cycle.file)
  File "/usr/share/cycle/save_load.py", line 46, in Save_Cycle
tmp=rt.encrypt( b'Cycle'+pickle.dumps(objSave) )
  File "/usr/share/cycle/p_rotor.py", line 63, in encrypt
return self.cryptmore(buf, 0)
  File "/usr/share/cycle/p_rotor.py", line 88, in cryptmore
c = rotors[i][c ^ pos[i]]
TypeError: unsupported operand type(s) for ^: 'int' and 'float'


Kind regards

   Andreas. 

-- 
http://fam-tille.de



Re: What is the process to update the DPMT and PAPT policies?

2019-09-16 Thread Mattia Rizzolo
Hi,

On Mon, 16 Sep 2019, 1:59 am Louis-Philippe Véronneau, 
wrote:

> What is the process to update the DPMT and PAPT policies? I feel the
> DPMT policy is pretty good and I feel the PAPT policy could copy a bunch
> of stuff from there.
>

I wonder, I think the historical reasons for papt and dpmt to be separated
don't quite stand anymore.  Perhaps the only difference left is the actual
technical difference between an application and a module as described in
the python policy (rather in the team one).

Could this be a good moment to make the two policies totally converge, and
actually merge them into a single "python team policy"?

Perhaps in a near-ish future the two "sub-"teams could even be merged?

>
>


Re: Streamlining the use of Salsa CI on team packages

2019-09-16 Thread Marcin Kulisz
On 15 September 2019 23:01:46 BST, Thomas Goirand  wrote:

snip

>This tells "instance_type: g1-small", which doesn't match any name at:
>https://cloud.google.com/compute/vm-instance-pricing
>
>Am I right that this is n1-standard-1, which is 1 VCPU and 3.75 GB?

Nop, this is incorrect you're looking for this
https://cloud.google.com/compute/docs/machine-types#sharedcore

snip

>Since we're talking about the smallest type of instance possible at
>google, then other people may have experience the lack of RAM for sure.

g1-small are not the smallest but problem with them is that those are shared 
CPU with 1.7GB of ram.

I agree that this is not suitable for heavy build packages.

I personally would hope that packages build by salsa are throw away and are 
just for testing then source is uploaded and they are rebuild by buildd's in 
such case there would be no need for root or any other heavy handed management 
of those.
But as some people already stated there is not much info from salsa team about 
their plans in this regard.



Re: Packages depending on python-testtools are now RC: is bzr still a thing?

2019-09-16 Thread Andrey Rahmatullin
Note that dak doesn't check autopkgtest deps. I'm using
grep-dctrl -FTestsuite-triggers $pkg -sPackage 
/var/lib/apt/lists/deb.debian.org_debian_dists_unstable_main_source_Sources
for that.

-- 
WBR, wRAR


signature.asc
Description: PGP signature


Re: Packages depending on python-testtools are now RC: is bzr still a thing?

2019-09-16 Thread Thomas Goirand
On 9/16/19 12:31 AM, Simon McVittie wrote:
> On Sun, 15 Sep 2019 at 23:39:46 +0200, Thomas Goirand wrote:
>> reverse-depends takes sometimes forever in Sid for a reason I
>> can't figure out. And if I'm not mistaking, that's the only tool we have
>> that can check reverse dependencies in a meaningful way. Or is there a
>> better way? I've read others using a dak command, how?
> 
> Log in to the developer-accessible archive copy
> (mirror.ftp-master.debian.org, currently hosted on coccia.debian.org),
> and use "dak rm -R -n" (aka "dak rm --rdep-check --no-action"). Use
> "dak rm --help" for help.

Thanks, I'll do that from now on.

Cheers,

Thomas Goirand (zigo)