Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-31 Thread Paul-Andre Panon 
Well, since the default rhn_check interval is 4 hours, you would have to
wait that long. Even after a manual rhn_check, the change isn’t
instantaneous because it takes a few minutes before the queued taskomatic
task executes. But if you wait overnight, all your servers should update.


On Sat, Dec 22, 2018 at 3:47 PM philippe bidault 
wrote:

> Bonjour Paul-André,
>
> I did try your procedure and ...amazing !
>
> Before applying the SP, I had to change the OWNER as it is rhnuser in my
> case :
>
> ALTER FUNCTION rpm.rpmstrcmp(string1 character varying, string2 character
> varying) OWNER TO *rhnuser*;
>
> To apply the SP :
>
> # su - postgres
> Last login: Sat Dec 22 16:20:46 CET 2018 on pts/1
> -bash-4.2$ psql rhnschema -f /var/tmp/rpm.rpmstrcmp.sql
> SET
> CREATE FUNCTION
> ALTER FUNCTION
>
> And I did restart the satellite service.
>
> However even after waiting and finally force the "rhn_check" on a
> registered Ubuntu 18.04 server, nothing occured. I did have to remove and
> register again the Ubuntu client server to see the difference, means no
> more packages flagged as upgradable appaearing on the Spacewalk server.
>
> Will continue to keep an eye on this, will let you know if I notice
> something strange.
>
> Thanks again !
>
> Regards,
> Philippe.
>
> On Sat, 22 Dec 2018 at 08:50, Paul-Andre Panon <
> paul-andre.pa...@avigilon.com> wrote:
>
>> Bonjour Phillipe,
>>
>> Robert can vouch that this issue's actually been bugging me since SW 2.8
>> came out, when I found out that this was still a problem despite PR500's
>> improvements. I just haven't had enough time until this month to look at it
>> and dig through all the layers to figure out the root cause.
>>
>> If you go to the bug, https://bugzilla.redhat.com/show_bug.cgi?id=1661347
>> 
>> , I had added an attachment there. You should be able to download the
>> attachment and
>> 1) stop your Spacewalk service
>> 2) backup your database
>> 3) sudo su postgres or add your spaceuser/password to psql -i
>> rpm.rpmstrcmp.sql
>> 4) exit psql if needed and restart your Spacewalk service
>> and then wait for servers to run the next rhn_check to update their
>> needed updates list.
>>
>> Cheers,
>>
>> Paul-Andre Panon
>> Senior Systems Administrator
>>
>> *Avigilon*
>> A Motorola Solutions Company
>>
>>
>> On Fri, 21 Dec 2018 at 04:00, philippe bidault <
>> philippe.bida...@gmail.com> wrote:
>>
>>> Indeed, really appreciated Paul-Andre, I did not expect somebody to work
>>> so fast to solve this issue !!
>>> Not sure that I will be able to implement your new PostgreSQL SP in our
>>> current Spacewalk platform, I need to understand before the full
>>> implication of such modification and the harmless way to do it.
>>>
>>> Do you think that a patch, update or something will be delivered soon to
>>> allow an easy implementation of your work ?
>>>
>>> Regards,
>>> Philippe.
>>>
>>>
>>>
>>>
>>> On Fri, 21 Dec 2018 at 07:27, Robert Paschedag 
>>> wrote:
>>>
 Am 21. Dezember 2018 00:44:02 MEZ schrieb Paul-Andre Panon <
 paul-andre.pa...@avigilon.com>:
 >I created Bugzilla Bug 1661347 regarding this issue

 Great job, Paul-Andre. Thanks.

 Robert

>>> --
Paul-Andre Panon
Senior Systems Administrator

*Avigilon*
A Motorola Solutions Company
Office +1 604-629-5182 ext. 2341  |  Mobile +1 604-679-1617
Support +1 888-281-5182  |  avigilon.com
Facebook   |  LinkedIn
  |  Twitter
 |  YouTube

This email, including any files attached hereto (the "email"), contains
privileged and confidential information and is only for the intended
addressee(s). If this email has been sent to you in error, such sending
does not constitute waiver of privilege and we request that you kindly
delete the email and notify the sender. Any unauthorized use or disclosure
of this email is prohibited. Avigilon and certain other trade names used
herein are the registered and/or unregistered trademarks of Avigilon
Corporation and/or its affiliates in Canada and other jurisdictions
worldwide.
___
Spacewalk-list mailing list
Spacewalk-list@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-22 Thread philippe bidault 
Bonjour Paul-André,

I did try your procedure and ...amazing !

Before applying the SP, I had to change the OWNER as it is rhnuser in my
case :

ALTER FUNCTION rpm.rpmstrcmp(string1 character varying, string2 character
varying) OWNER TO *rhnuser*;

To apply the SP :

# su - postgres
Last login: Sat Dec 22 16:20:46 CET 2018 on pts/1
-bash-4.2$ psql rhnschema -f /var/tmp/rpm.rpmstrcmp.sql
SET
CREATE FUNCTION
ALTER FUNCTION

And I did restart the satellite service.

However even after waiting and finally force the "rhn_check" on a
registered Ubuntu 18.04 server, nothing occured. I did have to remove and
register again the Ubuntu client server to see the difference, means no
more packages flagged as upgradable appaearing on the Spacewalk server.

Will continue to keep an eye on this, will let you know if I notice
something strange.

Thanks again !

Regards,
Philippe.

On Sat, 22 Dec 2018 at 08:50, Paul-Andre Panon <
paul-andre.pa...@avigilon.com> wrote:

> Bonjour Phillipe,
>
> Robert can vouch that this issue's actually been bugging me since SW 2.8
> came out, when I found out that this was still a problem despite PR500's
> improvements. I just haven't had enough time until this month to look at it
> and dig through all the layers to figure out the root cause.
>
> If you go to the bug, https://bugzilla.redhat.com/show_bug.cgi?id=1661347
> , I had added an attachment there. You should be able to download the
> attachment and
> 1) stop your Spacewalk service
> 2) backup your database
> 3) sudo su postgres or add your spaceuser/password to psql -i
> rpm.rpmstrcmp.sql
> 4) exit psql if needed and restart your Spacewalk service
> and then wait for servers to run the next rhn_check to update their needed
> updates list.
>
> Cheers,
>
> Paul-Andre Panon
> Senior Systems Administrator
>
> *Avigilon*
> A Motorola Solutions Company
>
>
> On Fri, 21 Dec 2018 at 04:00, philippe bidault 
> wrote:
>
>> Indeed, really appreciated Paul-Andre, I did not expect somebody to work
>> so fast to solve this issue !!
>> Not sure that I will be able to implement your new PostgreSQL SP in our
>> current Spacewalk platform, I need to understand before the full
>> implication of such modification and the harmless way to do it.
>>
>> Do you think that a patch, update or something will be delivered soon to
>> allow an easy implementation of your work ?
>>
>> Regards,
>> Philippe.
>>
>>
>>
>>
>> On Fri, 21 Dec 2018 at 07:27, Robert Paschedag 
>> wrote:
>>
>>> Am 21. Dezember 2018 00:44:02 MEZ schrieb Paul-Andre Panon <
>>> paul-andre.pa...@avigilon.com>:
>>> >I created Bugzilla Bug 1661347 regarding this issue
>>>
>>> Great job, Paul-Andre. Thanks.
>>>
>>> Robert
>>>
>>
___
Spacewalk-list mailing list
Spacewalk-list@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-21 Thread Paul-Andre Panon 
Bonjour Phillipe,

Robert can vouch that this issue's actually been bugging me since SW 2.8
came out, when I found out that this was still a problem despite PR500's
improvements. I just haven't had enough time until this month to look at it
and dig through all the layers to figure out the root cause.

If you go to the bug, https://bugzilla.redhat.com/show_bug.cgi?id=1661347 ,
I had added an attachment there. You should be able to download the
attachment and
1) stop your Spacewalk service
2) backup your database
3) sudo su postgres or add your spaceuser/password to psql -i
rpm.rpmstrcmp.sql
4) exit psql if needed and restart your Spacewalk service
and then wait for servers to run the next rhn_check to update their needed
updates list.

Cheers,

Paul-Andre Panon
Senior Systems Administrator

*Avigilon*
A Motorola Solutions Company


On Fri, 21 Dec 2018 at 04:00, philippe bidault 
wrote:

> Indeed, really appreciated Paul-Andre, I did not expect somebody to work
> so fast to solve this issue !!
> Not sure that I will be able to implement your new PostgreSQL SP in our
> current Spacewalk platform, I need to understand before the full
> implication of such modification and the harmless way to do it.
>
> Do you think that a patch, update or something will be delivered soon to
> allow an easy implementation of your work ?
>
> Regards,
> Philippe.
>
>
>
>
> On Fri, 21 Dec 2018 at 07:27, Robert Paschedag 
> wrote:
>
>> Am 21. Dezember 2018 00:44:02 MEZ schrieb Paul-Andre Panon <
>> paul-andre.pa...@avigilon.com>:
>> >I created Bugzilla Bug 1661347 regarding this issue
>>
>> Great job, Paul-Andre. Thanks.
>>
>> Robert
>>
>
___
Spacewalk-list mailing list
Spacewalk-list@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-21 Thread philippe bidault 
Indeed, really appreciated Paul-Andre, I did not expect somebody to work so
fast to solve this issue !!
Not sure that I will be able to implement your new PostgreSQL SP in our
current Spacewalk platform, I need to understand before the full
implication of such modification and the harmless way to do it.

Do you think that a patch, update or something will be delivered soon to
allow an easy implementation of your work ?

Regards,
Philippe.




On Fri, 21 Dec 2018 at 07:27, Robert Paschedag 
wrote:

> Am 21. Dezember 2018 00:44:02 MEZ schrieb Paul-Andre Panon <
> paul-andre.pa...@avigilon.com>:
> >I created Bugzilla Bug 1661347 regarding this issue
>
> Great job, Paul-Andre. Thanks.
>
> Robert
> >
> >On Thursday, December 20, 2018 1:24 PM, I wrote:
> >
> >>The updated SP corrected the erroneous false positives, and also
> >caught
> >some >packages which needed to be updated but weren't listed. Can
> >somebody
> >put >this in a PR?
> >
> >>Thanks,
> >
> >>Paul-Andre
> >
> >___
> >Spacewalk-list mailing list
> >Spacewalk-list@redhat.com
> >https://www.redhat.com/mailman/listinfo/spacewalk-list
>
>
> --
> sent from my mobile device
>
> ___
> Spacewalk-list mailing list
> Spacewalk-list@redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
>
___
Spacewalk-list mailing list
Spacewalk-list@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-20 Thread Robert Paschedag 
Am 21. Dezember 2018 00:44:02 MEZ schrieb Paul-Andre Panon 
:
>I created Bugzilla Bug 1661347 regarding this issue

Great job, Paul-Andre. Thanks.

Robert
>
>On Thursday, December 20, 2018 1:24 PM, I wrote:
>
>>The updated SP corrected the erroneous false positives, and also
>caught
>some >packages which needed to be updated but weren't listed. Can
>somebody
>put >this in a PR?
>
>>Thanks,
>
>>Paul-Andre
>
>___
>Spacewalk-list mailing list
>Spacewalk-list@redhat.com
>https://www.redhat.com/mailman/listinfo/spacewalk-list


-- 
sent from my mobile device

___
Spacewalk-list mailing list
Spacewalk-list@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-20 Thread Paul-Andre Panon 
I created Bugzilla Bug 1661347 regarding this issue

On Thursday, December 20, 2018 1:24 PM, I wrote:

>The updated SP corrected the erroneous false positives, and also caught
some >packages which needed to be updated but weren't listed. Can somebody
put >this in a PR?

>Thanks,

>Paul-Andre

___
Spacewalk-list mailing list
Spacewalk-list@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-20 Thread Paul-Andre Panon 
On Tuesday, December 18, 2018 2:59 PM, I wrote that I thought I had a
solution to the phantom packages with Ubuntu that I hadn't tested yet.

OK, there were a few typos and syntax errors, but this corrected stored
proc appears to work well.

SET search_path = rpm, pg_catalog;
create or replace FUNCTION rpmstrcmp (string1 IN VARCHAR, string2 IN
VARCHAR)
RETURNS INTEGER as $$
declare
str1 VARCHAR := string1;
str2 VARCHAR := string2;
digits VARCHAR(10) := '0123456789';
lc_alpha VARCHAR(27) := 'abcdefghijklmnopqrstuvwxyz';
uc_alpha VARCHAR(27) := 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
alpha VARCHAR(54) := lc_alpha || uc_alpha;
one VARCHAR;
two VARCHAR;
sep1 VARCHAR;
sep2 VARCHAR;
isnum BOOLEAN;
BEGIN
if str1 is NULL or str2 is NULL
then
RAISE EXCEPTION 'VALUE_ERROR.';
end if;

if str1 = str2
then
return 0;
end if;
one := str1;
two := str2;

<>
while one <> '' and two <> ''
loop
declare
segm1 VARCHAR;
segm2 VARCHAR;
begin
sep1 := '';
sep2 := '';
--DBMS_OUTPUT.PUT_LINE('Params: ' || one || ',' || two);
-- Pull out all separating non-alphanum characters
while one <> '' and not rpm.isalphanum(one)
loop
sep1 := sep1 || substr(one, 1, 1);
one := substr(one, 2);
end loop;
while two <> '' and not rpm.isalphanum(two)
loop
sep2 := sep2 || substr(two, 1, 1);
two := substr(two, 2);
end loop;

str1 := one;
str2 := two;
if str1 <> '' and rpm.isdigit(str1)
then
str1 := ltrim(str1, digits);
str2 := ltrim(str2, digits);
isnum := true;
else
str1 := ltrim(str1, alpha);
str2 := ltrim(str2, alpha);
isnum := false;
end if;
if str1 <> ''
then segm1 := substr(one, 1, length(one) - length(str1));
else segm1 := one;
end if;

if str2 <> ''
then segm2 := substr(two, 1, length(two) - length(str2));
else segm2 := two;
end if;

-- if one of the separators is for a point subversion
indicator and the other isn't, then the point subversion is considered
more recent.
if isnum and sep1 <> '' and sep2 <> ''
then
if sep1 = '.' and sep2 <> '.'
then
return 1;
elsif sep2 = '.' and sep1 <> '.'
then
return -1;
end if;
end if;

if segm1 = '' then return -1; end if; /* arbitrary */
if segm2 = '' then
if isnum then
return 1;
else
return -1;
end if;
end if;
if isnum
then
segm1 := ltrim(segm1, '0');
segm2 := ltrim(segm2, '0');

if segm1 = '' and segm2 <> ''
then
return -1;
end if;
if segm1 <> '' and segm2 = ''
then
return 1;
end if;
if length(segm1) > length(segm2) then return 1; end
if;
if length(segm2) > length(segm1) then return -1; end
if;
end if;
if segm1 < segm2 then return -1; end if;
if segm1 > segm2 then return 1; end if;
one := str1;
two := str2;
end;
end loop segment_loop;

if one = '' and two = '' then return 0; end if;
if one = '' then return -1; end if;
return 1;
END ;
$$ language 'plpgsql';

ALTER FUNCTION rpm.rpmstrcmp(string1 character varying, string2 character
varying) OWNER TO spaceuser;


I incorporated that into a duplicate of our postgres database. Then I ran
the following query on both the unpatched and modified DBs
select sp.server_id, count(*)
   FROM (SELECT sp_sp.server_id, sp_sp.name_id,
sp_sp.package_arch_id, max(sp_pe.evr) AS max_evr
   FROM rhnServerPackage sp_sp
   join rhnPackageEvr sp_pe ON sp_pe.id = sp_sp.evr_id
  GROUP BY sp_sp.server_id, sp_sp.name_id,
sp_sp.package_arch_id) sp
   join rhnPackage p ON p.name_id =

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-18 Thread Paul-Andre Panon 
On Tuesday, December 11, 2018 4:34 PM, I wrote:
that I had tracked down the false positive "newer" Ubuntu packages to the
rhnServerNeededCache and that it was getting repopulated by the rhn_check.

>After clearing the unwanted entries in the rhnServerNeededCache table, I
verified that the incorrect packages had been >cleared from the package
list for the servers. I then ran rhn_check on one of the clients that had
been cleared and after >some delay (due to queuing a taskomatic task to
update the server's package list?) the incorrect packages came back.
>Hopefully that can help someone else figure out the last few steps of
which task is incorrectly updating >rhnServerNeededCache, why it's doing
it, and fix the bug.

I tracked that down through the taskomatic task triggered by the rhn_check
to the ErrataCacheManager.java reference I had found earlier. Robert P.
helped me find the source for the rhn_server.update_needed_cache stored
proc that gets called in  ErrataCacheManager.java.

Anyways, digging through a few more layers into the definition of the
evr_t type and its comparison overload functions, you eventually get to
Function rpmstrcmp in
spacewalk/schema/spacewalk/postgres/packages/rpm.pkb, which breaks apart
the alphanumeric components of the package version and revision. That
function treats the non-alphanum separators as non-significant, so when
comparing versions in (,8.2.0,1ubuntu2~18.04)  and (,8-20180414,1ubuntu2)
8 = 8, but 20180414 > 2, so the latter evr is considered bigger.

So the latest question is, if we change that function to actually care
about separators, so that '.' (from 8.2.0) > '-' (from 8-20180414), will
that break the comparison for rpm versioning rules? The new function would
be something like (I haven't tested it)

create or replace FUNCTION rpmstrcmp (string1 IN VARCHAR, string2 IN
VARCHAR)
RETURNS INTEGER as $$
declare
str1 VARCHAR := string1;
str2 VARCHAR := string2;
digits VARCHAR(10) := '0123456789';
lc_alpha VARCHAR(27) := 'abcdefghijklmnopqrstuvwxyz';
uc_alpha VARCHAR(27) := 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
alpha VARCHAR(54) := lc_alpha || uc_alpha;
one VARCHAR;
two VARCHAR;
sep1 VARCHAR;
sep2 VARCHAR;
isnum BOOLEAN;
BEGIN
if str1 is NULL or str2 is NULL
then
RAISE EXCEPTION 'VALUE_ERROR.';
end if;

if str1 = str2
then
return 0;
end if;
one := str1;
two := str2;

<>
while one <> '' and two <> ''
loop
declare
segm1 VARCHAR;
segm2 VARCHAR;
begin
sep1 := ''
sep2 := ''
--DBMS_OUTPUT.PUT_LINE('Params: ' || one || ',' || two);
-- Pull out all separating non-alphanum characters
while one <> '' and not rpm.isalphanum(one)
loop
sep1 := sep1 || substr(one, 1, 1);
one := substr(one, 2);
end loop;
while two <> '' and not rpm.isalphanum(two)
loop
sep2 := sep2 || substr(two, 1, 1);
two := substr(two, 2);
end loop;

str1 := one;
str2 := two;
if str1 <> '' and rpm.isdigit(str1)
then
str1 := ltrim(str1, digits);
str2 := ltrim(str2, digits);
isnum := true;
else
str1 := ltrim(str1, alpha);
str2 := ltrim(str2, alpha);
isnum := false;
end if;
if str1 <> ''
then segm1 := substr(one, 1, length(one) - length(str1));
else segm1 := one;
end if;

if str2 <> ''
then segm2 := substr(two, 1, length(two) - length(str2));
else segm2 := two;
end if;

-- if one of the separators is for a point subversion
indicator and the other isn't, then the point subversion is considered
more recent.
if isnum and sep1 <> '' and sep2 <> ''
then
if sep1 = '.' and sep2 <> '.'
then
return 1;
  end if;
if sep2 = '.' and sep1 <> '.'
then
return -1;
  end if;
endif

if segm1 = '' then return -1; end if; /* arbitrary */
if segm2 = '' then
if isnum then
return 1;
else
return -1;
end if;
end if;

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-11 Thread Paul-Andre Panon 
After clearing the unwanted entries in the rhnServerNeededCache table, I
verified that the incorrect packages had been cleared from the package
list for the servers. I then ran rhn_check on one of the clients that had
been cleared and after some delay (due to queuing a taskomatic task to
update the server's package list?) the incorrect packages came back.
Hopefully that can help someone else figure out the last few steps of
which task is incorrectly updating rhnServerNeededCache, why it's doing
it, and fix the bug.

-Original Message-
From: Paul-Andre Panon 
Sent: Monday, December 10, 2018 6:06 PM
To: 'spacewalk-list@redhat.com' ;
'spacewalk-list@redhat.com' 
Subject: RE: Ubuntu 18.04 package management in Spacewalk 2.8

It looks like there aren't too many things that insert into the
rhnServerNeededCache.
./java/code/src/com/redhat/rhn/manager/errata/cache/ErrataCacheManager.jav
a calls some queries directly to insert into the table, and the
update_needed_cache stored procedure could also be a cause.

I've cleared out the table rows for some server/package combos in our
environment that I know are invalid, and I'll see if the entries get
repopulated for those combos during the next sync or at other times. That
will hopefully help narrow down the possibilities. Most of those table
inserts appear to have to do with the errata cache management so maybe
there's something in there that was missed during the PR500 fixes.

-Original Message-
From: Paul-Andre Panon 
Sent: Monday, December 10, 2018 4:51 PM
To: 'spacewalk-list@redhat.com' ;
'spacewalk-list@redhat.com' 
Subject: RE: Ubuntu 18.04 package management in Spacewalk 2.8

Earlier today I wrote about the changes in PR500. That parsing actually
seems to be OK after all because the EVR entries in the database are OK.

That said, it looks like the spurious package entries are due to spurious
entries in the rhnServerNeededCache table. So the question is what is
populating the rhnServerNeededCache table incorrectly, and why?

___
Spacewalk-list mailing list
Spacewalk-list@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-10 Thread Robert Paschedag 
Am 11. Dezember 2018 01:51:11 MEZ schrieb Paul-Andre Panon 
:
>Earlier today I wrote about the changes in PR500. That parsing actually
>seems to be OK after all because the EVR entries in the database are
>OK.
>
>That said, it looks like the query that generates the list of
>upgradeable
>packages in the Spacewalk GUI is system_upgradable_package_list in
>spacewalk/java/code/src/com/redhat/rhn/common/db/datasource/xml/Package_qu
>eries.xml
>
>class="com.redhat.rhn.frontend.dto.UpgradablePackageListItem">
>  
>SELECT  n.id AS id,
>n.id AS name_id,
>lookup_evr(((latest.evr)).epoch, (latest.evr).version,
>(latest.evr).release) AS evr_id,
>latest.package_arch_id AS arch_id,
>(latest.evr).epoch AS epoch,
>(latest.evr).version AS version,
>(latest.evr).release AS release,
>n.name AS name,
>n.name ||'-'|| evr_t_as_vre_simple(latest.evr) || '.' ||
>latest_pa.label AS nvrea,
> n.name ||'-'|| evr_t_as_vre_simple(spe.evr) || '.' || spa.label AS
>installed_package,
>n.id || '|' || lookup_evr((latest.evr).epoch,
>(latest.evr).version, (latest.evr).release)|| '|' ||
>latest.package_arch_id AS id_combo
>  FROM
>   rhnServerPackage sp
>  join rhnPackageName n
>on n.id = sp.name_id
>  join rhnPackageArch spa
>on spa.id = sp.package_arch_id
>  join rhnPackageEvr spe
>on spe.id = sp.evr_id
>  join (
>select sop.package_name_id,
>   sop.package_arch_id,
>   max(PE.evr) evr
>  from rhnServerOutdatedPackages sop
>  join rhnPackageEVR pe
>on sop.package_evr_id = pe.id
> where sop.server_id = :sid
> group by sop.package_name_id, sop.package_arch_id) latest
>on latest.package_name_id = sp.name_id
>  join rhnPackageArch latest_pa
>on latest_pa.id = latest.package_arch_id
>  join rhnPackageUpgradeArchCompat puac
>on puac.package_arch_id = sp.package_arch_id
>   and puac.package_upgrade_arch_id = latest.package_arch_id
> where sp.server_id = :sid
> order by upper(n.name)
>  
>  
>  SELECT  PN.id AS id,
>  SOP.errata_id AS errata_id,
>  SOP.errata_advisory AS errata_advisory,
>  E.advisory_type AS errata_advisory_type,
>  E.severity_id AS errata_severity_id
>FROM  rhnServerOutdatedPackages SOP
>  INNER JOIN rhnPackageName PN on SOP.package_name_id = PN.id
>  INNER JOIN rhnErrata E on SOP.errata_id = E.id
>   WHERE  PN.id IN (%s)
> AND  SOP.server_id = :sid
>  
>
>
>The sub-clause
>
> join (
>  select sop.package_name_id,
>   sop.package_arch_id,
>   max(PE.evr) evr
>  from rhnServerOutdatedPackages sop
>  join rhnPackageEVR pe
>on sop.package_evr_id = pe.id
> where sop.server_id = :sid
> group by sop.package_name_id, sop.package_arch_id) latest
>
>is returning the gcc_8-related packages because they are provided by
>the
>rhnServerOutdatedPackages view
>
>SELECT DISTINCT SNC.server_id,
>   P.name_id,
>   P.evr_id,
>   P.package_arch_id,
>   PN.name || '-' || evr_t_as_vre_simple( PE.evr ),
>   E.id,
>   E.advisory
>  FROM rhnPackageName PN,
>   rhnPackageEVR PE,
>   rhnPackage P,
>   rhnServerNeededCache SNC
> left outer join
>rhnErrata E
>  on SNC.errata_id = E.id
> WHERE SNC.package_id = P.id
>   AND P.name_id = PN.id
>   AND P.evr_id = PE.id
>
>and the view is listing those packages because of entries for those
>packages in the rhnServerNeededCache table. So the question is what is
>populating the rhnServerNeededCache table incorrectly, and why?

Great analysis, Paul-Andre.

Robert
>
>___
>Spacewalk-list mailing list
>Spacewalk-list@redhat.com
>https://www.redhat.com/mailman/listinfo/spacewalk-list


-- 
sent from my mobile device

___
Spacewalk-list mailing list
Spacewalk-list@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-10 Thread Paul-Andre Panon 
It looks like there aren't too many things that insert into the
rhnServerNeededCache.
./java/code/src/com/redhat/rhn/manager/errata/cache/ErrataCacheManager.jav
a calls some queries directly to insert into the table, and the
update_needed_cache stored procedure could also be a cause.

I've cleared out the table rows for some server/package combos in our
environment that I know are invalid, and I'll see if the entries get
repopulated for those combos during the next sync or at other times. That
will hopefully help narrow down the possibilities. Most of those table
inserts appear to have to do with the errata cache management so maybe
there's something in there that was missed during the PR500 fixes.

-Original Message-
From: Paul-Andre Panon 
Sent: Monday, December 10, 2018 4:51 PM
To: 'spacewalk-list@redhat.com' ;
'spacewalk-list@redhat.com' 
Subject: RE: Ubuntu 18.04 package management in Spacewalk 2.8

Earlier today I wrote about the changes in PR500. That parsing actually
seems to be OK after all because the EVR entries in the database are OK.

That said, it looks like the spurious package entries are due to spurious
entries in the rhnServerNeededCache table. So the question is what is
populating the rhnServerNeededCache table incorrectly, and why?

___
Spacewalk-list mailing list
Spacewalk-list@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-10 Thread Paul-Andre Panon 
Earlier today I wrote about the changes in PR500. That parsing actually
seems to be OK after all because the EVR entries in the database are OK.

That said, it looks like the query that generates the list of upgradeable
packages in the Spacewalk GUI is system_upgradable_package_list in
spacewalk/java/code/src/com/redhat/rhn/common/db/datasource/xml/Package_qu
eries.xml


  
SELECT  n.id AS id,
n.id AS name_id,
lookup_evr(((latest.evr)).epoch, (latest.evr).version,
(latest.evr).release) AS evr_id,
latest.package_arch_id AS arch_id,
(latest.evr).epoch AS epoch,
(latest.evr).version AS version,
(latest.evr).release AS release,
n.name AS name,
n.name ||'-'|| evr_t_as_vre_simple(latest.evr) || '.' ||
latest_pa.label AS nvrea,
n.name ||'-'|| evr_t_as_vre_simple(spe.evr) || '.' || spa.label AS
installed_package,
n.id || '|' || lookup_evr((latest.evr).epoch,
(latest.evr).version, (latest.evr).release)|| '|' ||
latest.package_arch_id AS id_combo
  FROM
   rhnServerPackage sp
  join rhnPackageName n
on n.id = sp.name_id
  join rhnPackageArch spa
on spa.id = sp.package_arch_id
  join rhnPackageEvr spe
on spe.id = sp.evr_id
  join (
select sop.package_name_id,
   sop.package_arch_id,
   max(PE.evr) evr
  from rhnServerOutdatedPackages sop
  join rhnPackageEVR pe
on sop.package_evr_id = pe.id
 where sop.server_id = :sid
 group by sop.package_name_id, sop.package_arch_id) latest
on latest.package_name_id = sp.name_id
  join rhnPackageArch latest_pa
on latest_pa.id = latest.package_arch_id
  join rhnPackageUpgradeArchCompat puac
on puac.package_arch_id = sp.package_arch_id
   and puac.package_upgrade_arch_id = latest.package_arch_id
 where sp.server_id = :sid
 order by upper(n.name)
  
  
  SELECT  PN.id AS id,
  SOP.errata_id AS errata_id,
  SOP.errata_advisory AS errata_advisory,
  E.advisory_type AS errata_advisory_type,
  E.severity_id AS errata_severity_id
FROM  rhnServerOutdatedPackages SOP
  INNER JOIN rhnPackageName PN on SOP.package_name_id = PN.id
  INNER JOIN rhnErrata E on SOP.errata_id = E.id
   WHERE  PN.id IN (%s)
 AND  SOP.server_id = :sid
  


The sub-clause

 join (
  select sop.package_name_id,
   sop.package_arch_id,
   max(PE.evr) evr
  from rhnServerOutdatedPackages sop
  join rhnPackageEVR pe
on sop.package_evr_id = pe.id
 where sop.server_id = :sid
 group by sop.package_name_id, sop.package_arch_id) latest

is returning the gcc_8-related packages because they are provided by the
rhnServerOutdatedPackages view

SELECT DISTINCT SNC.server_id,
   P.name_id,
   P.evr_id,
   P.package_arch_id,
   PN.name || '-' || evr_t_as_vre_simple( PE.evr ),
   E.id,
   E.advisory
  FROM rhnPackageName PN,
   rhnPackageEVR PE,
   rhnPackage P,
   rhnServerNeededCache SNC
 left outer join
rhnErrata E
  on SNC.errata_id = E.id
 WHERE SNC.package_id = P.id
   AND P.name_id = PN.id
   AND P.evr_id = PE.id

and the view is listing those packages because of entries for those
packages in the rhnServerNeededCache table. So the question is what is
populating the rhnServerNeededCache table incorrectly, and why?

___
Spacewalk-list mailing list
Spacewalk-list@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list

[Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-10 Thread Paul-Andre Panon 
On Thu, 6 Dec 2018 13:25:54 +0100, philippe bidault
 wrote:

>Hi Robert,
>
>Thanks for the script, really appreciated. Already implemented in my
>Spacewalk and for the moment, working like a charm.
>In fact it even solved an issue I had with the "netplan.io" package
always
>flagged as upgradable (I guess because of a missing header).
>
>Regarding the fake-upgradable Debian packages appearing on Spacewalk, I
>have 12 of them for the moment :
>
>? gcc-8-base-8-20180414-1ubuntu2.amd64-deb
>gcc-8-base-8.2.0-1ubuntu2~18.04.amd64-deb
>
>lib32gcc1-8-20180414-1ubuntu2:1.amd64-deb
>lib32gcc1-8.2.0-1ubuntu2~18.04:1.amd64-deb
>
>libatomic1-8-20180414-1ubuntu2.amd64-deb
>libatomic1-8.2.0-1ubuntu2~18.04.amd64-deb
>
>libcc1-0-8-20180414-1ubuntu2.amd64-deb
libcc1-0-8.2.0-1ubuntu2~18.04.amd64-deb
>
>libgcc1-8-20180414-1ubuntu2:1.amd64-deb
>libgcc1-8.2.0-1ubuntu2~18.04:1.amd64-deb
>
>libgomp1-8-20180414-1ubuntu2.amd64-deb
libgomp1-8.2.0-1ubuntu2~18.04.amd64-deb
>
>libitm1-8-20180414-1ubuntu2.amd64-deb
libitm1-8.2.0-1ubuntu2~18.04.amd64-deb
>
>liblsan0-8-20180414-1ubuntu2.amd64-deb
liblsan0-8.2.0-1ubuntu2~18.04.amd64-deb
>
>libmpx2-8-20180414-1ubuntu2.amd64-deb
libmpx2-8.2.0-1ubuntu2~18.04.amd64-deb
>
>libquadmath0-8-20180414-1ubuntu2.amd64-deb
>libquadmath0-8.2.0-1ubuntu2~18.04.amd64-deb
>
>libstdc++6-8-20180414-1ubuntu2.amd64-deb
>libstdc++6-8.2.0-1ubuntu2~18.04.amd64-deb
>
>libtsan0-8-20180414-1ubuntu2.amd64-deb
>libtsan0-8.2.0-1ubuntu2~18.04.amd64-deb
>
>Will try to dig, but indeed if the versioning motor is managed by Java
and
>not Python, will be hard to debug ...
>
>Regards,
>Philippe.

Hi Phillipe,

Back in January of last year, Marc Dalhaus created PR500, which fixed a
number of version parsing issues with Debian (but left the ones you
outlined above). The Git commits in
https://github.com/spacewalkproject/spacewalk/pull/500/commits appear to
show changes applied to backend/common/rhnLib.py,
backend/common/rhn_deb.py,  backend/server/rhnLib.py,
backend/satellite_tools/repo_plugins/deb_src.py, and
java/code/src/com/redhat/rhn/taskomatic/task/repomd/DebPackageWriter.java.
A quick glance appears to indicate the parsing is happening in the former.
If I remember correctly, after applying those changes we had to clear and
repopulate the repos so that all the DB package indexes would be
regenerated. Something to keep in mind to ensure that changes don't break
other stuff by fixing the cases you identified.

The two main parsing sections appear to be split between parseRPMFilename
in server/rhnLib.py

# 'n_n-n-v.v.v-r_r.r:e.ARCH.rpm' ---> [n,v,r,e,a]
def parseRPMFilename(pkgFilename):
"""
IN: Package Name: xxx-yyy-ver.ver.ver-rel.rel_rel:e.ARCH.rpm (string)
Understood rules:
   o Name can have nearly any char, but end in a - (well seperated
by).
 Any character; may include - as well.
   o Version cannot have a -, but ends in one.
   o Release should be an actual number, and can't have any -'s.
   o Release can include the Epoch, e.g.: 2:4 (4 is the epoch)
   o Epoch: Can include anything except a - and the : seperator???
 XXX: Is epoch info above correct?
OUT: [n,e,v,r, arch].
"""
if type(pkgFilename) != type(''):
raise rhnFault(21, str(pkgFilename))  # Invalid arg.

pkgFilename = os.path.basename(pkgFilename)

# Check that this is a package NAME (with arch.rpm) and strip
# that crap off.
pkg = string.split(pkgFilename, '.')

dist = string.lower(pkg[-1])

# 'rpm' at end?
if dist not in ['rpm', 'deb']:
raise rhnFault(21, 'neither an rpm nor a deb package name: %s' %
pkgFilename)

# Valid architecture next?
if check_package_arch(pkg[-2]) is None:
raise rhnFault(21, 'Incompatible architecture found: %s' %
pkg[-2])

_arch = pkg[-2]

# Nuke that arch.rpm.
pkg = string.join(pkg[:-2], '.')

if dist == "deb":
ret = list(parseDEBName(pkg))
else:
ret = list(parseRPMName(pkg))

if ret:
ret.append(_arch)
return ret


and parseDEPName in common/rhnLib.py


def parseDEPName(pkgName):
""" IN:  Package string in, n-n-n-v.v.v-r.r_r, format.
OUT: Four strings (in a tuple): name, epoch, version, release.
"""
if pkgName.find('_') == -1:
return None, None, None, None
e = ''
n, version = pkgName.split('_')
if version.find(':') != -1:
e, version = version.split(':')
version_tmpArr = version.split('-')
v = '-'.join(version_tmpArr[:-1])
r = version_tmpArr[-1]
return str(n), e, str(v), str(r)

That actually seems to be based on the Debian control fields definitions
here, https://www.debian.org/doc/debian-policy/ch-controlfields.html and a
big part of the issue appears to be that Ubuntu is breaking those rules
for the early package pre-releases.

If the filename is being parsed as
xxx-yyy-ver.ver.ver-rel.rel_rel:e.ARCH.rpm, how well does
libgcc1-8-20180414-1ubuntu2:1.amd64-deb fit? It doesn't, because
8-20180414

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-06 Thread philippe bidault 
Hi Robert,

Thanks for the script, really appreciated. Already implemented in my
Spacewalk and for the moment, working like a charm.
In fact it even solved an issue I had with the "netplan.io" package always
flagged as upgradable (I guess because of a missing header).

Regarding the fake-upgradable Debian packages appearing on Spacewalk, I
have 12 of them for the moment :

 gcc-8-base-8-20180414-1ubuntu2.amd64-deb
gcc-8-base-8.2.0-1ubuntu2~18.04.amd64-deb

lib32gcc1-8-20180414-1ubuntu2:1.amd64-deb
lib32gcc1-8.2.0-1ubuntu2~18.04:1.amd64-deb

libatomic1-8-20180414-1ubuntu2.amd64-deb
libatomic1-8.2.0-1ubuntu2~18.04.amd64-deb

libcc1-0-8-20180414-1ubuntu2.amd64-deb libcc1-0-8.2.0-1ubuntu2~18.04.amd64-deb

libgcc1-8-20180414-1ubuntu2:1.amd64-deb
libgcc1-8.2.0-1ubuntu2~18.04:1.amd64-deb

libgomp1-8-20180414-1ubuntu2.amd64-deb libgomp1-8.2.0-1ubuntu2~18.04.amd64-deb

libitm1-8-20180414-1ubuntu2.amd64-deb libitm1-8.2.0-1ubuntu2~18.04.amd64-deb

liblsan0-8-20180414-1ubuntu2.amd64-deb liblsan0-8.2.0-1ubuntu2~18.04.amd64-deb

libmpx2-8-20180414-1ubuntu2.amd64-deb libmpx2-8.2.0-1ubuntu2~18.04.amd64-deb

libquadmath0-8-20180414-1ubuntu2.amd64-deb
libquadmath0-8.2.0-1ubuntu2~18.04.amd64-deb

libstdc++6-8-20180414-1ubuntu2.amd64-deb
libstdc++6-8.2.0-1ubuntu2~18.04.amd64-deb

libtsan0-8-20180414-1ubuntu2.amd64-deb
libtsan0-8.2.0-1ubuntu2~18.04.amd64-deb

Will try to dig, but indeed if the versioning motor is managed by Java and
not Python, will be hard to debug ...

Regards,
Philippe.

On Wed, 5 Dec 2018 at 22:08, Robert Paschedag 
wrote:

> Hi Philippe,
>
> Welluntil now, I only have very few packages that are shown as
> upgradable for "Debian" systems. I only found some version comparision
> problem while testing Uyuni (and testing to get Debian system registered as
> Salt client). I think this could be further tuned to use the python-apt or
> python-dpkg package...but I did not have time to dig into that.
>
> Within spacewalk, I currently do not know, where the comparision takes
> place (within java or within python).
>
> The "modified" script is a "new" one. It does not depend on the - from
> myself - modified sync script. What it needs is the original Packages.gz
> file for the channel you sync.
>
> Just throwing this here so you can test yourself. Basically, it should add
> all "missing" headers for every package.
>
> import sys
> import os
> from debian.deb822 import *
> if len(sys.argv) < 3:
> print "Usage: %s 
> " % sys.argv[0]
> sys.exit(1)
> spacewalk_file = sys.argv[1]
> original_file = sys.argv[2]
> if not os.path.isfile(spacewalk_file):
> print "Error: Inputfile '%s' not available." % spacewalk_file
> sys.exit(1)
> if not os.path.isfile(original_file):
> print "Error: Inputfile '%s' not available." % original_file
> sys.exit(1)
> spacewalk_packages = {}
> original_packages = {}
> with open(spacewalk_file, 'r') as pkgs:
> for pkg in Packages.iter_paragraphs(pkgs):
> spacewalk_packages[pkg['Package'] + pkg['Version'] +
> pkg['Architecture']] = pkg
> with open(original_file, 'r') as orig_file:
> for pkg in Packages.iter_paragraphs(orig_file):
> p = pkg['Package']
> v = pkg['Version']
> a = pkg['Architecture']
> if spacewalk_packages.has_key(p + v + a):
> # found package. Check for missing headers
> for header in pkg.keys():
> if not header in spacewalk_packages[p + v + a].keys():
> spacewalk_packages[p + v + a][header] = pkg[header]
> # open new file
> new_package = open(spacewalk_file + '.new', 'w')
> for pkg in spacewalk_packages.values():
> pkg.dump(new_package)
> new_package.write("\n")
> new_package.close()
> sys.exit(0)
>
> Cheers,
> Robert
>
> *Gesendet:* Mittwoch, 05. Dezember 2018 um 21:10 Uhr
> *Von:* "philippe bidault" 
> *An:* robert.pasche...@web.de
> *Cc:* spacewalk-list@redhat.com
> *Betreff:* Re: [Spacewalk-list] Ubuntu 18.04 package management in
> Spacewalk 2.8
> Hi Robert,
>
> Good to know, thanks !!
>
> Regarding the version comparison logic issue for .deb packages, is there a
> solution that could be implemented ?
>
> Regards,
> Philippe.
>
> On Wed, 5 Dec 2018 at 20:18, Robert Paschedag 
> wrote:
>
>> Am 5. Dezember 2018 00:44:25 MEZ schrieb philippe bidault <
>> philippe.bida...@gmail.com>:
>> >Hi all,
>> >
>> >I am trying to make our Spacewalk 2.8 working with Ubuntu 18.04, but as
>> >well described here :
>> >
>> https://github.com/spacewalkproject/spacewalk/wiki/DebianUbuntuSupportIn27
>> >
>> >... there are 3 main issues

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-05 Thread Robert Paschedag 

Hi Philippe,

 

Welluntil now, I only have very few packages that are shown as upgradable for "Debian" systems. I only found some version comparision problem while testing Uyuni (and testing to get Debian system registered as Salt client). I think this could be further tuned to use the python-apt or python-dpkg package...but I did not have time to dig into that.

 

Within spacewalk, I currently do not know, where the comparision takes place (within java or within python).

 

The "modified" script is a "new" one. It does not depend on the - from myself - modified sync script. What it needs is the original Packages.gz file for the channel you sync.

 

Just throwing this here so you can test yourself. Basically, it should add all "missing" headers for every package.

 


import sys
import os
from debian.deb822 import *

if len(sys.argv) < 3:
    print "Usage: %s  " % sys.argv[0]
    sys.exit(1)

spacewalk_file = sys.argv[1]
original_file = sys.argv[2]

if not os.path.isfile(spacewalk_file):
    print "Error: Inputfile '%s' not available." % spacewalk_file
    sys.exit(1)
if not os.path.isfile(original_file):
    print "Error: Inputfile '%s' not available." % original_file
    sys.exit(1)

spacewalk_packages = {}
original_packages = {}

with open(spacewalk_file, 'r') as pkgs:
    for pkg in Packages.iter_paragraphs(pkgs):
    spacewalk_packages[pkg['Package'] + pkg['Version'] + pkg['Architecture']] = pkg

with open(original_file, 'r') as orig_file:
    for pkg in Packages.iter_paragraphs(orig_file):
    p = pkg['Package']
    v = pkg['Version']
    a = pkg['Architecture']
    if spacewalk_packages.has_key(p + v + a):
    # found package. Check for missing headers
    for header in pkg.keys():
    if not header in spacewalk_packages[p + v + a].keys():
    spacewalk_packages[p + v + a][header] = pkg[header]

# open new file
new_package = open(spacewalk_file + '.new', 'w')
for pkg in spacewalk_packages.values():
    pkg.dump(new_package)
    new_package.write("\n")
new_package.close()
sys.exit(0)

 

Cheers,

Robert


 

Gesendet: Mittwoch, 05. Dezember 2018 um 21:10 Uhr
Von: "philippe bidault" 
An: robert.pasche...@web.de
Cc: spacewalk-list@redhat.com
Betreff: Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8



Hi Robert,
 

Good to know, thanks !!

 

Regarding the version comparison logic issue for .deb packages, is there a solution that could be implemented ?

 

Regards,

Philippe.


 


On Wed, 5 Dec 2018 at 20:18, Robert Paschedag <robert.pasche...@web.de> wrote:

Am 5. Dezember 2018 00:44:25 MEZ schrieb philippe bidault <philippe.bida...@gmail.com>:
>Hi all,
>
>I am trying to make our Spacewalk 2.8 working with Ubuntu 18.04, but as
>well described here :
>https://github.com/spacewalkproject/spacewalk/wiki/DebianUbuntuSupportIn27
>
>... there are 3 main issues :
>
>"1- The current version comparison logic does not distinct a dot from
>an
>hyphen, a tilde or a plus character. This leads to some packages
>wrongly
>shown as an update for a client in spacewalk when the package is
>actually a
>downgrade. The client however uses a correct comparison and handles the
>package upgrade correctly
>
>2- The deb importer does not import all the package header information
>into
>the database and the repository-writer will not write the missing
>information to the repository metadata served by spacewalk. This will
>lead
>to problems on the client in case of the missing Multi-Arch header:
>Clients
>will try to reinstall the same package over and over again when this
>header
>is missing.
>
>3- A deb repository provided by spacewalk is not GPG signed and thus
>will
>not work without disabling secure-apt. Spacewalk imports and recreates
>the
>repository based on the imported package catalogue, this will destroy
>the
>GPG signing of the repository vendor."
>
>I did manage to solve the point 2 and 3. The point 3 thanks to this
>method
>:
>http://www.devops-blog.net/spacewalk/gpg-signing-apt-repository-in-spacewalk
>
>Regarding the point 2, I did have to customize the script
>https://github.com/rpasche/spacewalk-debian-sync/tree/add-multiarch-header
>,
>as it seems that in any case the add of the Multi-Arch header in
>Packages.gz is not enough for Ubuntu 18.04. I did only succeed in not
>ending in an infinite loop of packages flagged to be upgraded as the
>result
>of "apt upgrade" by adding more headers in Packages.gz :
>
>        p, v, a, multi, breaks, predeps = line.rstrip().split("||")
>       if (packages.has_key(p + v + a) and (multi is not "#")) :
>           packages[p + v + a]['Multi-Arch'] = multi
>       if (packages.has_k

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-05 Thread philippe bidault 
Hi Robert,

Good to know, thanks !!

Regarding the version comparison logic issue for .deb packages, is there a
solution that could be implemented ?

Regards,
Philippe.

On Wed, 5 Dec 2018 at 20:18, Robert Paschedag 
wrote:

> Am 5. Dezember 2018 00:44:25 MEZ schrieb philippe bidault <
> philippe.bida...@gmail.com>:
> >Hi all,
> >
> >I am trying to make our Spacewalk 2.8 working with Ubuntu 18.04, but as
> >well described here :
> >
> https://github.com/spacewalkproject/spacewalk/wiki/DebianUbuntuSupportIn27
> >
> >... there are 3 main issues :
> >
> >"1- The current version comparison logic does not distinct a dot from
> >an
> >hyphen, a tilde or a plus character. This leads to some packages
> >wrongly
> >shown as an update for a client in spacewalk when the package is
> >actually a
> >downgrade. The client however uses a correct comparison and handles the
> >package upgrade correctly
> >
> >2- The deb importer does not import all the package header information
> >into
> >the database and the repository-writer will not write the missing
> >information to the repository metadata served by spacewalk. This will
> >lead
> >to problems on the client in case of the missing Multi-Arch header:
> >Clients
> >will try to reinstall the same package over and over again when this
> >header
> >is missing.
> >
> >3- A deb repository provided by spacewalk is not GPG signed and thus
> >will
> >not work without disabling secure-apt. Spacewalk imports and recreates
> >the
> >repository based on the imported package catalogue, this will destroy
> >the
> >GPG signing of the repository vendor."
> >
> >I did manage to solve the point 2 and 3. The point 3 thanks to this
> >method
> >:
> >
> http://www.devops-blog.net/spacewalk/gpg-signing-apt-repository-in-spacewalk
> >
> >Regarding the point 2, I did have to customize the script
> >
> https://github.com/rpasche/spacewalk-debian-sync/tree/add-multiarch-header
> >,
> >as it seems that in any case the add of the Multi-Arch header in
> >Packages.gz is not enough for Ubuntu 18.04. I did only succeed in not
> >ending in an infinite loop of packages flagged to be upgraded as the
> >result
> >of "apt upgrade" by adding more headers in Packages.gz :
> >
> >p, v, a, multi, breaks, predeps = line.rstrip().split("||")
> >   if (packages.has_key(p + v + a) and (multi is not "#")) :
> >   packages[p + v + a]['Multi-Arch'] = multi
> >   if (packages.has_key(p + v + a) and (breaks is not "#")) :
> >   packages[p + v + a]['Breaks'] = breaks
> >   if (packages.has_key(p + v + a) and (predeps is not "#")) :
> >   packages[p + v + a]['Pre-Depends'] = predeps
>
> I did improve the script (still testing). Basically it just adds all
> missing headers from the original file to that generated by spacewalk. The
> repo is not yet updated.
>
> Robert
> >
> >Right now, what is really causing me much more trouble, is the point 1,
> >which is making Spacewalk not really reliable for deb package
> >management ,
> >as I have a permanent list of upgradable packages in the Spacewalk
> >console,
> >whereas the installed versions are already the latest.
> >
> >Example :
> >
> >
> >Latest Package
> >Installed Package
> >gcc-8-base-8-20180414-1ubuntu2.amd64-deb
> ><
> https://space01/rhn/software/packages/Details.do?sid=110058_combo=21120|7544|145
> >
> >gcc-8-base-8.2.0-1ubuntu2~18.04.amd64-deb
> >lib32gcc1-8-20180414-1ubuntu2:1.amd64-deb
> ><
> https://space01/rhn/software/packages/Details.do?sid=110058_combo=21933|7796|145
> >
> >lib32gcc1-8.2.0-1ubuntu2~18.04:1.amd64-deb
> >
> >
> >So in resume, I have 2 questions :
> >
> >- Regarding the point 1, is there already an existing solution I could
> >use,
> >or at least some clues ?
> >- Regarding the point 2 : Did I miss something, or do we really need to
> >add
> >some extra headers (Breaks and Pre-depends) to have the Ubuntu 18.04
> >client
> >servers correctly listing packages to be upgraded ?
> >
> >Regards,
> >Philippe.
>
>
> --
> sent from my mobile device
>
___
Spacewalk-list mailing list
Spacewalk-list@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list

Re: [Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-05 Thread Robert Paschedag 
Am 5. Dezember 2018 00:44:25 MEZ schrieb philippe bidault 
:
>Hi all,
>
>I am trying to make our Spacewalk 2.8 working with Ubuntu 18.04, but as
>well described here :
>https://github.com/spacewalkproject/spacewalk/wiki/DebianUbuntuSupportIn27
>
>... there are 3 main issues :
>
>"1- The current version comparison logic does not distinct a dot from
>an
>hyphen, a tilde or a plus character. This leads to some packages
>wrongly
>shown as an update for a client in spacewalk when the package is
>actually a
>downgrade. The client however uses a correct comparison and handles the
>package upgrade correctly
>
>2- The deb importer does not import all the package header information
>into
>the database and the repository-writer will not write the missing
>information to the repository metadata served by spacewalk. This will
>lead
>to problems on the client in case of the missing Multi-Arch header:
>Clients
>will try to reinstall the same package over and over again when this
>header
>is missing.
>
>3- A deb repository provided by spacewalk is not GPG signed and thus
>will
>not work without disabling secure-apt. Spacewalk imports and recreates
>the
>repository based on the imported package catalogue, this will destroy
>the
>GPG signing of the repository vendor."
>
>I did manage to solve the point 2 and 3. The point 3 thanks to this
>method
>:
>http://www.devops-blog.net/spacewalk/gpg-signing-apt-repository-in-spacewalk
>
>Regarding the point 2, I did have to customize the script
>https://github.com/rpasche/spacewalk-debian-sync/tree/add-multiarch-header
>,
>as it seems that in any case the add of the Multi-Arch header in
>Packages.gz is not enough for Ubuntu 18.04. I did only succeed in not
>ending in an infinite loop of packages flagged to be upgraded as the
>result
>of "apt upgrade" by adding more headers in Packages.gz :
>
>p, v, a, multi, breaks, predeps = line.rstrip().split("||")
>   if (packages.has_key(p + v + a) and (multi is not "#")) :
>   packages[p + v + a]['Multi-Arch'] = multi
>   if (packages.has_key(p + v + a) and (breaks is not "#")) :
>   packages[p + v + a]['Breaks'] = breaks
>   if (packages.has_key(p + v + a) and (predeps is not "#")) :
>   packages[p + v + a]['Pre-Depends'] = predeps

I did improve the script (still testing). Basically it just adds all missing 
headers from the original file to that generated by spacewalk. The repo is not 
yet updated.

Robert
>
>Right now, what is really causing me much more trouble, is the point 1,
>which is making Spacewalk not really reliable for deb package
>management ,
>as I have a permanent list of upgradable packages in the Spacewalk
>console,
>whereas the installed versions are already the latest.
>
>Example :
>
>
>Latest Package
>Installed Package
>gcc-8-base-8-20180414-1ubuntu2.amd64-deb
>
>gcc-8-base-8.2.0-1ubuntu2~18.04.amd64-deb
>lib32gcc1-8-20180414-1ubuntu2:1.amd64-deb
>
>lib32gcc1-8.2.0-1ubuntu2~18.04:1.amd64-deb
>
>
>So in resume, I have 2 questions :
>
>- Regarding the point 1, is there already an existing solution I could
>use,
>or at least some clues ?
>- Regarding the point 2 : Did I miss something, or do we really need to
>add
>some extra headers (Breaks and Pre-depends) to have the Ubuntu 18.04
>client
>servers correctly listing packages to be upgraded ?
>
>Regards,
>Philippe.


-- 
sent from my mobile device

___
Spacewalk-list mailing list
Spacewalk-list@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list

[Spacewalk-list] Ubuntu 18.04 package management in Spacewalk 2.8

2018-12-04 Thread philippe bidault 
Hi all,

I am trying to make our Spacewalk 2.8 working with Ubuntu 18.04, but as
well described here :
https://github.com/spacewalkproject/spacewalk/wiki/DebianUbuntuSupportIn27

... there are 3 main issues :

"1- The current version comparison logic does not distinct a dot from an
hyphen, a tilde or a plus character. This leads to some packages wrongly
shown as an update for a client in spacewalk when the package is actually a
downgrade. The client however uses a correct comparison and handles the
package upgrade correctly

2- The deb importer does not import all the package header information into
the database and the repository-writer will not write the missing
information to the repository metadata served by spacewalk. This will lead
to problems on the client in case of the missing Multi-Arch header: Clients
will try to reinstall the same package over and over again when this header
is missing.

3- A deb repository provided by spacewalk is not GPG signed and thus will
not work without disabling secure-apt. Spacewalk imports and recreates the
repository based on the imported package catalogue, this will destroy the
GPG signing of the repository vendor."

I did manage to solve the point 2 and 3. The point 3 thanks to this method
:
http://www.devops-blog.net/spacewalk/gpg-signing-apt-repository-in-spacewalk

Regarding the point 2, I did have to customize the script
https://github.com/rpasche/spacewalk-debian-sync/tree/add-multiarch-header ,
as it seems that in any case the add of the Multi-Arch header in
Packages.gz is not enough for Ubuntu 18.04. I did only succeed in not
ending in an infinite loop of packages flagged to be upgraded as the result
of "apt upgrade" by adding more headers in Packages.gz :

p, v, a, multi, breaks, predeps = line.rstrip().split("||")
   if (packages.has_key(p + v + a) and (multi is not "#")) :
   packages[p + v + a]['Multi-Arch'] = multi
   if (packages.has_key(p + v + a) and (breaks is not "#")) :
   packages[p + v + a]['Breaks'] = breaks
   if (packages.has_key(p + v + a) and (predeps is not "#")) :
   packages[p + v + a]['Pre-Depends'] = predeps

Right now, what is really causing me much more trouble, is the point 1,
which is making Spacewalk not really reliable for deb package management ,
as I have a permanent list of upgradable packages in the Spacewalk console,
whereas the installed versions are already the latest.

Example :


Latest Package
Installed Package
gcc-8-base-8-20180414-1ubuntu2.amd64-deb

gcc-8-base-8.2.0-1ubuntu2~18.04.amd64-deb
lib32gcc1-8-20180414-1ubuntu2:1.amd64-deb

lib32gcc1-8.2.0-1ubuntu2~18.04:1.amd64-deb


So in resume, I have 2 questions :

- Regarding the point 1, is there already an existing solution I could use,
or at least some clues ?
- Regarding the point 2 : Did I miss something, or do we really need to add
some extra headers (Breaks and Pre-depends) to have the Ubuntu 18.04 client
servers correctly listing packages to be upgraded ?

Regards,
Philippe.
___
Spacewalk-list mailing list
Spacewalk-list@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list