Re: [Xen-devel] preparations for 4.8.2

[Jan Beulich]

>>> On 17.08.17 at 16:37,  wrote:
> it’s been a while. Did you want to pick this up at some point again?

Yes, once Ian is back from vacation (and has sufficiently recovered
from mail and other backlog).

> I guess the check we have done so far is by now out-of-date.

Yes, with the recent XSAs and in particular with 226 not having had
its patches pushed right away.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations for 4.8.2

[Lars Kurth]

Jan,
it’s been a while. Did you want to pick this up at some point again? I guess 
the check we have done so far is by now out-of-date. Not sure whether anyone 
tagged anything
It would also be a good opportunity for you guys to test run my script (Wei ran 
it and it worked fine, but he didn’t comb through any results)
Lars

On 27/07/2017, 19:34, "Lars Kurth"  wrote:

Quick info/update:

> XSA-222: line 51 in the log shows a real difference: this is a known bug
> in the tool where the diff file chunks are in a different order

This is now fixed in the last version of the scripts and the script
correctly handles this case

Lars

On 18/07/2017, 18:43, "Lars Kurth"  wrote:

>Hi all,
>
>@Jan: you may want to check the note on XSA-218 and XSA-224
>
>I removed Text::Diff module, which should fix the dependency problem.
>
>I also fixed the script such that it will fetch patches from
>http://xenbits.xenproject.org/xsa if the xsa.git has not been checked out
>in the location in
>
>The script still depends on: Getopt, Cwd, File packages, which I hope are
>standard.
>
>Crude check
>===
>I first ran the scripts using
>
>./match-xsa --version 4 --major 8 --since 1 --xsa xsa-213-225 --getlogs
>--html > xsamatch.html
>
>Which checks name signatures only.
>Note that 
>https://xenproject.org/downloads/xen-archives/xen-project-48-series/xen-48
>1
>.html tells us that XSA 212 was applied last.
>
>The output shows that XSA-215 has not been applied. Not a problem, because
>XSA-215 applies to 64-bit Xen versions of 4.6 and earlier only.
>
>All the other ones have patches with matching names that have been
>applied.
>
>Detailed check
>==
>I then ran using
>
>
>./match-xsa --version 4 --major 8 --since 1 --xsa xsa-213-225 --html
>--smart > xsamatchsmart.html
>
>
>which requires that xsa.git is checked out, which has restricted access
>(security team members only).
>
>The output shows some problems, for which I used
>
>./match-xsa --version 4 --major 8 --since 1 --xsa xsa-213-225 --html
>--smart --debug > xsamatchsmartdebug.html
>
>
>This then tells me that there are a few real differences between 4.8.2 and
>the XSA database
>
>XSA-218: line 32 in the log shows a real difference: see XSA-218-32.png
>XSA-224: line 72 in the log shows a real difference: see XSA-224-72a.png &
>XSA-224-72b.png
>
>
>XSA-222: line 51 in the log shows a real difference: this is a known bug
>in the tool where the diff file chunks are in a different order
>
>Script Improvements
>===
>I can't use --xsadir https://xenbits.xenproject.org/xsa as I can't read
>files from a website. I can, fetch the file from
>https://xenbits.xenproject.org/xsa via the LWP:Simple package, which I
>don't think is installed on Linux distros by default. Alternatively I
>could use wget, which may be better.
>
>
>I will play with this and see whether I can add it.
>
>Cheers
>Lars
>
>
>On 18/07/2017, 14:53, "Wei Liu"  wrote:
>
>>On Tue, Jul 18, 2017 at 12:21:42PM +0100, Lars Kurth wrote:
>>> Wei,
>>> I attached the list output from xsa-list-send starting from 206
>>> If you look at 
>>> 
>>>https://xenproject.org/downloads/xen-archives/xen-project-48-series/xen-
>>>4
>>>81
>>> .html, you may want to start using from 213+
>>
>>[$]> ./match-xsa --version 4 --major 8 --since 2 --getlogs --xsa xsa-225
>>Can't locate Text/Diff.pm in @INC (you may need to install the
>>Text::Diff module) (@INC contains: /etc/perl
>>/usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1
>>/usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5
>>/usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24
>>/usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at
>>./match-xsa line 14.
>>BEGIN failed--compilation aborted at ./match-xsa line 14.
>>
>>Would be useful to give a list of perl modules required.
>



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations for 4.8.2

[Lars Kurth]

Quick info/update:

> XSA-222: line 51 in the log shows a real difference: this is a known bug
> in the tool where the diff file chunks are in a different order

This is now fixed in the last version of the scripts and the script
correctly handles this case

Lars

On 18/07/2017, 18:43, "Lars Kurth"  wrote:

>Hi all,
>
>@Jan: you may want to check the note on XSA-218 and XSA-224
>
>I removed Text::Diff module, which should fix the dependency problem.
>
>I also fixed the script such that it will fetch patches from
>http://xenbits.xenproject.org/xsa if the xsa.git has not been checked out
>in the location in
>
>The script still depends on: Getopt, Cwd, File packages, which I hope are
>standard.
>
>Crude check
>===
>I first ran the scripts using
>
>./match-xsa --version 4 --major 8 --since 1 --xsa xsa-213-225 --getlogs
>--html > xsamatch.html
>
>Which checks name signatures only.
>Note that 
>https://xenproject.org/downloads/xen-archives/xen-project-48-series/xen-48
>1
>.html tells us that XSA 212 was applied last.
>
>The output shows that XSA-215 has not been applied. Not a problem, because
>XSA-215 applies to 64-bit Xen versions of 4.6 and earlier only.
>
>All the other ones have patches with matching names that have been
>applied.
>
>Detailed check
>==
>I then ran using
>
>
>./match-xsa --version 4 --major 8 --since 1 --xsa xsa-213-225 --html
>--smart > xsamatchsmart.html
>
>
>which requires that xsa.git is checked out, which has restricted access
>(security team members only).
>
>The output shows some problems, for which I used
>
>./match-xsa --version 4 --major 8 --since 1 --xsa xsa-213-225 --html
>--smart --debug > xsamatchsmartdebug.html
>
>
>This then tells me that there are a few real differences between 4.8.2 and
>the XSA database
>
>XSA-218: line 32 in the log shows a real difference: see XSA-218-32.png
>XSA-224: line 72 in the log shows a real difference: see XSA-224-72a.png &
>XSA-224-72b.png
>
>
>XSA-222: line 51 in the log shows a real difference: this is a known bug
>in the tool where the diff file chunks are in a different order
>
>Script Improvements
>===
>I can't use --xsadir https://xenbits.xenproject.org/xsa as I can't read
>files from a website. I can, fetch the file from
>https://xenbits.xenproject.org/xsa via the LWP:Simple package, which I
>don't think is installed on Linux distros by default. Alternatively I
>could use wget, which may be better.
>
>
>I will play with this and see whether I can add it.
>
>Cheers
>Lars
>
>
>On 18/07/2017, 14:53, "Wei Liu"  wrote:
>
>>On Tue, Jul 18, 2017 at 12:21:42PM +0100, Lars Kurth wrote:
>>> Wei,
>>> I attached the list output from xsa-list-send starting from 206
>>> If you look at 
>>> 
>>>https://xenproject.org/downloads/xen-archives/xen-project-48-series/xen-
>>>4
>>>81
>>> .html, you may want to start using from 213+
>>
>>[$]> ./match-xsa --version 4 --major 8 --since 2 --getlogs --xsa xsa-225
>>Can't locate Text/Diff.pm in @INC (you may need to install the
>>Text::Diff module) (@INC contains: /etc/perl
>>/usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1
>>/usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5
>>/usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24
>>/usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at
>>./match-xsa line 14.
>>BEGIN failed--compilation aborted at ./match-xsa line 14.
>>
>>Would be useful to give a list of perl modules required.
>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations for 4.8.2

[Lars Kurth]

On 18/07/2017, 14:53, "Wei Liu"  wrote:

>On Tue, Jul 18, 2017 at 12:21:42PM +0100, Lars Kurth wrote:
>> Wei,
>> I attached the list output from xsa-list-send starting from 206
>> If you look at 
>> 
>>https://xenproject.org/downloads/xen-archives/xen-project-48-series/xen-4
>>81
>> .html, you may want to start using from 213+
>
>[$]> ./match-xsa --version 4 --major 8 --since 2 --getlogs --xsa xsa-225
>Can't locate Text/Diff.pm in @INC (you may need to install the
>Text::Diff module) (@INC contains: /etc/perl
>/usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1
>/usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5
>/usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24
>/usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at
>./match-xsa line 14.
>BEGIN failed--compilation aborted at ./match-xsa line 14.
>
>Would be useful to give a list of perl modules required.

These are at the top of the file: Getopt::Long qw(GetOptions), Cwd,
File::Slurp, Text::Diff, File::Spec;
Text::Diff may be obsolete - I used the diff function and then removed it
later because system ('diff ...') worked better for me. I can check and
remove the "use"

Lars 



>

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations for 4.8.2

[Wei Liu]

On Tue, Jul 18, 2017 at 12:21:42PM +0100, Lars Kurth wrote:
> Wei,
> I attached the list output from xsa-list-send starting from 206
> If you look at 
> https://xenproject.org/downloads/xen-archives/xen-project-48-series/xen-481
> .html, you may want to start using from 213+

[$]> ./match-xsa --version 4 --major 8 --since 2 --getlogs --xsa xsa-225
Can't locate Text/Diff.pm in @INC (you may need to install the
Text::Diff module) (@INC contains: /etc/perl
/usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1
/usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5
/usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24
/usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at
./match-xsa line 14.
BEGIN failed--compilation aborted at ./match-xsa line 14.

Would be useful to give a list of perl modules required.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations for 4.8.2

[Lars Kurth]

Wei,
I attached the list output from xsa-list-send starting from 206
If you look at 
https://xenproject.org/downloads/xen-archives/xen-project-48-series/xen-481
.html, you may want to start using from 213+
Lars

On 17/07/2017, 12:40, "Wei Liu"  wrote:

>On Mon, Jul 17, 2017 at 09:17:23AM +0100, Lars Kurth wrote:
>> Folks,
>> 
>> I didn't run the XSA script. Maybe someone can have a go and test out
>>the
>> instructions in 
>> 
>>https://xenbits.xenproject.org/gitweb/?p=people/larsk/xen-release-scripts
>>.g
>> it;a=summary
>> The scripts does requireS XSA.GIT to be checked out, but can be changed
>> easily to fetch XSAs from xenbits: line 26, and then follow $XSADIR
>> 
>> In fact --xsadir http://xenbits.xenproject.org/xsa may just work
>> 
>> Lars
>> 
>
>I tried to follow the instructions in README for match-xsa. I believe
>the xsa-list-send script in step 3 depends on xsa.git, which I don't
>have access to.

206 
xsa206-unstable/0001-xenstored-apply-a-write-transaction-rate-limit.patch   
xenstored: apply a write transaction rate limit
206 
xsa206-unstable/0002-xenstored-Log-when-the-write-transaction-rate-limit-.patch 
xenstored: Log when the write transaction rate limit bites
206 
xsa206-unstable/0003-oxenstored-comments-explaining-some-variables.patch
oxenstored: comments explaining some variables
206 
xsa206-unstable/0004-oxenstored-handling-of-domain-conflict-credit.patch
oxenstored: handling of domain conflict-credit
206 
xsa206-unstable/0005-oxenstored-ignore-domains-with-no-conflict-credit.patch
oxenstored: ignore domains with no conflict-credit
206 
xsa206-unstable/0006-oxenstored-add-transaction-info-relevant-to-history-.patch 
oxenstored: add transaction info relevant to history-tracking
206 xsa206-unstable/0007-oxenstored-support-commit-history-tracking.patch   
oxenstored: support commit history tracking
206 
xsa206-unstable/0008-oxenstored-only-record-operations-with-side-effects-.patch 
oxenstored: only record operations with side-effects in history
206 
xsa206-unstable/0009-oxenstored-discard-old-commit-history-on-txn-end.patch 
oxenstored: discard old commit-history on txn end
206 xsa206-unstable/0010-oxenstored-track-commit-history.patch  
oxenstored: track commit history
206 
xsa206-unstable/0011-oxenstored-blame-the-connection-that-caused-a-transa.patch 
oxenstored: blame the connection that caused a transaction conflict
206 xsa206-unstable/0012-oxenstored-allow-self-conflicts.patch  
oxenstored: allow self-conflicts
206 
xsa206-unstable/0013-oxenstored-do-not-commit-read-only-transactions.patch  
oxenstored: do not commit read-only transactions
206 
xsa206-unstable/0014-oxenstored-don-t-wake-to-issue-no-conflict-credit.patch
oxenstored: don't wake to issue no conflict-credit
206 
xsa206-unstable/0015-oxenstored-transaction-conflicts-improve-logging.patch 
oxenstored transaction conflicts: improve logging
206 
xsa206-unstable/0016-oxenstored-trim-history-in-the-frequent_ops-function.patch 
oxenstored: trim history in the frequent_ops function
206 xsa206-4.4/0001-xenstored-apply-a-write-transaction-rate-limit.patch
xenstored: apply a write transaction rate limit
206 
xsa206-4.4/0002-xenstored-Log-when-the-write-transaction-rate-limit-.patch  
xenstored: Log when the write transaction rate limit bites
206 xsa206-4.4/0003-oxenstored-exempt-dom0-from-domU-node-quotas.patch  
oxenstored: exempt dom0 from domU node quotas
206 
xsa206-4.4/0004-oxenstored-perform-a-3-way-merge-of-the-quota-after-.patch  
oxenstored: perform a 3-way merge of the quota after a transaction
206 
xsa206-4.4/0005-oxenstored-catch-the-error-when-a-connection-is-alre.patch  
oxenstored: catch the error when a connection is already deleted
206 
xsa206-4.4/0006-oxenstored-use-hash-table-to-store-socket-connection.patch  
oxenstored: use hash table to store socket connections
206 
xsa206-4.4/0007-oxenstored-enable-domain-connection-indexing-based-o.patch  
oxenstored: enable domain connection indexing based on eventchn port
206 
xsa206-4.4/0008-oxenstored-only-process-domain-connections-that-noti.patch  
oxenstored: only process domain connections that notify us by events
206 
xsa206-4.4/0009-oxenstored-add-a-safe-net-mechanism-for-existing-ill.patch  
oxenstored: add a safe net mechanism for existing ill-behaved clients
206 xsa206-4.4/0010-oxenstored-refactor-putting-response-on-wire.patch  
oxenstored: refactor putting response on wire
206 xsa206-4.4/0011-oxenstored-remove-some-unused-parameters.patch  
oxenstored: remove some unused parameters
206 

Re: [Xen-devel] preparations for 4.8.2

[Lars Kurth]

> I tried to follow the instructions in README for match-xsa. I believe
> the xsa-list-send script in step 3 depends on xsa.git, which I don't
> have access to.
That is unfortunately correct: we ought to fix this.
Lars


On 17/07/2017, 12:40, "Wei Liu"  wrote:

>On Mon, Jul 17, 2017 at 09:17:23AM +0100, Lars Kurth wrote:
>> Folks,
>> 
>> I didn't run the XSA script. Maybe someone can have a go and test out
>>the
>> instructions in 
>> 
>>https://xenbits.xenproject.org/gitweb/?p=people/larsk/xen-release-scripts
>>.g
>> it;a=summary
>> The scripts does requireS XSA.GIT to be checked out, but can be changed
>> easily to fetch XSAs from xenbits: line 26, and then follow $XSADIR
>> 
>> In fact --xsadir http://xenbits.xenproject.org/xsa may just work
>> 
>> Lars
>> 
>
>I tried to follow the instructions in README for match-xsa. I believe
>the xsa-list-send script in step 3 depends on xsa.git, which I don't
>have access to.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations for 4.8.2

[Wei Liu]

On Mon, Jul 17, 2017 at 09:17:23AM +0100, Lars Kurth wrote:
> Folks,
> 
> I didn't run the XSA script. Maybe someone can have a go and test out the
> instructions in 
> https://xenbits.xenproject.org/gitweb/?p=people/larsk/xen-release-scripts.g
> it;a=summary
> The scripts does requireS XSA.GIT to be checked out, but can be changed
> easily to fetch XSAs from xenbits: line 26, and then follow $XSADIR
> 
> In fact --xsadir http://xenbits.xenproject.org/xsa may just work
> 
> Lars
> 

I tried to follow the instructions in README for match-xsa. I believe
the xsa-list-send script in step 3 depends on xsa.git, which I don't
have access to.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations for 4.8.2

[Lars Kurth]

Folks,

I didn't run the XSA script. Maybe someone can have a go and test out the
instructions in 
https://xenbits.xenproject.org/gitweb/?p=people/larsk/xen-release-scripts.g
it;a=summary
The scripts does requireS XSA.GIT to be checked out, but can be changed
easily to fetch XSAs from xenbits: line 26, and then follow $XSADIR

In fact --xsadir http://xenbits.xenproject.org/xsa may just work

Lars

On 17/07/2017, 10:01, "Wei Liu"  wrote:

>On Thu, Jul 06, 2017 at 01:17:02AM -0600, Jan Beulich wrote:
>> All,
>> 
>> with the goal of releasing in the first half of August (once I'm back
>> from vacation and had time to sync back up, and the tree has got
>> the necessary push), please point out backport candidates you
>> find missing from the respective staging branches, but which you
>> consider relevant. Note that commit 2ff229643b ("livepatch: Don't
>> crash on encountering STN_UNDEF relocations") is already on my
>> list; I'm not fully decided on bd53b85156 ("livepatch: Use zeroed
>> memory allocations for arrays") yet, but I tend towards taking it as
>> long as it applies reasonably cleanly (which I expect it will do).
>> 
>> Thanks, Jan
>> 
>
>xen-RELEASE-4.8.2 tagged in mini-os.git.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] preparations for 4.8.2

[Wei Liu]

On Thu, Jul 06, 2017 at 01:17:02AM -0600, Jan Beulich wrote:
> All,
> 
> with the goal of releasing in the first half of August (once I'm back
> from vacation and had time to sync back up, and the tree has got
> the necessary push), please point out backport candidates you
> find missing from the respective staging branches, but which you
> consider relevant. Note that commit 2ff229643b ("livepatch: Don't
> crash on encountering STN_UNDEF relocations") is already on my
> list; I'm not fully decided on bd53b85156 ("livepatch: Use zeroed
> memory allocations for arrays") yet, but I tend towards taking it as
> long as it applies reasonably cleanly (which I expect it will do).
> 
> Thanks, Jan
> 

xen-RELEASE-4.8.2 tagged in mini-os.git.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] preparations for 4.8.2

[Jan Beulich]

All,

with the goal of releasing in the first half of August (once I'm back
from vacation and had time to sync back up, and the tree has got
the necessary push), please point out backport candidates you
find missing from the respective staging branches, but which you
consider relevant. Note that commit 2ff229643b ("livepatch: Don't
crash on encountering STN_UNDEF relocations") is already on my
list; I'm not fully decided on bd53b85156 ("livepatch: Use zeroed
memory allocations for arrays") yet, but I tend towards taking it as
long as it applies reasonably cleanly (which I expect it will do).

Thanks, Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel