Re: Verifiable Releases/The Build System is Ridiculous

[R0b0t1]

Thank you for the contributions related to W^X.

In regards to signed releases, well, I can't make Perl 6 developers do
anything they don't want to do. I can explain the process or commands
or even manage it myself (which would admittedly be a bit strange,
having made no other contributions) but if none of those solutions are
satisfactory and release signing isn't undertaken then that's it. I
think it is reasonable that it hasn't been considered yet. But
opposing it, or opposing secure use of Git, is something I would
suggest is completely indefensible, especially in a large project.


I very much intend to not clutter the list with drivel, but I think it
useful to add responses to some of the latest comments for when the
issue of release signing invariably comes up again:

On Sat, Jul 29, 2017 at 2:13 AM, Steve Mynott  wrote:
> Submodules *are* already used as I said previously or would be obvious
> to anyone reading the code. I'd recommend doing the latter before
> posting.
>

The specific instance where they aren't used is the Star build system,
where things are downloaded from a link in what used to be an insecure
manner. If I am understanding the explanation this isn't how it's
supposed to work but I'm still trying to figure out a way that
generates Star from, essentially, a repository full of submodules,
because as I understand it R* is just the compiler and a module
system.

Yes, I will contribute if I think it makes sense to do so. I'm pretty
sure I understand what is going on and haven't been able to receive
clarification. I don't have any problem being told my question doesn't
make sense, but it doesn't seem like that has happened.

> Anyway I'm not replying to this thread anymore since it's obvious we
> are getting to the point of diminishing returns.
>

Despite my choice of words I was hoping the presence of a well formed
argument would indicate that I was trying to have a discussion.
Ignoring what someone has to say because you disagree with them is a
great way to culture poorly founded opinions.

> An example of what isn't useful is a long self indulgent wordy rant
> about how no one else understands security but yourself.  Most people
> will start ignoring this sort of thing after a while if they haven't
> already.
>

I spent three or four hours researching and putting together all of
those replies. I did not do them for my enjoyment, save for that I get
from making Perl 6 and its ecosystem more secure.


On Sat, Jul 29, 2017 at 3:49 AM, Joachim Durchholz  wrote:
> Am 29.07.2017 um 05:20 schrieb R0b0t1:
>> They behave more like a tracked file and are fairly opaque.
>
>
> Actually they behave like a directory with *untracked* files.
>

Everything in them is untracked in relation to the main repo, but you
can specify a tag for the submodule that refers to a point in time for
the referenced repository. In a sense the specified tag is like the
contents of a file.

>> I wasn't referring to the use of bytecode, but the inclusion of
>> bytecode blobs in the distribution.
>
>
> Yeah, I didn't like that myself.
> These blobs are there just for bootstrapping, and are supposed to be the
> bytecode equivalent of the source code in the nqp tree. So it's not making
> stuff opaque, but it's creating a loophole for perpetuating compilation
> bugs, and for inserting malicious modifications.
>

At the same time, it's not source code and is harder to look at.

> I once reported that but it didn't seem relevant enough to get fixed -
> probably because the mere possibility of a problem combined with a lot of
> work to fix is less value for effort than all the other things that aren't
> up to speed yet.
>
> On the plus side, postponing getting rid of the blobs does not increase the
> effort involved, so it can be fixed whenever somebody with enough time and
> willingness comes along.
>

>
>>> Yes and our existing build system is the one we have now. The way to
>>> fix it is gradually by submitting small fixes not talking about
>>> replacing it by your pet build system.
>>
>>
>> If you think Git is my pet project I am afraid you have mistaken me
>> for Linux Torvalds.
>
>
> git is not a build system.
>

True, but it can be used in one.

>>> Security is desirable but not currently a main objective of this
>>> project which is more about creating a reasonably fast and very
>>> expressive computer language. If your main interest is security then
>>> you might be better off with a project like OpenBSD and helping update
>>> their Rakudo ports. MoarVM has problems with their W^X system if you
>>> run certain of the roast tests which needs investigation.
>>
>>
>> Security is something that needs to be designed in from the start,
>> otherwise it can be almost impossible to accomplish.
>
>
> Fortunately, binary blobs are not a problem of that kind, nor is changing
> the build system.

The build system I'm referring to until very recently ran `wget
--no-check-certificates`. How 

Re: Verifiable Releases/The Build System is Ridiculous

[Mark Montague]

On 2017-07-29 18:38, Timo Paulssen wrote:

However, an executable heap is still necessary even though an
executable stack is not needed when MoarVM built to use libffi 3.1 or
later:

This is most likely due to the jit, which allocates a frame, generates
machine code into it, then jumps into it. Can you check if the
environment variable MVM_JIT_DISABLE makes it work even with deny_execmem?


Yes, that makes it work even with deny_execmem:

[markmont@f26test rakudo-star-2017.07]$ getsebool deny_execmem
deny_execmem --> on
[markmont@f26test rakudo-star-2017.07]$ env MVM_JIT_DISABLE=1 perl6 -e 
"say 'hello, world';"

hello, world
[markmont@f26test rakudo-star-2017.07]$


Maybe we can react to not being allowed to set the page executable and
just turn off the jit "forever" at run time.

thanks
   - Timo


Thanks for the insight!  I'm not sure if turning off JIT is a good 
thing, but it's great to know what is happening and why.


--
  Mark Montague
  m...@catseye.org

Re: Verifiable Releases/The Build System is Ridiculous

[Mark Montague]

On 2017-07-29 08:28, Timo Paulssen wrote:

>> The reliance on W^X violating behavior is something I would like >> to see 
removed,
Actually what they are refering to is that dyncall and libffi both 
require an executable stack. We can't get around that without making 
changes to libffi and dyncall, sadly.


I believe that libffi versions 3.1 and later do not require executable 
stacks per

https://access.redhat.com/security/cve/CVE-2017-1000376

I've tested this with rakudo-star-2017.07 under Fedora 26.  A default 
build of Rakudo Star does NOT use libffi and DOES require an executable 
stack:


[markmont@f26test rakudo-star-2017.07]$ getsebool selinuxuser_execstack
selinuxuser_execstack --> on
[markmont@f26test rakudo-star-2017.07]$ perl6 -e "say 'hello, world';"
hello, world
[markmont@f26test rakudo-star-2017.07]$ sudo setsebool 
selinuxuser_execstack off

[markmont@f26test rakudo-star-2017.07]$ getsebool selinuxuser_execstack
selinuxuser_execstack --> off
[markmont@f26test rakudo-star-2017.07]$ perl6 -e "say 'hello, world';"
/home/markmont/perl6/perl6/bin/moar: error while loading shared 
libraries: libmoar.so: cannot enable executable stack as shared object 
requires: Permission denied

[markmont@f26test rakudo-star-2017.07]$

I then made sure I had the libffi and libffi-devel RPMs installed, 
edited MoarVM/Configure.pl to turn on "--has-libffi", and rebuilt all of 
Rakudo Star.  When using libffi 3.1 or later, executable stacks are no 
longer required:


[markmont@f26test rakudo-star-2017.07]$ getsebool selinuxuser_execstack
selinuxuser_execstack --> off
[markmont@f26test rakudo-star-2017.07]$ perl6 -e "say 'hello, world';"
hello, world
[markmont@f26test rakudo-star-2017.07]$ make rakudo-test | tail -4
All tests successful.
Files=47, Tests=616, 26 wallclock secs ( 0.12 usr  0.04 sys + 23.96 
cusr  2.84 csys = 26.96 CPU)

Result: PASS
make[1]: Leaving directory '/home/markmont/perl6/rakudo-star-2017.07/rakudo'
[markmont@f26test rakudo-star-2017.07]$ make rakudo-spectest | tail -3
Files=1092, Tests=51281, 895 wallclock secs ( 5.86 usr  1.34 sys + 
632.08 cusr 69.38 csys = 708.66 CPU)

Result: PASS
make[1]: Leaving directory '/home/markmont/perl6/rakudo-star-2017.07/rakudo'
[markmont@f26test rakudo-star-2017.07]$


However, an executable heap is still necessary even though an executable 
stack is not needed when MoarVM built to use libffi 3.1 or later:


[markmont@f26docker rakudo-star-2017.07]$ getsebool deny_execmem
deny_execmem --> off
[markmont@f26docker rakudo-star-2017.07]$ perl6 -e "say 'hello, world';"
hello, world
[markmont@f26docker rakudo-star-2017.07]$ sudo setsebool deny_execmem on
[markmont@f26docker rakudo-star-2017.07]$ getsebool deny_execmem
deny_execmem --> on
[markmont@f26docker rakudo-star-2017.07]$ perl6 -e "say 'hello, world';"
Segmentation fault (core dumped)
[markmont@f26docker rakudo-star-2017.07]$ sudo grep denied 
/var/log/audit/audit.log
type=AVC msg=audit(1501367074.731:347): avc:  denied  { execmem } for  
pid=40078 comm="moar" 
scontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 
tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 
tclass=process permissive=0

[markmont@f26docker rakudo-star-2017.07]$


--
  Mark Montague
  m...@catseye.org

Re: Verifiable Releases/The Build System is Ridiculous

[Joachim Durchholz]

Am 29.07.2017 um 14:28 schrieb Timo Paulssen:


Actually what they are refering to is that dyncall and libffi both 
require an executable stack. We can't get around that without making 
changes to libffi and dyncall, sadly.


Ah okay, that was outside the bounds of my knowledge.
And I agree it's unfortunate.

OTOH if these calls are constructed at runtime, we can't avoid W|X 
memory. The best we can do is to limit its size, and move it off the 
normal call stack so that forcing an array overflow doesn't instantly 
give the attacker access to an executable area of which he can control 
the content.


Question is: Is Perl's bytecode guaranteed to never smash an array 
bound? E.g. JVM bytecode does an array bounds check before any access.
There's a simlar question about object type mixups, which can cause code 
to access offsets beyond the end of the object. (The JVM prevents this 
kind of object bound smashing via type validation, mostly at code load 
time and the remaining cases when dereferencing.)

Re: Verifiable Releases/The Build System is Ridiculous

[Timo Paulssen]

>> The reliance on W^X violating behavior is something I would like >> to see 
>> removed, > > That behaviour does not exist. The binary blobs
aren't created as > part of the normal build process, and even if they
were, the code > writes the bytecodes to disk, it does not directly
execute them.
Actually what they are refering to is that dyncall and libffi both
require an executable stack. We can't get around that without making
changes to libffi and dyncall, sadly.

Re: Verifiable Releases/The Build System is Ridiculous

[Joachim Durchholz]

Am 29.07.2017 um 05:20 schrieb R0b0t1:

Most issues I have seen that arise with submodules come from people
trying to treat the submodule directory in a way that is different
than other objects tracked by Git. If you treat it like a source file
you're tracking most problems should disappear, at least in theory.


Dunno what the problems were for Perl6.
Versioning generated (non-source) files can indeed create problems, but
that's independently of whether it's submodule or not, so don't know what
problem you're seeing here.


No, I meant that I think people were expecting symlink or directory
like behavior from submodules based on the criticism I was reading.


Ah ok.


They behave more like a tracked file and are fairly opaque.


Actually they behave like a directory with *untracked* files.


I wasn't referring to the use of bytecode, but the inclusion of
bytecode blobs in the distribution.


Yeah, I didn't like that myself.
These blobs are there just for bootstrapping, and are supposed to be the 
bytecode equivalent of the source code in the nqp tree. So it's not 
making stuff opaque, but it's creating a loophole for perpetuating 
compilation bugs, and for inserting malicious modifications.


I once reported that but it didn't seem relevant enough to get fixed - 
probably because the mere possibility of a problem combined with a lot 
of work to fix is less value for effort than all the other things that 
aren't up to speed yet.


On the plus side, postponing getting rid of the blobs does not increase 
the effort involved, so it can be fixed whenever somebody with enough 
time and willingness comes along.


> If there are handwritten pieces

that must remain as blobs then I would hope they are commented.


AFAIK there are not.


Yes and our existing build system is the one we have now. The way to
fix it is gradually by submitting small fixes not talking about
replacing it by your pet build system.


If you think Git is my pet project I am afraid you have mistaken me
for Linux Torvalds.


git is not a build system.


Security is desirable but not currently a main objective of this
project which is more about creating a reasonably fast and very
expressive computer language. If your main interest is security then
you might be better off with a project like OpenBSD and helping update
their Rakudo ports. MoarVM has problems with their W^X system if you
run certain of the roast tests which needs investigation.


Security is something that needs to be designed in from the start,
otherwise it can be almost impossible to accomplish.


Fortunately, binary blobs are not a problem of that kind, nor is 
changing the build system.
It's more a social issue because you need consensus, but once consensus 
is there, it's not one of the typical "you can't retrofit security" 
issues but merely a "it's always been a lot of work but it can still be 
done" issue.


E.g. for the binary blobs, Steve Mynott reported that he wasn't 
successful recreating the blobs in the way they were originally made, 
but I can still imagine that it should be possible to write an NQP 
interpreter in Perl5. With that, you can bootstrap the NQP interpreter 
written in NQP without needing non-reviewable binary blobs.
Another approach would be to write a bytecode disassembler, just so that 
the bytecode becomes reviewable.


Similar considerations apply to the build system. It's essentially a set 
of makefiles that run various Perl5 tools and git commands - which is 
actually appropriate for a project like Perl6 which assumes that 
everybody knows Perl5.
Like with all large codebases, there's always a lot of details that can 
be improved. Sometimes it's as easy as regularizing a makefile so it 
becomes more readable, sometimes it's architectural changes to make 
tools interoperate better.
I agree with Steve that it's going to be more useful to understand how 
things are working, and proposing improvements, rather than make 
sweeping statements about general principles without sending patches.



The reliance on W^X violating behavior is something I would like to
see removed,


That behaviour does not exist. The binary blobs aren't created as part 
of the normal build process, and even if they were, the code writes the 
bytecodes to disk, it does not directly execute them.



I can say with no small amount of surety that if an OpenBSD developer
ever read what you just had to say about security they would laugh in
your face and call you an ignorant boob.

Please try to separate what I am saying from any personal attack. I
use strong language to indicate how severely misinformed you are.


Actually that *is* a personal attack. You are hypothesizing about a 
ridiculing reaction from somebody you don't even know personally (the 
hypothetical OpenBSD developer you're talking about), and saying 
"severely misinformed" is one of the strongest insults you can fling at 
somebody in a technical group.



On another note, I am please that I remind you of an 

Re: Verifiable Releases/The Build System is Ridiculous

[Steve Mynott]

Submodules *are* already used as I said previously or would be obvious
to anyone reading the code. I'd recommend doing the latter before
posting.

Anyway I'm not replying to this thread anymore since it's obvious we
are getting to the point of diminishing returns.

If you have improvements please submit short pull requests via github
-- a good example of the sort of thing which is useful is

https://github.com/rakudo/star/commit/9c2f2b51a9a078ea34d4cbeb7122196948d05d48

Short, simple non-controversial pull requests stand the greatest
chance of being merged. In most cases you will need to run the code
successfully to confirm it works.

An example of what isn't useful is a long self indulgent wordy rant
about how no one else understands security but yourself.  Most people
will start ignoring this sort of thing after a while if they haven't
already.

Cheers Steve

On 29 July 2017 at 04:29, R0b0t1  wrote:
> On Fri, Jul 28, 2017 at 10:20 PM, R0b0t1  wrote:
>> Hello,
>>
>> I apologize for the very long message. Yes, replies to three people
>> are in there. The responses were appreciated and I have replied to
>> them where replies were warranted.
>>
>>
>> On Fri, Jul 28, 2017 at 2:39 AM, Joachim Durchholz  
>> wrote:
>>> Am 28.07.2017 um 04:35 schrieb R0b0t1:
>
> The earliest versions of the Rakudo Star build system started out by
> trying to use Git submodules to manage packages, but it quickly proved to 
> be
> unwieldy and almost impossible to understand and maintain.  Perhaps the
> submodule ecosystem has changed since then, though.


 Can you give an example of how submodules were insufficient?
>>>
>>>
>>> I don't know what was unwieldy for the Perl6 guys, but having to manage
>>> multiple repositories for a given task is always some extra steps when
>>> synchronizing new code to the public repositories. If one of those steps is
>>> forgotten, everybody will see repositories that won't work, or show weird
>>> problems.
>>>
>>> Submodules are built for the use case that repositories evolve independently
>>> of each other, subtrees for the case that they evolve in sync. It's possible
>>> that submodules were the wrong approach, or that subtrees didn't work well
>>> enough at the point in time, or that nobody found the time to set everything
>>> up well enough to make it really work, or for lack of knowledge how to get
>>> submodules to work well.
>>> Since everybody's time is constrained, and Perl6 is still a work in
>>> progress, there is a long list of things that could be improved, so it's no
>>> surprise to see defects. The more important question is whether defects are
>>> important.
>>>
>>
>> That is a decent enough explanation. If anyone can chime in with
>> specifics I am still interested, as I don't see how learning the dance
>> for submodules is any different than learning the dance for Git in
>> general. That explanation makes sense if submodules were investigated
>> before being used in depth, but
>>
>
> Whoops.
>
> That explanation makes sense if submodules were investigated before
> being used in depth, but someone already commented and said that they
> were being used but were abandoned.
>
> R0b0t1.



-- 
4096R/EA75174B Steve Mynott 

Re: Verifiable Releases/The Build System is Ridiculous

[R0b0t1]

On Fri, Jul 28, 2017 at 10:20 PM, R0b0t1  wrote:
> Hello,
>
> I apologize for the very long message. Yes, replies to three people
> are in there. The responses were appreciated and I have replied to
> them where replies were warranted.
>
>
> On Fri, Jul 28, 2017 at 2:39 AM, Joachim Durchholz  wrote:
>> Am 28.07.2017 um 04:35 schrieb R0b0t1:

 The earliest versions of the Rakudo Star build system started out by
 trying to use Git submodules to manage packages, but it quickly proved to 
 be
 unwieldy and almost impossible to understand and maintain.  Perhaps the
 submodule ecosystem has changed since then, though.
>>>
>>>
>>> Can you give an example of how submodules were insufficient?
>>
>>
>> I don't know what was unwieldy for the Perl6 guys, but having to manage
>> multiple repositories for a given task is always some extra steps when
>> synchronizing new code to the public repositories. If one of those steps is
>> forgotten, everybody will see repositories that won't work, or show weird
>> problems.
>>
>> Submodules are built for the use case that repositories evolve independently
>> of each other, subtrees for the case that they evolve in sync. It's possible
>> that submodules were the wrong approach, or that subtrees didn't work well
>> enough at the point in time, or that nobody found the time to set everything
>> up well enough to make it really work, or for lack of knowledge how to get
>> submodules to work well.
>> Since everybody's time is constrained, and Perl6 is still a work in
>> progress, there is a long list of things that could be improved, so it's no
>> surprise to see defects. The more important question is whether defects are
>> important.
>>
>
> That is a decent enough explanation. If anyone can chime in with
> specifics I am still interested, as I don't see how learning the dance
> for submodules is any different than learning the dance for Git in
> general. That explanation makes sense if submodules were investigated
> before being used in depth, but
>

Whoops.

That explanation makes sense if submodules were investigated before
being used in depth, but someone already commented and said that they
were being used but were abandoned.

R0b0t1.

Re: Verifiable Releases/The Build System is Ridiculous

[R0b0t1]

Hello,

I apologize for the very long message. Yes, replies to three people
are in there. The responses were appreciated and I have replied to
them where replies were warranted.


On Fri, Jul 28, 2017 at 2:39 AM, Joachim Durchholz  wrote:
> Am 28.07.2017 um 04:35 schrieb R0b0t1:
>>>
>>> The earliest versions of the Rakudo Star build system started out by
>>> trying to use Git submodules to manage packages, but it quickly proved to be
>>> unwieldy and almost impossible to understand and maintain.  Perhaps the
>>> submodule ecosystem has changed since then, though.
>>
>>
>> Can you give an example of how submodules were insufficient?
>
>
> I don't know what was unwieldy for the Perl6 guys, but having to manage
> multiple repositories for a given task is always some extra steps when
> synchronizing new code to the public repositories. If one of those steps is
> forgotten, everybody will see repositories that won't work, or show weird
> problems.
>
> Submodules are built for the use case that repositories evolve independently
> of each other, subtrees for the case that they evolve in sync. It's possible
> that submodules were the wrong approach, or that subtrees didn't work well
> enough at the point in time, or that nobody found the time to set everything
> up well enough to make it really work, or for lack of knowledge how to get
> submodules to work well.
> Since everybody's time is constrained, and Perl6 is still a work in
> progress, there is a long list of things that could be improved, so it's no
> surprise to see defects. The more important question is whether defects are
> important.
>

That is a decent enough explanation. If anyone can chime in with
specifics I am still interested, as I don't see how learning the dance
for submodules is any different than learning the dance for Git in
general. That explanation makes sense if submodules were investigated
before being used in depth, but

Something being hard to teach to people to use is a valid concern, but
I am still surprised at the amount of effort spent to replace
submodules. I still think there is something I am missing.

>> Most issues I have seen that arise with submodules come from people
>> trying to treat the submodule directory in a way that is different
>> than other objects tracked by Git. If you treat it like a source file
>> you're tracking most problems should disappear, at least in theory.
>
>
> Dunno what the problems were for Perl6.
> Versioning generated (non-source) files can indeed create problems, but
> that's independently of whether it's submodule or not, so don't know what
> problem you're seeing here.
>

No, I meant that I think people were expecting symlink or directory
like behavior from submodules based on the criticism I was reading.
They behave more like a tracked file and are fairly opaque.

>> There's still some unfortunate submodule command names.
>
>
> git's ergonomics is generally not that good. Projects still stick with it, I
> have several theories about the reasons but don't know which of them apply.
>
>> NQP? I was told that has bytecode in it. If possible I would request
>> that this is changed in the future.
>
>
> NQP is just the language that large parts of the Perl6 compiler are written
> in.
>
> Bytecode is what the AST is compiled to. One could write a different
> backend, translating either AST or bytecode down to machine code; it's just
> that nobody did it yet. It would be a pretty large project, so I don't count
> on that happening anytime soon (but then the Perl community can be pretty
> amazing and incredible things happen on a semi-regular basis, so I'm not
> counting on that *not* happening either).

I wasn't referring to the use of bytecode, but the inclusion of
bytecode blobs in the distribution. If there are handwritten pieces
that must remain as blobs then I would hope they are commented.



On Fri, Jul 28, 2017 at 3:57 AM, Steve Mynott  wrote:
>>> On 27 July 2017 at 09:13, R0b0t1  wrote:
>
>> Of course there is still the problem of communicating the release keys
>> to someone in the first place, but if the release key is on a public
>> keyserver and its ID is referenced on the project site somewhere that
>> typically works well enough.
>
> The main problem is management of the *private keys* and passphrases.
> You shouldn't keep private keys on shared servers but secured personal
> laptops, which are exactly the sort of systems which suffer data lose
> due to a lack of secure backups.
>

If you have no interest in security then you really have no business
developing for a large, widely consumed project and are doing all of
your users a disservice.

Despite the risk of sounding paranoid, I think it is very important
that I point out the likely target audiences of Perl 6: developers
with expensive workstations, researchers using high performance
computing clusters, or web developers with very capable servers that
have high bandwidth 

Re: Verifiable Releases/The Build System is Ridiculous

[Steve Mynott]

On 28 July 2017 at 11:00, Timo Paulssen  wrote:
> On 28/07/17 10:57, Steve Mynott wrote:
>> Yes we ship binary blobs to bootstrap. I spent a day trying to
>> reproduce the current binary blobs and it's not possible. Well maybe
>> it is possible if you reproduce the exact directory (Windows)
>> directory structure of the original build system and spoof the system
>> clock but its an incredibly difficult process and quite frankly a
>> waste of thing right now since there are more pressing issues like bug
>> fixing, speed increases and wider adoption of perl 6.
>
> It looks like the only thing you need to fake here is the build
> directory? I got the exact same files doing "make m-bootstrap-files"
> twice with a pause of about a minute in between. Maybe we need one last
> bootstrap files update to get that behaviour for the future.

This was some time ago and I was probably mistaken about the time and
thinking of GCC reproducible builds which use libfaketime  (some good
recent work done by Debian).

A quick strings on stage0 stuff in NQP shows maybe all that's needed
now is to use a
"/home/jnthn/dev/MoarVM" directory.

At least the slashes lean the right way now as opposed to last time I looked :-)


-- 
4096R/EA75174B Steve Mynott 

Re: Verifiable Releases/The Build System is Ridiculous

[Steve Mynott]

On 28 July 2017 at 11:00, Timo Paulssen  wrote:
> On 28/07/17 10:57, Steve Mynott wrote:
>> Yes we ship binary blobs to bootstrap. I spent a day trying to
>> reproduce the current binary blobs and it's not possible. Well maybe
>> it is possible if you reproduce the exact directory (Windows)
>> directory structure of the original build system and spoof the system
>> clock but its an incredibly difficult process and quite frankly a
>> waste of thing right now since there are more pressing issues like bug
>> fixing, speed increases and wider adoption of perl 6.
>
> It looks like the only thing you need to fake here is the build
> directory? I got the exact same files doing "make m-bootstrap-files"
> twice with a pause of about a minute in between. Maybe we need one last
> bootstrap files update to get that behaviour for the future.

This was some time ago and I was probably mistaken about the time and
thinking of GCC reproducible builds which use libfaketime  (some good
recent work done by Debian).

A quick strings on stage0 stuff in NQP shows maybe all that's needed
now is to use a
"/home/jnthn/dev/MoarVM" directory. At least the slashes lean the
right way now as opposed to last time I looked :-)




-- 
4096R/EA75174B Steve Mynott 

Re: Verifiable Releases/The Build System is Ridiculous

[Aleks-Daniel Jakimenko-Aleksejev]

> If your main interest is security then
> you might be better off with a project like OpenBSD and helping update
> their Rakudo ports

Uh… this sounds like we are not welcoming contributions, which is not the
case. As clearly pointed out in this thread, there are things that can be
improved (even if they bring too little benefit, still). If somebody agrees
to put some work into it, then why not?

Writing email rants may help, but it's unlikely. I'd love to see a list of
things that should be improved (e.g. on rakudo wiki, or maybe a couple of
tickets on RT), and proposed solutions for each item. Then perhaps we can
start moving into some direction, instead of writing rants here and then
replying with rants on how rants are bad. Even if we reject some of these
proposals, we'll then have the reasoning for it stored in a structured way
instead of having it buried in this horrible thread.

TL;DR complain in a useful way by creating tickets or writing plans on how
to improve things

On Fri, Jul 28, 2017 at 11:57 AM, Steve Mynott 
wrote:

> >> On 27 July 2017 at 09:13, R0b0t1  wrote:
>
> > Of course there is still the problem of communicating the release keys
> > to someone in the first place, but if the release key is on a public
> > keyserver and its ID is referenced on the project site somewhere that
> > typically works well enough.
>
> The main problem is management of the *private keys* and passphrases.
> You shouldn't keep private keys on shared servers but secured personal
> laptops, which are exactly the sort of systems which suffer data lose
> due to a lack of secure backups.
>
> It's hard enough to create a Rakudo Star release with the present
> process (as you yourself have shown) and putting more obstacles in the
> way of people to do this by requiring keys isn't progress.  I'd rather
> see an easier, simpler more robust process.  I'd rather see more
> people forking the repo and creating their own distros rather than
> rubberstamping an official release.  Centralised, authorised releases
> isn't really in the Perl spirit.
>
> We can sign tags but if we aren't creating new tags like 2017.07.1 for
> NQP and are releasing 2017.07 with known issues this really isn't good
> enough.  I'd rather use working code which isn't signed than broken
> code that is.
>
> Signing isn't a Magic Security Bullet and coming to Open Source
> projects saying "Sign All The Things!" isn't really helpful.
>
> > What I want is to verify the code I run before I run it. From my
> > position the easiest way to do this was to try to grab code from Git.
> > The star repository was the only one that looked like it had
> > everything in one place and a way to use those things.
> >
> >>> Are there any signed releases, or do I have to do the equivalent of
> curl|sudo?
> >>
> >> Extracting a tarball and manually running scripts isn't the same as
> >> running curl|sudo since you have the chance to read what the scripts
> >> do before running them.
> >>
> >> There is one subsystem where this isn't the case and its left as an
> >> exercise for the reader :-)
> >>
> >
> > NQP? I was told that has bytecode in it. If possible I would request
> > that this is changed in the future.
>
> Yes we ship binary blobs to bootstrap. I spent a day trying to
> reproduce the current binary blobs and it's not possible. Well maybe
> it is possible if you reproduce the exact directory (Windows)
> directory structure of the original build system and spoof the system
> clock but its an incredibly difficult process and quite frankly a
> waste of thing right now since there are more pressing issues like bug
> fixing, speed increases and wider adoption of perl 6.
>
> >> The whole rakudo (and star) build process doesn't fit in well with any
> >> third party systems (gentoo, cmake whatever).  Having Perl 5 as a
> >> dependency for Perl 6 seems to be more reasonable than using anything
> >> else.
> >>
> >
> > Why are those things unsuitable? I have used them enough to know that
> > they can not solve every problem, but it has eventually become clear
> > to me that it is almost always easiest to try to add whatever
> > functionality is necessary to an already existing build system than
> > trying to manage it myself.
>
> Yes and our existing build system is the one we have now. The way to
> fix it is gradually by submitting small fixes not talking about
> replacing it by your pet build system.
>
> > Please understand that even should I have the time to contribute, I
> > still have to read and understand the project. My message is more a
> > question of why the build system is the way it is.
>
> MoarVM is used to build NQP (a subset of Perl 6) which is used to
> build Rakudo (Perl 6)
>
> These sorts of technical issue are not really explainable in emails or
> IRC. You actually have to use it to understand it. Start with building
> R* from a tarball and then try using the github version to build the
> tarball. It's probably 

Re: Verifiable Releases/The Build System is Ridiculous

[Steve Mynott]

>> On 27 July 2017 at 09:13, R0b0t1  wrote:

> Of course there is still the problem of communicating the release keys
> to someone in the first place, but if the release key is on a public
> keyserver and its ID is referenced on the project site somewhere that
> typically works well enough.

The main problem is management of the *private keys* and passphrases.
You shouldn't keep private keys on shared servers but secured personal
laptops, which are exactly the sort of systems which suffer data lose
due to a lack of secure backups.

It's hard enough to create a Rakudo Star release with the present
process (as you yourself have shown) and putting more obstacles in the
way of people to do this by requiring keys isn't progress.  I'd rather
see an easier, simpler more robust process.  I'd rather see more
people forking the repo and creating their own distros rather than
rubberstamping an official release.  Centralised, authorised releases
isn't really in the Perl spirit.

We can sign tags but if we aren't creating new tags like 2017.07.1 for
NQP and are releasing 2017.07 with known issues this really isn't good
enough.  I'd rather use working code which isn't signed than broken
code that is.

Signing isn't a Magic Security Bullet and coming to Open Source
projects saying "Sign All The Things!" isn't really helpful.

> What I want is to verify the code I run before I run it. From my
> position the easiest way to do this was to try to grab code from Git.
> The star repository was the only one that looked like it had
> everything in one place and a way to use those things.
>
>>> Are there any signed releases, or do I have to do the equivalent of 
>>> curl|sudo?
>>
>> Extracting a tarball and manually running scripts isn't the same as
>> running curl|sudo since you have the chance to read what the scripts
>> do before running them.
>>
>> There is one subsystem where this isn't the case and its left as an
>> exercise for the reader :-)
>>
>
> NQP? I was told that has bytecode in it. If possible I would request
> that this is changed in the future.

Yes we ship binary blobs to bootstrap. I spent a day trying to
reproduce the current binary blobs and it's not possible. Well maybe
it is possible if you reproduce the exact directory (Windows)
directory structure of the original build system and spoof the system
clock but its an incredibly difficult process and quite frankly a
waste of thing right now since there are more pressing issues like bug
fixing, speed increases and wider adoption of perl 6.

>> The whole rakudo (and star) build process doesn't fit in well with any
>> third party systems (gentoo, cmake whatever).  Having Perl 5 as a
>> dependency for Perl 6 seems to be more reasonable than using anything
>> else.
>>
>
> Why are those things unsuitable? I have used them enough to know that
> they can not solve every problem, but it has eventually become clear
> to me that it is almost always easiest to try to add whatever
> functionality is necessary to an already existing build system than
> trying to manage it myself.

Yes and our existing build system is the one we have now. The way to
fix it is gradually by submitting small fixes not talking about
replacing it by your pet build system.

> Please understand that even should I have the time to contribute, I
> still have to read and understand the project. My message is more a
> question of why the build system is the way it is.

MoarVM is used to build NQP (a subset of Perl 6) which is used to
build Rakudo (Perl 6)

These sorts of technical issue are not really explainable in emails or
IRC. You actually have to use it to understand it. Start with building
R* from a tarball and then try using the github version to build the
tarball. It's probably easier to understand if you start with 2017.01
for example since the last two versions used awful hacks with NQP and
MoarVM.  You are probably safest using Debian stable (or oldstable).
It will be probably hard and frustrating and not easy. You will
probably end up spending more time on it than you thought. You at
least will end up with more knowledge of the sort of things you are
asking people to spend their free time on.

Security is desirable but not currently a main objective of this
project which is more about creating a reasonably fast and very
expressive computer language. If your main interest is security then
you might be better off with a project like OpenBSD and helping update
their Rakudo ports. MoarVM has problems with their W^X system if you
run certain of the roast tests which needs investigation.

Re: Verifiable Releases/The Build System is Ridiculous

[Joachim Durchholz]

Am 28.07.2017 um 04:35 schrieb R0b0t1:

The earliest versions of the Rakudo Star build system started out by trying to 
use Git submodules to manage packages, but it quickly proved to be unwieldy and 
almost impossible to understand and maintain.  Perhaps the submodule ecosystem 
has changed since then, though.


Can you give an example of how submodules were insufficient?


I don't know what was unwieldy for the Perl6 guys, but having to manage 
multiple repositories for a given task is always some extra steps when 
synchronizing new code to the public repositories. If one of those steps 
is forgotten, everybody will see repositories that won't work, or show 
weird problems.


Submodules are built for the use case that repositories evolve 
independently of each other, subtrees for the case that they evolve in 
sync. It's possible that submodules were the wrong approach, or that 
subtrees didn't work well enough at the point in time, or that nobody 
found the time to set everything up well enough to make it really work, 
or for lack of knowledge how to get submodules to work well.
Since everybody's time is constrained, and Perl6 is still a work in 
progress, there is a long list of things that could be improved, so it's 
no surprise to see defects. The more important question is whether 
defects are important.



Most issues I have seen that arise with submodules come from people
trying to treat the submodule directory in a way that is different
than other objects tracked by Git. If you treat it like a source file
you're tracking most problems should disappear, at least in theory.


Dunno what the problems were for Perl6.
Versioning generated (non-source) files can indeed create problems, but 
that's independently of whether it's submodule or not, so don't know 
what problem you're seeing here.



There's still some unfortunate submodule command names.


git's ergonomics is generally not that good. Projects still stick with 
it, I have several theories about the reasons but don't know which of 
them apply.



NQP? I was told that has bytecode in it. If possible I would request
that this is changed in the future.


NQP is just the language that large parts of the Perl6 compiler are 
written in.


Bytecode is what the AST is compiled to. One could write a different 
backend, translating either AST or bytecode down to machine code; it's 
just that nobody did it yet. It would be a pretty large project, so I 
don't count on that happening anytime soon (but then the Perl community 
can be pretty amazing and incredible things happen on a semi-regular 
basis, so I'm not counting on that *not* happening either).

Re: Verifiable Releases/The Build System is Ridiculous

[R0b0t1]

I apologize if there was any tone to my post, the only place I
intended there to be any was when I pointed out the
--no-check-certificates flag. Please understand Perl 6 is the second
or third language I have found that does not secure its ecosystem. The
number of non-language projects which suffer from this problem is in
the double digits.

Despite the fact I know what Perl 6 is and have found its mailing
list, I am not a smart man. I have merely watched myself break a great
number of things and watched other people fix them.

On Thu, Jul 27, 2017 at 3:51 AM, Patrick R. Michaud  wrote:
> The details of how to use the star GitHub repository are in 
> tools/star/release_guide.pod .
>
> You're entirely welcome to create a bundling that has a better build system 
> than what Rakudo Star uses -- indeed, Rakudo Star has always been intended to 
> be just one of many possible bundlings of Rakudo and useful modules.  Even if 
> you just create a repo to demonstrate a better way of organizing modules, 
> that'd be helpful.
>
> The earliest versions of the Rakudo Star build system started out by trying 
> to use Git submodules to manage packages, but it quickly proved to be 
> unwieldy and almost impossible to understand and maintain.  Perhaps the 
> submodule ecosystem has changed since then, though.

Can you give an example of how submodules were insufficient? I will
take whatever you have to add at face value, I'm just interested in
some concrete example because I have seen plenty of things done for
very diffuse and nebulous reasons.

Most issues I have seen that arise with submodules come from people
trying to treat the submodule directory in a way that is different
than other objects tracked by Git. If you treat it like a source file
you're tracking most problems should disappear, at least in theory.
There's still some unfortunate submodule command names.


On Thu, Jul 27, 2017 at 4:22 AM, Steve Mynott  wrote:
> On 27 July 2017 at 09:13, R0b0t1  wrote:
>> Are there any releases signed by the developers? The official releases
>> located at https://rakudo.perl6.org/downloads/star/ do not seem to
>> have signatures available.
>
> Can you be more specific about the technologies you suggest and the
> exact problems you are trying to solve?
>
> Maybe you mean something like tarballs signed with GPG? Or SHA256
> sums? In both cases there is still an issue with distributing
> signatures and the checksum files.
>
> I could upload a checksum file but since this would be in the same
> directory as the tarball and both could be manipulated by a potential
> attacker there seems little point.
>
> Ultimately you are able to (mostly) read the files in the source
> tarballs and audit for backdoors yourself (if this is your threat
> model).
>

The solution is to either give a GPG signature of the release tarball,
or, because signing and verifying the signatures of large files can
take a long time, summing the file in multiple ways and then signing
the list of digests. Example:
http://distfiles.gentoo.org/releases/amd64/autobuilds/current-stage3-amd64/,
see release.DIGESTS.asc.

Of course there is still the problem of communicating the release keys
to someone in the first place, but if the release key is on a public
keyserver and its ID is referenced on the project site somewhere that
typically works well enough.

>> As a stopgap measure I attempted to clone
>> https://github.com/rakudo/star and run Configure.pl. I felt it strange
>> that it warned me against using the code in the repository as cloned
>> from the repository (why does the repository exist?) but continued
>
> The repository exists to create the distributed source tarball itself.
> You need to read the documentation starting with release-guide.pod and
> understand it.
>
> This is really a prerequisite for any useful criticism of the build process.
>
> Yes it does look strange and needs improvement but there are reasons
> for it and most of it is written in easily understandable perl 5.
>
>> with --force, only to find out that some of the folders in the
>> directory structure need to be initialized. A mailing list posting was
>> found detailing how to continue
>> (http://www.nntp.perl.org/group/perl.perl6.compiler/2013/05/msg8915.html)
>> and the steps were followed. Unfortunately, the nqp package download
>> failed with error 404, which made me see the following command was
>> part of the preparation script:
>>
>>>wget --no-check-certificate 
>>>https://rakudo.perl6.org/downloads/nqp/nqp-2017.07-9-gc0abee7.tar.gz -O 
>>>src/nqp-2017.07-9-gc0abee7.tar.gz
>>
>> I have no words. Can anyone justify this choice?
>
> Odd since there seemed to be enough words in your email! Really I
> don't have to justify anything to you but spreading knowledge of how
> this works is generally useful.
>
> The reason the released NQP package wasn't shipped was because there
> was a bug in the released one. This is documented 

Re: Verifiable Releases/The Build System is Ridiculous

[zoffix]

Quoting R0b0t1 :


Are there any releases signed by the developers? The official releases
located at https://rakudo.perl6.org/downloads/star/ do not seem to
have signatures available.


There's Rakudo the compiler and Rakudo Star distribution
which is the compiler + docs + some modules.

Yes, the compiler releases *are* signed. You can get the signatures off
GitHub: https://github.com/rakudo/rakudo/tags or the compiler directory
on the downloads page: https://rakudo.perl6.org/downloads/rakudo/

Rakudo Star release guide does advise to sign and it's unfortunate  
it's not being
done with the explanation of it being too much effort. It's just two  
commands to run,

which can be automated, like it was done for the compiler releases.


wget --no-check-certificate


I have no words. Can anyone justify this choice?


No, that's awful and should be removed.


[…] should actually be done […] I would suggest […] the project needs
[…] There are various articles […] they are slightly more complicated to use.


Keep in mind a large portion of this project is maintained by  
volunteers, using their
spare time and whatever knowledge they possess. Jumping in with a  
mountain of criticisms
to tear our volunteers a new asshole isn't very helpful, I think, as  
the replies sent

to you so far indicate.

There was talk of making a leaner distribution: Rakudo compiler + zef  
module installer.
Perhaps, you could start on that effort and do it the Right Way and  
once that's working well,

Rakudo Star can be similarly improved.

Cheers,
ZZ

Re: Verifiable Releases/The Build System is Ridiculous

[Steve Mynott]

On 27 July 2017 at 09:13, R0b0t1  wrote:
> Are there any releases signed by the developers? The official releases
> located at https://rakudo.perl6.org/downloads/star/ do not seem to
> have signatures available.

Can you be more specific about the technologies you suggest and the
exact problems you are trying to solve?

Maybe you mean something like tarballs signed with GPG? Or SHA256
sums? In both cases there is still an issue with distributing
signatures and the checksum files.

I could upload a checksum file but since this would be in the same
directory as the tarball and both could be manipulated by a potential
attacker there seems little point.

Ultimately you are able to (mostly) read the files in the source
tarballs and audit for backdoors yourself (if this is your threat
model).

> As a stopgap measure I attempted to clone
> https://github.com/rakudo/star and run Configure.pl. I felt it strange
> that it warned me against using the code in the repository as cloned
> from the repository (why does the repository exist?) but continued

The repository exists to create the distributed source tarball itself.
You need to read the documentation starting with release-guide.pod and
understand it.

This is really a prerequisite for any useful criticism of the build process.

Yes it does look strange and needs improvement but there are reasons
for it and most of it is written in easily understandable perl 5.

> with --force, only to find out that some of the folders in the
> directory structure need to be initialized. A mailing list posting was
> found detailing how to continue
> (http://www.nntp.perl.org/group/perl.perl6.compiler/2013/05/msg8915.html)
> and the steps were followed. Unfortunately, the nqp package download
> failed with error 404, which made me see the following command was
> part of the preparation script:
>
>>wget --no-check-certificate 
>>https://rakudo.perl6.org/downloads/nqp/nqp-2017.07-9-gc0abee7.tar.gz -O 
>>src/nqp-2017.07-9-gc0abee7.tar.gz
>
> I have no words. Can anyone justify this choice?

Odd since there seemed to be enough words in your email! Really I
don't have to justify anything to you but spreading knowledge of how
this works is generally useful.

The reason the released NQP package wasn't shipped was because there
was a bug in the released one. This is documented in
"lastmin-fixes.txt". This process isn't ideal as this file admits.
This is only the second time the release tarballs weren't used.

The NQP tarball you want is in the actual Rakudo Star tarball not the
git repo for the build process.  In hindsight I probably could have
pointed it straight at a github versioned zip (although this would
have added unzip as a dependency!) or just not bothered and shipped
with a broken -ll-exception.

> Are there any signed releases, or do I have to do the equivalent of curl|sudo?

Extracting a tarball and manually running scripts isn't the same as
running curl|sudo since you have the chance to read what the scripts
do before running them.

There is one subsystem where this isn't the case and its left as an
exercise for the reader :-)

> What is being done should actually be done with Git submodules, or the
> more recent feature, subtrees. There is no reason for the releases to
> be prepared in this way. What has been created is a very inflexible
> package management system. If what you want is a package management
> system, then I would suggest adapting Gentoo's portage to the
> project's needs, as KDE seems to have done with their Craft build
> system. However what the project needs is version control that will
> handle dependencies.

The Rakudo Star system does use git submodules for the module dependencies.

Maybe submodules in R* should be used for the Rakudo, NQP and MoarVM
dependencies -- normally based on a tagged release but allowing a
version a few commits later for bug fixes (which also should be tagged
as .1).  This isn't a bad idea.

The problem with this is that the monthly Rakudo (not Star) tarballs
still wouldn't get these fixes (and it's quite a lot of work for
Rakudo upstream to produce potentially three extra tarballs for each
patch) so most of the third party packaging (which doesn't tend to use
Star) wouldn't get the fixes anyway.  Still any extra tagging of bug
fixes as .1 would help. I still think we need a -STABLE type branch as
well.

The whole rakudo (and star) build process doesn't fit in well with any
third party systems (gentoo, cmake whatever).  Having Perl 5 as a
dependency for Perl 6 seems to be more reasonable than using anything
else.

It's open source and if you feel strongly enough about this to
actually do the work you can fork the repo and make changes in your
own branch. We would actively encourage contributions and it's easier
to evaluate actual code changes than emailed suggestions.

But be aware talk is cheap and code costs time.

-- 
4096R/EA75174B Steve Mynott 

Re: Verifiable Releases/The Build System is Ridiculous

[Patrick R. Michaud]

The details of how to use the star GitHub repository are in 
tools/star/release_guide.pod .

You're entirely welcome to create a bundling that has a better build system 
than what Rakudo Star uses -- indeed, Rakudo Star has always been intended to 
be just one of many possible bundlings of Rakudo and useful modules.  Even if 
you just create a repo to demonstrate a better way of organizing modules, 
that'd be helpful.

The earliest versions of the Rakudo Star build system started out by trying to 
use Git submodules to manage packages, but it quickly proved to be unwieldy and 
almost impossible to understand and maintain.  Perhaps the submodule ecosystem 
has changed since then, though.

Pm


On Thu, Jul 27, 2017 at 03:13:19AM -0500, R0b0t1 wrote:
> Are there any releases signed by the developers? The official releases
> located at https://rakudo.perl6.org/downloads/star/ do not seem to
> have signatures available.
> 
> As a stopgap measure I attempted to clone
> https://github.com/rakudo/star and run Configure.pl. I felt it strange
> that it warned me against using the code in the repository as cloned
> from the repository (why does the repository exist?) but continued
> with --force, only to find out that some of the folders in the
> directory structure need to be initialized. A mailing list posting was
> found detailing how to continue
> (http://www.nntp.perl.org/group/perl.perl6.compiler/2013/05/msg8915.html)
> and the steps were followed. Unfortunately, the nqp package download
> failed with error 404, which made me see the following command was
> part of the preparation script:
> 
> >wget --no-check-certificate 
> >https://rakudo.perl6.org/downloads/nqp/nqp-2017.07-9-gc0abee7.tar.gz -O 
> >src/nqp-2017.07-9-gc0abee7.tar.gz
> 
> I have no words. Can anyone justify this choice?
> 
> Are there any signed releases, or do I have to do the equivalent of curl|sudo?
> 
> 
> What is being done should actually be done with Git submodules, or the
> more recent feature, subtrees. There is no reason for the releases to
> be prepared in this way. What has been created is a very inflexible
> package management system. If what you want is a package management
> system, then I would suggest adapting Gentoo's portage to the
> project's needs, as KDE seems to have done with their Craft build
> system. However what the project needs is version control that will
> handle dependencies.
> 
> Git's submodules fit perfectly because they pin a directory to a
> branch which is exactly what is being done here, but it is integrated
> with Git and can use existing authentication and verification
> mechanisms (HTTPS, SSH, GPG). There are various articles which try to
> paint submodules as unsuitable for most uses but I hope it is easy to
> see that what they do is exactly what is desired in this instance.
> 
> If submodules do not seem suitable, there are subtrees, but these blur
> the lines between projects. That might be okay, but they are slightly
> more complicated to use.
> 
> R0b0t1.