Re: Rust as build requirement was Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Cameron Kaiser]

Replying to a couple messages at once.

On 12/14/15 2:19 PM, Mike Hoye wrote:

On 2015-12-14 4:06 PM, Justin Dolske wrote:

On 12/14/15 2:51 AM, Ted Mielczarek wrote:


[...]Obviously this isn't something we like to see, but
we shouldn't let the support of non-Tier 1 platforms guide our decision
making to that extent. Enabling Rust components in Gecko is important
work, and outweighs the value of supporting Firefox on minority
platforms.


+1. We shouldn't be doing any work to maintain Tier-3 platforms, nor
should they hold us back from modernizing and securing the platforms
used by the overwhelming majority of our users.


While I agree with this, giving those tier-3 platform maintainers as
much advanced notice as possible and good-faith-if-low-touch guidance as
to how they can keep rolling would be a fine thing.


It would be nice, at least. Obviously I'm closer to this than other 
people on this thread, and maybe this reflects the paucity of platforms 
today compared with, say, 15 years ago, but the attitude towards tier-3 
has degenerated a little beyond benign neglect lately. I don't think 
Justin is wrong for his position, but implicit in saying they aren't 
worth any effort is also saying they aren't worth maintaining a good 
relationship with or serve a useful purpose. The roughly 25,000 people 
who regularly use TenFourFox, a stable number confirmed by update 
checkins and download stats, would probably disagree. That's a rounding 
error in the Firefox user stats but they're loyal all the same.


Candidly, I'd like to see a little more recognition of the work it takes 
to keep a minority port alive, even if that shouldn't necessarily 
translate into material assistance, and I don't see much of this from 
many Firefox developers lately. If for no other reason than portability 
and diversity, we do matter to the ecosystem. I'm just asking for mutual 
understanding, since the position of MoFo is glaringly clear to us and I 
think we'd all agree it's not unreasonable even if it is unfortunate.





I see a few names with no mailing addresses for a number of people on
the supported build configurations page, so I'm going to see if I can
find them and ask them to add their contact information.


I posted a heads-up to the OS/2 port maintainer, since I have some 
contacts there.


On 12/14/15 2:22 PM, Nathan Froyd wrote:
> On Sat, Dec 12, 2015 at 5:17 PM, Cameron Kaiser 
> wrote:
>
>> This would essentially mandate, then, that Gecko can only be built on
>> platforms with a Rust toolchain. That may be desirable, but it would
>> probably bust some of the obscure Tier-3 platforms and it would 
definitely

>> bust TenFourFox (we can't even get clang to be happy on 10.4 currently).
>> Not that we haven't been on borrowed time for awhile; I just point 
it out

>> for the record. 
>>
>
> FWIW, we'd also like to make Gecko require a C++11-compliant standard
> library, and that will also have roughly the same effect as requiring
> Rust.  I don't have exact dates on that happening, but I'd really like to
> see it happen in the first half of 2016.

I don't think these are really in the same category. All of the extant 
Tier-3 ports compile with gcc too, and keeping gcc up (assuming gcc 
itself doesn't obsolete the platform) should bring along a C++11 
libstdc++ along with it. I'm using gcc 4.8 now on OS X/ppc, and 
Tigerbrew even already has 5.2.


Rust, on the other hand:

On 12/14/15 5:55 AM, Henri Sivonen wrote:
> As for tier-3, a quick search indicates that rustc has already been
> bootstrapped on (x86ish?) FreeBSD and OpenBSD. Wikipedia says that
> LLVM supports MIPS, PowerPC, SPARC and Z/Architecture. It looks like
> Debian has dropped 68K. So it should be feasible for *BSD and
> non-mainstream CPU arch versions of Debian to come along. Maybe there
> is rustc porting work to be done, but I think people who want these
> platforms to be supported internalize the cost of supporting them
> instead of expecting Gecko to stop progressing because of them.

Independent of the policy decision, "rustc porting work" is not trivial, 
since none of these arches has an existing rustc (only x86 and ARM32), 
even on Linux, and none of the Tier-3 Rusts have cargo support.[1] I 
imagine this would require either a cross-compiling rust or building the 
stage0 from scratch.


[1] https://doc.rust-lang.org/book/installing-rust.html#tier-3

In any case,

> Firefox 45 will be ESR. I think it's reasonable to make Rust a Gecko
> build requirement after 45 and let tier-3 platforms use the ESR cycle
> to get rustc/LLVM up and running where it's not up and running
> already.

Let me be clear that I don't find this to be unreasonable and I would be 
willing to work within this timeframe, though I think we're going to 
have to fork anyway for other reasons. I'm still going to do some 
feasibility exploration, though.


That said, I kind of object to the fact that no 

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Ted Mielczarek]

On Sun, Dec 13, 2015, at 09:27 PM, Nicholas Nethercote wrote:
> On Sun, Dec 13, 2015 at 11:28 AM, Bobby Holley 
> wrote:
> >>
> >> I've been wondering about this. There's a big difference between (a)
> >> permitting Rust components (while still allowing fallback C++
> >> equivalents) and (b) mandating Rust components.
> >
> > I don't know why we would allow there to be a long gap between (a) and (b).
> > Maintaining/supporting two sets of the same code is costly. So if we get the
> > rust code working and shipping on all platforms, I can't think of a reason
> > why we wouldn't move as quickly as possible to requiring it.
> 
> The "if" in your second sentence is exactly what I'm worried about. My
> gut tells me that step (b) is a *lot* harder than step (a). I could be
> too pessimistic, but Android and the tier 3 platforms worry me.

The Rust team has been very supportive of meeting the needs that we (the
build folks on behalf of Gecko) have stated as requirements for enabling
Rust code everywhere. I'm quite confident that the Rust compiler already
supports targeting all of our Tier-1 platforms, it's just a matter of
getting things wired up in our production build environments.

We will definitely hit a point where we want to make Rust a hard
requirement for builds. This will likely cause some existing platforms
to no longer build. Obviously this isn't something we like to see, but
we shouldn't let the support of non-Tier 1 platforms guide our decision
making to that extent. Enabling Rust components in Gecko is important
work, and outweighs the value of supporting Firefox on minority
platforms. (Incidentally, the Rust compiler has been ported to other
platforms by community members, so this is not entirely out of the
question.)

-Ted
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rust as build requirement was Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Gregory Szorc]

On Mon, Dec 14, 2015 at 6:58 PM, Cameron Kaiser 
wrote:

> Replying to a couple messages at once.
>
> On 12/14/15 2:19 PM, Mike Hoye wrote:
>
>> On 2015-12-14 4:06 PM, Justin Dolske wrote:
>>
>>> On 12/14/15 2:51 AM, Ted Mielczarek wrote:
>>>
>>> [...]Obviously this isn't something we like to see, but
 we shouldn't let the support of non-Tier 1 platforms guide our decision
 making to that extent. Enabling Rust components in Gecko is important
 work, and outweighs the value of supporting Firefox on minority
 platforms.

>>>
>>> +1. We shouldn't be doing any work to maintain Tier-3 platforms, nor
>>> should they hold us back from modernizing and securing the platforms
>>> used by the overwhelming majority of our users.
>>>
>>
>> While I agree with this, giving those tier-3 platform maintainers as
>> much advanced notice as possible and good-faith-if-low-touch guidance as
>> to how they can keep rolling would be a fine thing.
>>
>
> It would be nice, at least. Obviously I'm closer to this than other people
> on this thread, and maybe this reflects the paucity of platforms today
> compared with, say, 15 years ago, but the attitude towards tier-3 has
> degenerated a little beyond benign neglect lately. I don't think Justin is
> wrong for his position, but implicit in saying they aren't worth any effort
> is also saying they aren't worth maintaining a good relationship with or
> serve a useful purpose. The roughly 25,000 people who regularly use
> TenFourFox, a stable number confirmed by update checkins and download
> stats, would probably disagree. That's a rounding error in the Firefox user
> stats but they're loyal all the same.
>
> Candidly, I'd like to see a little more recognition of the work it takes
> to keep a minority port alive, even if that shouldn't necessarily translate
> into material assistance, and I don't see much of this from many Firefox
> developers lately. If for no other reason than portability and diversity,
> we do matter to the ecosystem. I'm just asking for mutual understanding,
> since the position of MoFo is glaringly clear to us and I think we'd all
> agree it's not unreasonable even if it is unfortunate.
>
> 
>
> I see a few names with no mailing addresses for a number of people on
>> the supported build configurations page, so I'm going to see if I can
>> find them and ask them to add their contact information.
>>
>
> I posted a heads-up to the OS/2 port maintainer, since I have some
> contacts there.
>
> On 12/14/15 2:22 PM, Nathan Froyd wrote:
> > On Sat, Dec 12, 2015 at 5:17 PM, Cameron Kaiser 
> > wrote:
> >
> >> This would essentially mandate, then, that Gecko can only be built on
> >> platforms with a Rust toolchain. That may be desirable, but it would
> >> probably bust some of the obscure Tier-3 platforms and it would
> definitely
> >> bust TenFourFox (we can't even get clang to be happy on 10.4 currently).
> >> Not that we haven't been on borrowed time for awhile; I just point it
> out
> >> for the record. 
> >>
> >
> > FWIW, we'd also like to make Gecko require a C++11-compliant standard
> > library, and that will also have roughly the same effect as requiring
> > Rust.  I don't have exact dates on that happening, but I'd really like to
> > see it happen in the first half of 2016.
>
> I don't think these are really in the same category. All of the extant
> Tier-3 ports compile with gcc too, and keeping gcc up (assuming gcc itself
> doesn't obsolete the platform) should bring along a C++11 libstdc++ along
> with it. I'm using gcc 4.8 now on OS X/ppc, and Tigerbrew even already has
> 5.2.
>
> Rust, on the other hand:
>
> On 12/14/15 5:55 AM, Henri Sivonen wrote:
> > As for tier-3, a quick search indicates that rustc has already been
> > bootstrapped on (x86ish?) FreeBSD and OpenBSD. Wikipedia says that
> > LLVM supports MIPS, PowerPC, SPARC and Z/Architecture. It looks like
> > Debian has dropped 68K. So it should be feasible for *BSD and
> > non-mainstream CPU arch versions of Debian to come along. Maybe there
> > is rustc porting work to be done, but I think people who want these
> > platforms to be supported internalize the cost of supporting them
> > instead of expecting Gecko to stop progressing because of them.
>
> Independent of the policy decision, "rustc porting work" is not trivial,
> since none of these arches has an existing rustc (only x86 and ARM32), even
> on Linux, and none of the Tier-3 Rusts have cargo support.[1] I imagine
> this would require either a cross-compiling rust or building the stage0
> from scratch.
>
> [1] https://doc.rust-lang.org/book/installing-rust.html#tier-3
>
> In any case,
>
> > Firefox 45 will be ESR. I think it's reasonable to make Rust a Gecko
> > build requirement after 45 and let tier-3 platforms use the ESR cycle
> > to get rustc/LLVM up and running where it's not up and running
> > already.
>
> Let me be clear that 

Re: Rust as build requirement was Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[David Rajchenbach-Teller]

Well, this wouldn't be the first time that we are not as clear
communicating our intents as we thought or should. Sorry about that.

Thanks for getting in touch with the OS/2 maintainer. We definitely need
to spread the word asap. I hope that Rust can be ported to OS/2 and
other Tier-3 platforms but there is no guarantee and that will
definitely take time and commitment. On the upside, Rust is a great
language, so any new port will be good news for the target platform.

Cheers,
 David

On 15/12/15 05:00, Cameron Kaiser wrote:
> That's not what I said. Even though I politely contest your assertion
> that it was well-noticed -- I even searched and found a grand total of
> *two* threads directly related to Rust in this newsgroup prior to this
> thread, and one was about Servo -- no one had said anything about Rust
> being *demanded* by Gecko, period. If those discussions happened
> somewhere else, then there should have been some acknowledgement here,
> since after all this newsgroup does ostensibly deal with the platform.
> 
> Even though I can't argue it's unreasonable to allege the presence of
> Rust code should imply that one day it might be obligatory, no one
> seemed to realise (or be interested) that making it obligatory could be
> a portbreaker, and that's the part that stinks.
> 
> Cameron Kaiser
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rust as build requirement was Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Cameron Kaiser]

On 12/14/15 7:36 PM, Gregory Szorc wrote:

On Mon, Dec 14, 2015 at 6:58 PM, Cameron Kaiser 
wrote:

That said, I kind of object to the fact that no one brought this up until
I noticed it in passing, and the work to get Rust up on a tier-3 platform
-- a language that currently has no relevance to those platforms other than
this purpose -- is certainly more than it is to keep the compiler
maintained, which I don't think is acknowledged. Everyone expects Servo to
demand Rust, but there wasn't really any warning about Gecko doing so.


There have been serious discussions about adding Rust components to Gecko
for well over a year. Basic Rust support has been in mozilla-central since
May (https://hg.mozilla.org/mozilla-central/rev/b811c7d4f39b) - about the
same time Rust 1.0 (stable) was released. By the time we ship a Rust
component in Firefox, Rust stable and basic build system support will be
over 1 year old. To say there hasn't been any warning about shipping Rust
in Gecko just isn't true.


That's not what I said. Even though I politely contest your assertion 
that it was well-noticed -- I even searched and found a grand total of 
*two* threads directly related to Rust in this newsgroup prior to this 
thread, and one was about Servo -- no one had said anything about Rust 
being *demanded* by Gecko, period. If those discussions happened 
somewhere else, then there should have been some acknowledgement here, 
since after all this newsgroup does ostensibly deal with the platform.


Even though I can't argue it's unreasonable to allege the presence of 
Rust code should imply that one day it might be obligatory, no one 
seemed to realise (or be interested) that making it obligatory could be 
a portbreaker, and that's the part that stinks.


Cameron Kaiser
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Henri Sivonen]

On Sun, Dec 13, 2015 at 10:50 AM, Nicholas Nethercote
 wrote:
> On Sun, Dec 13, 2015 at 9:17 AM, Cameron Kaiser  wrote:
>>
>> This would essentially mandate, then, that Gecko can only be built on
>> platforms with a Rust toolchain. That may be desirable, but it would
>> probably bust some of the obscure Tier-3 platforms and it would definitely
>> bust TenFourFox (we can't even get clang to be happy on 10.4 currently). Not
>> that we haven't been on borrowed time for awhile; I just point it out for
>> the record.
>
> I've been wondering about this. There's a big difference between (a)
> permitting Rust components (while still allowing fallback C++
> equivalents) and (b) mandating Rust components.

I think it doesn't make sense for us to support two MP4 demuxers, two
URL parsers, two encoding conversion libraries, etc. Once we put Rust
code into Gecko, I think we should commit to that code being a real
part of Gecko, and ports need to deal.

https://github.com/servo/servo/wiki/Building-for-Android suggests that
Android will be OK. For tier-1, I think the questions are XP, Snow
Leopard and Windows & Linux on CPUs without SSE2, but my understanding
is that all of these can already be supported by custom builds of
rustc (and if not, it wouldn't be unreasonable to drop some of those
three to get to the Rust future, IMO).

As for tier-3, a quick search indicates that rustc has already been
bootstrapped on (x86ish?) FreeBSD and OpenBSD. Wikipedia says that
LLVM supports MIPS, PowerPC, SPARC and Z/Architecture. It looks like
Debian has dropped 68K. So it should be feasible for *BSD and
non-mainstream CPU arch versions of Debian to come along. Maybe there
is rustc porting work to be done, but I think people who want these
platforms to be supported internalize the cost of supporting them
instead of expecting Gecko to stop progressing because of them.

Firefox 45 will be ESR. I think it's reasonable to make Rust a Gecko
build requirement after 45 and let tier-3 platforms use the ESR cycle
to get rustc/LLVM up and running where it's not up and running
already.

-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Nathan Froyd]

On Sat, Dec 12, 2015 at 5:17 PM, Cameron Kaiser 
wrote:

> This would essentially mandate, then, that Gecko can only be built on
> platforms with a Rust toolchain. That may be desirable, but it would
> probably bust some of the obscure Tier-3 platforms and it would definitely
> bust TenFourFox (we can't even get clang to be happy on 10.4 currently).
> Not that we haven't been on borrowed time for awhile; I just point it out
> for the record. 
>

FWIW, we'd also like to make Gecko require a C++11-compliant standard
library, and that will also have roughly the same effect as requiring
Rust.  I don't have exact dates on that happening, but I'd really like to
see it happen in the first half of 2016.

-Nathan
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Mike Hoye]

On 2015-12-14 4:06 PM, Justin Dolske wrote:

On 12/14/15 2:51 AM, Ted Mielczarek wrote:


[...]Obviously this isn't something we like to see, but
we shouldn't let the support of non-Tier 1 platforms guide our decision
making to that extent. Enabling Rust components in Gecko is important
work, and outweighs the value of supporting Firefox on minority
platforms.


+1. We shouldn't be doing any work to maintain Tier-3 platforms, nor 
should they hold us back from modernizing and securing the platforms 
used by the overwhelming majority of our users.


While I agree with this, giving those tier-3 platform maintainers as 
much advanced notice as possible and good-faith-if-low-touch guidance as 
to how they can keep rolling would be a fine thing.


I see a few names with no mailing addresses for a number of people on 
the supported build configurations page, so I'm going to see if I can 
find them and ask them to add their contact information.


- mhoye
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Justin Dolske]

On 12/14/15 2:51 AM, Ted Mielczarek wrote:


[...]Obviously this isn't something we like to see, but
we shouldn't let the support of non-Tier 1 platforms guide our decision
making to that extent. Enabling Rust components in Gecko is important
work, and outweighs the value of supporting Firefox on minority
platforms.


+1. We shouldn't be doing any work to maintain Tier-3 platforms, nor 
should they hold us back from modernizing and securing the platforms 
used by the overwhelming majority of our users.


Justin
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Nicholas Nethercote]

On Sun, Dec 13, 2015 at 9:17 AM, Cameron Kaiser  wrote:
>
> This would essentially mandate, then, that Gecko can only be built on
> platforms with a Rust toolchain. That may be desirable, but it would
> probably bust some of the obscure Tier-3 platforms and it would definitely
> bust TenFourFox (we can't even get clang to be happy on 10.4 currently). Not
> that we haven't been on borrowed time for awhile; I just point it out for
> the record.

I've been wondering about this. There's a big difference between (a)
permitting Rust components (while still allowing fallback C++
equivalents) and (b) mandating Rust components.

Step (a) is close at hand but I'm not aware of any planning or
predictions for when (b) will happen. The most detail I've seen was in
an exchange I had on Hackers News [1] where I said "it feels like
something that is multiple years away" and pcwalton replied "I agree
with you on the probable time frame here."

[1] https://news.ycombinator.com/item?id=10522194

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Nicholas Nethercote]

On Sun, Dec 13, 2015 at 11:28 AM, Bobby Holley  wrote:
>>
>> I've been wondering about this. There's a big difference between (a)
>> permitting Rust components (while still allowing fallback C++
>> equivalents) and (b) mandating Rust components.
>
> I don't know why we would allow there to be a long gap between (a) and (b).
> Maintaining/supporting two sets of the same code is costly. So if we get the
> rust code working and shipping on all platforms, I can't think of a reason
> why we wouldn't move as quickly as possible to requiring it.

The "if" in your second sentence is exactly what I'm worried about. My
gut tells me that step (b) is a *lot* harder than step (a). I could be
too pessimistic, but Android and the tier 3 platforms worry me.

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rust as build requirement was Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Cameron Kaiser]

On 12/13/15 12:50 AM, Nicholas Nethercote wrote:

On Sun, Dec 13, 2015 at 9:17 AM, Cameron Kaiser  wrote:


This would essentially mandate, then, that Gecko can only be built on
platforms with a Rust toolchain. That may be desirable, but it would
probably bust some of the obscure Tier-3 platforms and it would definitely
bust TenFourFox (we can't even get clang to be happy on 10.4 currently). Not
that we haven't been on borrowed time for awhile; I just point it out for
the record.


I've been wondering about this. There's a big difference between (a)
permitting Rust components (while still allowing fallback C++
equivalents) and (b) mandating Rust components.

Step (a) is close at hand but I'm not aware of any planning or
predictions for when (b) will happen. The most detail I've seen was in
an exchange I had on Hackers News [1] where I said "it feels like
something that is multiple years away" and pcwalton replied "I agree
with you on the probable time frame here."

[1] https://news.ycombinator.com/item?id=10522194


On the other hand, in that same thread, metajack said, "The current plan 
is to have Rust (in the form of rust-url replacing nsUrlParser) riding 
the trains later this quarter."


My usual genial scepticism alleges "this quarter" is optimistic, but I 
can well see it occurring Q1/Q2 2016. I did some Bugzilla digging and 
while the current plan in bug 1151899 is to run them in parallel, I 
foresee that lasting at most a release or two. That said, there hasn't 
been any activity on it since April.


Cameron Kaiser
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Bobby Holley]

On Sun, Dec 13, 2015 at 3:50 AM, Nicholas Nethercote  wrote:

> On Sun, Dec 13, 2015 at 9:17 AM, Cameron Kaiser 
> wrote:
> >
> > This would essentially mandate, then, that Gecko can only be built on
> > platforms with a Rust toolchain. That may be desirable, but it would
> > probably bust some of the obscure Tier-3 platforms and it would
> definitely
> > bust TenFourFox (we can't even get clang to be happy on 10.4 currently).
> Not
> > that we haven't been on borrowed time for awhile; I just point it out for
> > the record.
>
> I've been wondering about this. There's a big difference between (a)
> permitting Rust components (while still allowing fallback C++
> equivalents) and (b) mandating Rust components.
>
> Step (a) is close at hand but I'm not aware of any planning or
> predictions for when (b) will happen.


I don't know why we would allow there to be a long gap between (a) and (b).
Maintaining/supporting two sets of the same code is costly. So if we get
the rust code working and shipping on all platforms, I can't think of a
reason why we wouldn't move as quickly as possible to requiring it.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rust as build requirement was Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Jack Moffitt]

> On the other hand, in that same thread, metajack said, "The current plan is
> to have Rust (in the form of rust-url replacing nsUrlParser) riding the
> trains later this quarter."

And indeed, Rust is now riding the trains with the moov parser. It's
not required for the build yet, but Nightly (and soon Aurora)
dsitributions will both have it.

You can see this by looking at telemetry for MEDIA_RUST_MP4PARSER_SUCCESS.

jack.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Robert O'Callahan]

On Sun, Dec 13, 2015 at 2:28 PM, Bobby Holley  wrote:

> I don't know why we would allow there to be a long gap between (a) and (b).
> Maintaining/supporting two sets of the same code is costly. So if we get
> the rust code working and shipping on all platforms, I can't think of a
> reason why we wouldn't move as quickly as possible to requiring it.
>

I agree. I'd like to know why Nick and Patrick think this is multiple years
away. It had better not be!

Rob
-- 
lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe uresyf
toD
selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or rnea
lurpr
.a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Bobby Holley]

On Sun, Dec 13, 2015 at 6:27 PM, Nicholas Nethercote  wrote:

> On Sun, Dec 13, 2015 at 11:28 AM, Bobby Holley 
> wrote:
> >>
> >> I've been wondering about this. There's a big difference between (a)
> >> permitting Rust components (while still allowing fallback C++
> >> equivalents) and (b) mandating Rust components.
> >
> > I don't know why we would allow there to be a long gap between (a) and
> (b).
> > Maintaining/supporting two sets of the same code is costly. So if we get
> the
> > rust code working and shipping on all platforms, I can't think of a
> reason
> > why we wouldn't move as quickly as possible to requiring it.
>
> The "if" in your second sentence is exactly what I'm worried about. My
> gut tells me that step (b) is a *lot* harder than step (a). I could be
> too pessimistic, but Android


I believe there's a plan there, but don't have a good window into how long
it will take.


> and the tier 3 platforms


I'm pretty sure we wouldn't block on those, precisely because they're
tier-3.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Cameron Kaiser]

On 12/4/15 3:53 AM, Henri Sivonen wrote:

Hi,

I have written a proposal to a) rewrite Gecko's encoding converters
and b) to do it in Rust:
https://docs.google.com/document/d/13GCbdvKi83a77ZcKOxaEteXp1SOGZ_9Fmztb9iX22v0/edit

[...]

Furthermore, I think the rewrite should be in Rust, because

a) Now that we have Rust and are starting to include Rust code in
Gecko, it doesn't make sense to write new C++ code when the component
is isolated enough to be suited for being written in Rust.

b ) Importing a separate UTF-8-oriented conversion library written in
Rust for use by future Gecko components written in Rust (which would
ideally use UTF-8 internally, since Rust strings are UTF-8) would be a
footprint problem compared to a single conversion library designed for
both UTF-16 and UTF-8 with the same data tables. (For example, the URL
parser is being rewritten in Rust and the URL parser depends on the
rust-encoding library which doesn’t share data with our
UTF-16-oriented C++-based converters.)


This would essentially mandate, then, that Gecko can only be built on 
platforms with a Rust toolchain. That may be desirable, but it would 
probably bust some of the obscure Tier-3 platforms and it would 
definitely bust TenFourFox (we can't even get clang to be happy on 10.4 
currently). Not that we haven't been on borrowed time for awhile; I just 
point it out for the record.


Cameron Kaiser

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Bobby Holley]

On Sat, Dec 12, 2015 at 5:17 PM, Cameron Kaiser 
wrote:

> On 12/4/15 3:53 AM, Henri Sivonen wrote:
>
>> Hi,
>>
>> I have written a proposal to a) rewrite Gecko's encoding converters
>> and b) to do it in Rust:
>>
>> https://docs.google.com/document/d/13GCbdvKi83a77ZcKOxaEteXp1SOGZ_9Fmztb9iX22v0/edit
>>
> [...]
>
>> Furthermore, I think the rewrite should be in Rust, because
>>
>> a) Now that we have Rust and are starting to include Rust code in
>> Gecko, it doesn't make sense to write new C++ code when the component
>> is isolated enough to be suited for being written in Rust.
>>
>> b ) Importing a separate UTF-8-oriented conversion library written in
>> Rust for use by future Gecko components written in Rust (which would
>> ideally use UTF-8 internally, since Rust strings are UTF-8) would be a
>> footprint problem compared to a single conversion library designed for
>> both UTF-16 and UTF-8 with the same data tables. (For example, the URL
>> parser is being rewritten in Rust and the URL parser depends on the
>> rust-encoding library which doesn’t share data with our
>> UTF-16-oriented C++-based converters.)
>>
>
> This would essentially mandate, then, that Gecko can only be built on
> platforms with a Rust toolchain. That may be desirable, but it would
> probably bust some of the obscure Tier-3 platforms and it would definitely
> bust TenFourFox (we can't even get clang to be happy on 10.4 currently).
> Not that we haven't been on borrowed time for awhile; I just point it out
> for the record.
>

FWIW, this is most likely coming regardless of what we do with the encoding
converters. The rust mp4 demuxer is already on nightly - see bug 1175322.
It is an experiment to dig up any tooling and workflow issues that may
exist, and if it succeeds, we will presumably remove libstagefright. We are
also actively working on something similar for the URL parser.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Henri Sivonen]

Hi,

I have written a proposal to a) rewrite Gecko's encoding converters
and b) to do it in Rust:
https://docs.google.com/document/d/13GCbdvKi83a77ZcKOxaEteXp1SOGZ_9Fmztb9iX22v0/edit

I'd appreciate comments--especially from the owners of the uconv
module and from people who have worked on encoding-related Rust code
and on Rust code that needs encoding converters and is on track to be
included in Gecko.

I've put the proposal on Google Docs in order to benefit from the GDoc
commenting feature that allows comments from multiple reviewers to be
attached to particular bits of text.

The document is rather long. The summary is:

I think we should rewrite of Gecko's character encoding converters
such that conversion to and from both in-memory UTF-16 and UTF-8 is
supported, because

1) Currently, we can convert to and from UTF-16, which steers us to
write parsers that operate on UTF-16. This is bad: ideally parsers
would operate on UTF-8 to allow parsers to traverse a more compact
memory representation (even HTML has plenty of ASCII markup; other
formats we parse are even more ASCII-dominated). To make sure we don't
write more UTF-16-based parsers in the future, we should have
converters that can convert to and from UTF-8, too, but without paying
the footprint cost of two independent sets of converters.

2) The footprint of Gecko is still a relevant concern in the Fennec
case. (See e.g. the complications arising from Gecko developers being
blocked from including ICU [not its converters] into Gecko on
Android.) Our current converters are bloated due to optimizing the
encode operation for legacy encoding for speed at the expense of
lookup table size and we could make Gecko a bit smaller (i.e. make
some room for good stuff on Android) by being smarter about encoding
converter data tables. (Optimizing the relatively rare and performance
non-sensitive encode operation for legacy encodings for size instead
of speed.)

3) We should ensure the correctness of our converters and then stop
tweaking them.

4) ...But our current converters are so unmaintainable that making
these changes would be the easiest to accomplish via a rewrite.

Furthermore, I think the rewrite should be in Rust, because

a) Now that we have Rust and are starting to include Rust code in
Gecko, it doesn't make sense to write new C++ code when the component
is isolated enough to be suited for being written in Rust.

b ) Importing a separate UTF-8-oriented conversion library written in
Rust for use by future Gecko components written in Rust (which would
ideally use UTF-8 internally, since Rust strings are UTF-8) would be a
footprint problem compared to a single conversion library designed for
both UTF-16 and UTF-8 with the same data tables. (For example, the URL
parser is being rewritten in Rust and the URL parser depends on the
rust-encoding library which doesn’t share data with our
UTF-16-oriented C++-based converters.)

-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Henri Sivonen]

On Fri, Dec 4, 2015 at 5:54 PM, Henri Sivonen  wrote:
> On Fri, Dec 4, 2015 at 3:18 PM, Ted Mielczarek  wrote:
>> 2) Instead of a clean-room implementation, would it be possible to fix
>> the problems you see with rust-encoding so that it's suitable for our
>> use? Especially if Servo is already using it, it would be a shame to
>> wind up with two separate implementations.
>
> I'm shy to suggest to the rust-encoding maintainer that I should be
> able to come in and trample all over the rust-encoding internals
> because of Gecko and what I see as priorities from the Gecko-informed
> point of view.

I should have mentioned that Ms2ger filed
https://github.com/lifthrasiir/rust-encoding/issues/92

-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Ms2ger]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/04/2015 04:54 PM, Henri Sivonen wrote:
> On Fri, Dec 4, 2015 at 3:18 PM, Ted Mielczarek 
> wrote:
>> 1) What does Servo do, just use rust-encoding directly?
> 
> That is my understanding, but it would be good if a Servo
> developer could confirm.

This is correct.

>> 2) Instead of a clean-room implementation, would it be possible
>> to fix the problems you see with rust-encoding so that it's
>> suitable for our use? Especially if Servo is already using it, it
>> would be a shame to wind up with two separate implementations.
> 
> I'm shy to suggest to the rust-encoding maintainer that I should
> be able to come in and trample all over the rust-encoding
> internals because of Gecko and what I see as priorities from the
> Gecko-informed point of view. It doesn't make sense to implement a
> Gecko-ish API where the caller allocates the output buffer (like
> I'm proposing) on top of rust-encoding. However, it would make
> sense to implement (at least the common call patterns of) the
> rust-encoding API on top of the kind of API that I'm proposing. But
> in order to do so, the internals of rust-encoding would end up
> replaced with the kind of library that I'm proposing.
> 
> As for whether any parts of the rust-encoding internals would be 
> reusable in the kind of library that I'm proposing, as noted in
> the proposal document, it generally makes no sense to adapt one 
> implementation strategy for the single-byte encodings to another.
> If you want a different implementation strategy for the
> single-byte encodings, it simpler to just rewrite from scratch to
> the strategy that you want.
> 
> As for the multi-byte converters, the concern I have is that 
> rust-encoding implements them using a macro that generates a state 
> machine and this makes the code look different from the algorithms 
> from the spec. It might be really dumb of me to suggest not using
> that macro, but I think there is value to having the code look like
> the algorithms in the spec so that it's easy to compare the two. So
> in the case of multi-byte converters it's mainly a question of
> whether we prefer the (possible really cool) macro or code that is
> easy to compare with the spec. I currently prefer code looking like
> the spec, but maybe I could be convinced otherwise. (Either way,
> I'd get rid of the encode-optimized lookup tables and search the
> decode tables in reverse instead.)
> 
> It would probably be worthwhile to copy and paste from the 
> rust-encoding UTF-8 and UTF-16 converters.

I agree that it is useful to have code looking like the spec in the
general case. However, if we were to get to the point where that was
the only argument against unifying your proposed library and
rust-unicode, I think we should probably go for the unification anyway.

HTH
Ms2ger

-BEGIN PGP SIGNATURE-

iQEcBAEBAgAGBQJWYblSAAoJEOXgvIL+s8n2uwQH/ipxhkqZpqZIEZIZcAezbKfw
1on3mC/0cnwJywu9yqqlTSXAoQJxONbdWeLJnRU9RvEgreT+EOp+0ktRgUubg34h
qAew1zdRhS7ldIZTWyePX4EOpUsvtIqXXpyJcw3Tl76bTx+skp3mov+lhZxTLS/3
ZsDayhHuYwhSB/h2KfYP09ee5i3AyKPjWkPyWIMw9jRXxJD+bWVcj++V1s2/3V9R
A4MnAJOB8Cqyhp+aMi1+mbx2QTYEqXqLak9ifKV0hHfF80+qI3aGkFQhr/fbhdgl
9rSBz7gI4lVzXtt8wSNaBGnRSf5mGvGmz5wQC7VyGtj2NMYKcmbCErXavhiSW6g=
=ixRO
-END PGP SIGNATURE-
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Henri Sivonen]

On Fri, Dec 4, 2015 at 3:18 PM, Ted Mielczarek  wrote:
> 1) What does Servo do, just use rust-encoding directly?

That is my understanding, but it would be good if a Servo developer
could confirm.

> 2) Instead of a clean-room implementation, would it be possible to fix
> the problems you see with rust-encoding so that it's suitable for our
> use? Especially if Servo is already using it, it would be a shame to
> wind up with two separate implementations.

I'm shy to suggest to the rust-encoding maintainer that I should be
able to come in and trample all over the rust-encoding internals
because of Gecko and what I see as priorities from the Gecko-informed
point of view. It doesn't make sense to implement a Gecko-ish API
where the caller allocates the output buffer (like I'm proposing) on
top of rust-encoding. However, it would make sense to implement (at
least the common call patterns of) the rust-encoding API on top of the
kind of API that I'm proposing. But in order to do so, the internals
of rust-encoding would end up replaced with the kind of library that
I'm proposing.

As for whether any parts of the rust-encoding internals would be
reusable in the kind of library that I'm proposing, as noted in the
proposal document, it generally makes no sense to adapt one
implementation strategy for the single-byte encodings to another. If
you want a different implementation strategy for the single-byte
encodings, it simpler to just rewrite from scratch to the strategy
that you want.

As for the multi-byte converters, the concern I have is that
rust-encoding implements them using a macro that generates a state
machine and this makes the code look different from the algorithms
from the spec. It might be really dumb of me to suggest not using that
macro, but I think there is value to having the code look like the
algorithms in the spec so that it's easy to compare the two. So in the
case of multi-byte converters it's mainly a question of whether we
prefer the (possible really cool) macro or code that is easy to
compare with the spec. I currently prefer code looking like the spec,
but maybe I could be convinced otherwise. (Either way, I'd get rid of
the encode-optimized lookup tables and search the decode tables in
reverse instead.)

It would probably be worthwhile to copy and paste from the
rust-encoding UTF-8 and UTF-16 converters.

-- 
Henri Sivonen
hsivo...@hsivonen.fi
https://hsivonen.fi/
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to a) rewrite Gecko's encoding converters and b) to do it in Rust

[Ted Mielczarek]

On Fri, Dec 4, 2015, at 06:53 AM, Henri Sivonen wrote:
> Hi,
> 
> I have written a proposal to a) rewrite Gecko's encoding converters
> and b) to do it in Rust:
> https://docs.google.com/document/d/13GCbdvKi83a77ZcKOxaEteXp1SOGZ_9Fmztb9iX22v0/edit
> 
> I'd appreciate comments--especially from the owners of the uconv
> module and from people who have worked on encoding-related Rust code
> and on Rust code that needs encoding converters and is on track to be
> included in Gecko.

I don't really know anything about our encoding story, so I'll leave
that to others, but I'm generally in favor of writing new code in Rust
and replacing bits of Gecko with new Rust implementations. I don't know
that we've worked out all the kinks in including Rust code in Gecko
yet[1], but we're getting pretty close.

I have two questions:
1) What does Servo do, just use rust-encoding directly?
2) Instead of a clean-room implementation, would it be possible to fix
the problems you see with rust-encoding so that it's suitable for our
use? Especially if Servo is already using it, it would be a shame to
wind up with two separate implementations.

-Ted

1. https://bugzilla.mozilla.org/show_bug.cgi?id=oxidation
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform