Re: Improved stacks in Linux crash reports

[ISHIKAWA,chiaki]

On 2020/11/10 14:49, Gabriele Svelto wrote:

On 10/11/20 05:57, ISHIKAWA,chiaki wrote:

Does this support split dwarf object files?

Not yet, but it should be possible to add support easily because the
crates we depend on for parsing DWARF debuginfo have just been updated
to support split DWARF [1].

  Gabriele

[1] https://github.com/gimli-rs/gimli/blob/master/CHANGELOG.md#added


Wow, thank you for the quick response.

It sounds so encouraging (!)

Now it is a matter of figuring out how to find out reliably if an object 
uses split dwarf scheme...


Chiaki



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Improved stacks in Linux crash reports

[ISHIKAWA,chiaki]

On 2020/11/06 17:56, Gabriele Svelto wrote:

[cross-posting to dev-platform]

  Hello all,
I'm glad to announce that we switched our symbol scrapers for Linux
distributions to our new and improved Rust-based tooling [1]. This means
that when looking at crash reports or profiling you will not find stack
traces with missing or omitted function names anymore.

If you're tracking Linux crashes this might cause signature changes
starting with release version 83; the new signatures should be
significantly improved.

The impact of this change will be more visible to users on Fedora and
OpenSUSE.

  Gabriele

[1] https://github.com/mozilla/dump_syms/
___


Does this support split dwarf object files?
I use GCC -split-dwarf to generate separate object and dwarf files 
locally to speed up gdb startup.
Unfortunately, this is not grokked by stack dump code since rust-based 
one replaced the older ruby code that used objdump, etc.

So I get either empty or undecoded stack dump there.
I have not the time to work around on this issue, even the hack to 
revert back to the old code if the object files turn out to be 
split-dwarf type.


Just wondering.

(Maybe I should switch back and not to use -split-dwarf until I need to 
run gdb seriously. There WERE times that I needed to run thunderbird 
under gdb to check subtle bugs.)


Thank you for improving the backends.

Chiaki


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal: remove support for running desktop Firefox in single-process mode (e10s disabled) anywhere but in tests

[ISHIKAWA,chiaki]

I think I have found an inconsistency in TB's pref setting during 
mochitest regarding e10s.
The gory detail is in 
https://bugzilla.mozilla.org/show_bug.cgi?id=1629433#c10


Major find (excerpt from the above bugzilla.)

--- begin quote ---
1) Dynamically generated user.js files during tests of C-C TB mochitest 
execution, contain conflicting stetting.


pref("browser.tabs.remote.autostart", {true|false});

The value is either true or false. Not consistent. In mochitest of C-C 
TB, there may be a few very browser-like test. In those cases,
|browser.tabs.remote.autostart| can be true? However, if we ALWAYS need 
to set the value for false for C-C TB mochitest,


    >> // No e10s in Thunderbird for now.
    >> pref("browser.tabs.remote.autostart", false); <=== This

we may be in a big trouble.

I observe that there are multiple lines to set 
"browser.tabs.remote.autostart" in a user.js with different
setting. Of course, the last one sets the used value. Inquiring mind 
wants to know how the setting is composed (presumably concatenate some
settings prepackaged in user.js files in test file directories? I have 
noticed that there are user.js files in test directory. See (3)
below.) Clear description of how the run-time user.js is created for 
each test during c-c TB mochitest run would be desirable.


In any case, I DID find cases where sub-tests (albeit web 
browser-oriented?) run with a user.js with
|pref("browser.tabs.remote.autostart", true);| which may not sit well 
for C-C thunderbird binary.


I have a nagging feeling that this bad setting may/could be one of the 
causes of the failure to run valgrind because we need more than 1500+

threads to run the C-C TB mochitest(!)

(2) user.js file under MOZ_OBJ/temp, 
/NEW-SSD/moz-obj-dir/objdir-tb3/temp/user.js, does NOT set 
|pref("browser.tabs.remote.autostart", false);|


I wonder if this user.js in MOZ_OBJ directory is used or not during C-C 
mochitest. If it is used, I think we are in a big trouble since it does 
not set the value to false.


(3) There are a few |user.js| files with conflicting settings in the 
test directories. Some set the pref to false, but some set to true.
I hope they are used appropriately. Since the C-C mochitest works more 
or less, I believe they are used correctly, but a clear
documentation/explanation of how each is used and how each test creates 
user.js dynamically from the prepackaged user.js for TB

mochitest would be desirable.
--- end quote ---


So if the testing of web browser function in C-C TB mochitest runs with 
e10s enabled, then it is not testing the real world usage because

e10s is disabled for TB usually.
(OTOH, it is hard to believe that if TB needs to disable e10s, the 
built-in web renderer in TB can run with e10s enabled without disrupting 
the behavior of the rest of the code. Something is screwed up in 
dynamically generated mochitest user.js )


I hope someone familiar with C-C TB mochitest investigates this and 
figures if the current state of the affairs is correct or not.


Chiaki

On 2020/06/18 16:46, Magnus Melin wrote:

Currently Thunderbird doesn't work with e10s.

Longer term I'm assuming we'll need to do necessary adaptions so that 
we can - but I suspect this is a slightly larger project...


I've filed bug 1646648 to track this work.

 -Magnus

On 2020-06-10 22:05, Gijs Kruitbosch wrote:
I can't speak for Thunderbird's plans, but either way these plans 
shouldn't affect them and is restricted to desktop Firefox; the pref 
still works there: 
https://searchfox.org/mozilla-central/rev/4bb2401ecbfce89af06fb2b4d0ea3557682bd8ff/toolkit/xre/nsAppRunner.cpp#5020-5024 
, and they set it: 
https://searchfox.org/comm-central/rev/e62a0af3cba6e1c65b2d4be02d3fefce88cf3f8f/mail/app/profile/all-thunderbird.js#654 



Of course, if TB needs this configuration then that may complicate 
removing support for non-e10s entirely...


~ Gijs

On 10/06/2020 19:56, Emilio Cobos Álvarez wrote:
What is the situation of Thunderbird? I think they don't have e10s 
enabled

yet, and it may be worth at least knowing what their plans are.

  -- Emilio

On Wed, Jun 10, 2020, 8:44 PM Dave Townsend  
wrote:


Non-e10s is such a different environment that I don't think we have 
any
hope of keeping it working without running the full test suite in 
that mode

and I don't think anyone wants to do that. Now that this has started
breaking I think it is actively harmful to our users for us to 
allow them

to disable e10s.

On Wed, Jun 10, 2020 at 11:30 AM Gijs Kruitbosch 



wrote:


(Copied to fx-dev; Replies to dev-platform please.)

Hello,

Just over a year ago, I started a discussion[0] about our support for
disabling e10s. The outcome of that was that we removed support for
disabling e10s with a pref on Desktop Firefox with version 68, except
for use from automation. We kept support for using the environment
variable. [1]

Last week, we released Firefox 77, which turned out to break all
webpages sent using compression (like 

Re: Proposal: remove support for running desktop Firefox in single-process mode (e10s disabled) anywhere but in tests

[ISHIKAWA,chiaki]

I can't speak for what TB development plan is.

One thing I observe as an occasional submitter of TB patches is this.
Thunderbird ditched |mozmill| test December 2019, and switched to 
mochitest in place of mozmill test.

Unfortunately, valgrind no longer works locally for mochitest.
This is quite a loss for keeping the code in sane state. I have 
uncovered quite a few memory-related bugs over the years by running TB 
under valgrind during mozmill-based test.
(I don't know if there is any official valgrind run of TB on tryserver 
despite arcane description I read years ago that

it was performed from time to time.).
xpcshell-tests still runs under valgrind. I found a few memory-related 
errors by running TB under valgrind during xpcshell-tests this year.


So for me, valgrind test is very important for keeping TB in sane state 
after a large patch set lands, etc.
(Actually, I have a large patch set that enables buffered-write for 
message handling and this will boost write performance very much. But it 
is a rather lage patch set, and I want to run tests under valgrind to 
make sure nothing undesirable is introduced.)


Looking at the posts, I have a feeling that TB may be able to run under 
valgrind during mochitest if this
e10s setting is properly handled.  Right now, TB under valgrind during 
mochitest starts more than 1500 threads (too many IMHO), but less than 
5000 threads (I have not figured out exactly how many threads are 
required), and valgrind barfs. I need to increase the number of threads 
valgrind supports,
but maybe because of the large number of threads, tests time out before 
anything useful is performed. Thus no useful memory test is done under 
mochitest currently.


Something is wrong there.

I suspect this e10s setting *may* have a bit to do with the situation.
I think I have to make sure TB runs in non e10s mode to run under 
valgrind during mochitest.
Specifically I am not sure if the setting is honored in test user 
profile prepared for  mochitest.:
https://searchfox.org/comm-central/rev/e62a0af3cba6e1c65b2d4be02d3fefce88cf3f8f/mail/app/profile/all-thunderbird.js#654 


||
|// No e10s in Thunderbird for now. |
|pref("browser.tabs.remote.autostart", false); |
||
*Maybe*, by making sure that this is set to false, valgrind might work 
under mochitest.

Maybe a fishing trip in the coming weekend.

Chiaki

On 2020/06/11 4:05, Gijs Kruitbosch wrote:
I can't speak for Thunderbird's plans, but either way these plans 
shouldn't affect them and is restricted to desktop Firefox; the pref 
still works there: 
https://searchfox.org/mozilla-central/rev/4bb2401ecbfce89af06fb2b4d0ea3557682bd8ff/toolkit/xre/nsAppRunner.cpp#5020-5024 
, and they set it: 
https://searchfox.org/comm-central/rev/e62a0af3cba6e1c65b2d4be02d3fefce88cf3f8f/mail/app/profile/all-thunderbird.js#654


Of course, if TB needs this configuration then that may complicate 
removing support for non-e10s entirely...


~ Gijs

On 10/06/2020 19:56, Emilio Cobos Álvarez wrote:
What is the situation of Thunderbird? I think they don't have e10s 
enabled

yet, and it may be worth at least knowing what their plans are.

  -- Emilio

On Wed, Jun 10, 2020, 8:44 PM Dave Townsend  
wrote:



Non-e10s is such a different environment that I don't think we have any
hope of keeping it working without running the full test suite in 
that mode

and I don't think anyone wants to do that. Now that this has started
breaking I think it is actively harmful to our users for us to allow 
them

to disable e10s.

On Wed, Jun 10, 2020 at 11:30 AM Gijs Kruitbosch 



wrote:


(Copied to fx-dev; Replies to dev-platform please.)

Hello,

Just over a year ago, I started a discussion[0] about our support for
disabling e10s. The outcome of that was that we removed support for
disabling e10s with a pref on Desktop Firefox with version 68, except
for use from automation. We kept support for using the environment
variable. [1]

Last week, we released Firefox 77, which turned out to break all
webpages sent using compression (like gzip) if you had disabled e10s
using this environment variable. [2]

So here we are again. I'd like to propose we also stop honouring the
environment variable unless we're running tests in automation. We
clearly do not have sufficient test coverage to guarantee basic things
like "the browser works", it lacks security sandboxing, and a 
number of

other projects require it (fission, gpu process, socket process, ...),
so I think it's time to stop supporting this configuration at all.

I hope to make this change for the 79 cycle. I'm open to arguments
either way about what to do for 78 esr (assuming the patch for 79 
turns

out to be simple; the work to remove the pref had a number of annoying
corner-cases at the time).

Please speak up if you think that this plan needs adjusting.

~ Gijs


[0]


https://groups.google.com/d/msg/mozilla.dev.platform/cJMzxi7_PmI/Pi1IOg_wCQAJ 


[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1548941
[2] 

Re: Cargo.lock in hg repository?

[ISHIKAWA,chiaki]

I have rebased and the changes are gone.

Thank you for keeping the tree in good shape.

Chiaki

On 2020/05/19 18:35, Emilio Cobos Álvarez wrote:

On 5/19/20 3:11 AM, ISHIKAWA,chiaki wrote:
Thank you for the feedback. So I am not alone in seeing this. I will 
ignore the anomaly for now although there are a few strange local 
changes due to this now.


You shouldn't see it anymore if you rebase:

  https://hg.mozilla.org/mozilla-central/rev/6db4b71d0c20

 -- Emilio



Chiaki

On 2020/05/18 1:08, Emilio Cobos Álvarez wrote:
I see this too, I think it's a rebase messup between bug 1631630 and 
bug 1636068.


 -- Emilio

On 5/17/20 2:44 PM, ISHIKAWA,chiaki wrote:

I am developing TB patches locally on my linux PC.

Have anyone noticed mozilla/Cargo.lock seems to be in mercurial 
repository and that it got updated

during development?
That is, as far as *I* am concerned, I never tough it directly.
But, it seems |mach clobber|, |mach configure|, |mach build| and 
other |mach| invocations and the subsequent commands invoked within 
during configure and compilation/build seemed to
change the file, mozilla/Cargo.lock. And during local hg update 
(applying my patches, etc.), hg commands suddenly states
that there is a local diff and refresh my repository before 
proceeding.

Surprised, I check what changed. The changed file is Cargo.lock.
This happened twice over the weekend.
(I updated the source tree from the M-C and C-C repository a few 
times. Each time the update worked OK. Only after a compilation and 
I tried to apply my local patch, this happened. My using hg MQUEE 
feature should not be a problem, though...)


The changes I noticed seem to be just the relocations of version 
information of some tools/packages in different part of Cargo.lock.
(Simple deletion/addition of same block in different parts of the 
file.)


This is so strange, and I wonder if there is something wrong with 
my setup.


I thought this is the first time it happened on my PC, but a close 
check of local patches revealed
something like this happened last August (Aug. 2019). Back then, I 
was trying very hard to accommodate the source tree layout change 
(C-C now having become the subtree of ./mozilla as mozilla/comm, 
and there were many local patches and locally crafted scripts  I 
needed to update and it seems that I simply accepted Cargo.lock 
change as part of the big tree layout and never bothered about it 
until today.)


Again, has anyone seen Cargo.lock being modified without direct 
user intervention on their PC?


TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Cargo.lock in hg repository?

[ISHIKAWA,chiaki]

Thank you for the feedback. So I am not alone in seeing this. I will 
ignore the anomaly for now although there are a few strange local 
changes due to this now.


Chiaki

On 2020/05/18 1:08, Emilio Cobos Álvarez wrote:
I see this too, I think it's a rebase messup between bug 1631630 and 
bug 1636068.


 -- Emilio

On 5/17/20 2:44 PM, ISHIKAWA,chiaki wrote:

I am developing TB patches locally on my linux PC.

Have anyone noticed mozilla/Cargo.lock seems to be in mercurial 
repository and that it got updated

during development?
That is, as far as *I* am concerned, I never tough it directly.
But, it seems |mach clobber|, |mach configure|, |mach build| and 
other |mach| invocations and the subsequent commands invoked within 
during configure and compilation/build seemed to
change the file, mozilla/Cargo.lock. And during local hg update 
(applying my patches, etc.), hg commands suddenly states

that there is a local diff and refresh my repository before proceeding.
Surprised, I check what changed. The changed file is Cargo.lock.
This happened twice over the weekend.
(I updated the source tree from the M-C and C-C repository a few 
times. Each time the update worked OK. Only after a compilation and I 
tried to apply my local patch, this happened. My using hg MQUEE 
feature should not be a problem, though...)


The changes I noticed seem to be just the relocations of version 
information of some tools/packages in different part of Cargo.lock.

(Simple deletion/addition of same block in different parts of the file.)

This is so strange, and I wonder if there is something wrong with my 
setup.


I thought this is the first time it happened on my PC, but a close 
check of local patches revealed
something like this happened last August (Aug. 2019). Back then, I 
was trying very hard to accommodate the source tree layout change 
(C-C now having become the subtree of ./mozilla as mozilla/comm, and 
there were many local patches and locally crafted scripts  I needed 
to update and it seems that I simply accepted Cargo.lock change as 
part of the big tree layout and never bothered about it until today.)


Again, has anyone seen Cargo.lock being modified without direct user 
intervention on their PC?


TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Cargo.lock in hg repository?

[ISHIKAWA,chiaki]

I am developing TB patches locally on my linux PC.

Have anyone noticed mozilla/Cargo.lock seems to be in mercurial 
repository and that it got updated

during development?
That is, as far as *I* am concerned, I never tough it directly.
But, it seems |mach clobber|, |mach configure|, |mach build| and other 
|mach| invocations and the subsequent commands invoked within during 
configure and compilation/build seemed to
change the file, mozilla/Cargo.lock. And during local hg update 
(applying my patches, etc.), hg commands suddenly states

that there is a local diff and refresh my repository before proceeding.
Surprised, I check what changed. The changed file is Cargo.lock.
This happened twice over the weekend.
(I updated the source tree from the M-C and C-C repository a few times. 
Each time the update worked OK. Only after a compilation and I tried to 
apply my local patch, this happened. My using hg MQUEE feature should 
not be a problem, though...)


The changes I noticed seem to be just the relocations of version 
information of some tools/packages in different part of Cargo.lock.

(Simple deletion/addition of same block in different parts of the file.)

This is so strange, and I wonder if there is something wrong with my setup.

I thought this is the first time it happened on my PC, but a close check 
of local patches revealed
something like this happened last August (Aug. 2019). Back then, I was 
trying very hard to accommodate the source tree layout change (C-C now 
having become the subtree of ./mozilla as mozilla/comm, and there were 
many local patches and locally crafted scripts  I needed to update and 
it seems that I simply accepted Cargo.lock change as part of the big 
tree layout and never bothered about it until today.)


Again, has anyone seen Cargo.lock being modified without direct user 
intervention on their PC?


TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Is --disable-libjpeg-turbo gone?

[ISHIKAWA,chiaki]

On 2020/05/16 7:51, Mike Hommey wrote:

On Sat, May 16, 2020 at 07:24:44AM +0900, ISHIKAWA,chiaki wrote:

I have --disable-libjpeg-turbo in my mozconfig for many years (I build TB
locally).

After an update of the local source tree this morning, it is no longer
recognized?
(The last update was several days ago.)

Has anyone also experienced this?

Traceback (most recent call last):
   File "/NEW-SSD/NREF-COMM-CENTRAL/mozilla/configure.py", line 181, in

     sys.exit(main(sys.argv))
   File "/NEW-SSD/NREF-COMM-CENTRAL/mozilla/configure.py", line 52, in main
     sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))
   File 
"/NEW-SSD/NREF-COMM-CENTRAL/mozilla/python/mozbuild/mozbuild/configure/__init__.py",
line 494, in run
     raise InvalidOptionError(msg)
mozbuild.configure.options.InvalidOptionError: Unknown option:
--disable-libjpeg-turbo
*** Fix above errors and then restart with\
    "./mach build"
make: *** [client.mk:115: configure] Error 1

I removed it in bug 1638195, but it hasn't done anything since bug
1515852.

Mike



Thank you for the pointer.

Does this mean I can safely remove --disable-libjpeg-turbo?
[I am not sure why I have this. It has been there for years, and I 
forgot about it.
Possibly YASM dependency issues mentioned in the bug under Debian 
GNU/Linux.]


Or should I say explicitly --with-system-jpeg in mozconfig?

TIA



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Is --disable-libjpeg-turbo gone?

[ISHIKAWA,chiaki]

I have --disable-libjpeg-turbo in my mozconfig for many years (I build 
TB locally).


After an update of the local source tree this morning, it is no longer 
recognized?

(The last update was several days ago.)

Has anyone also experienced this?

Traceback (most recent call last):
  File "/NEW-SSD/NREF-COMM-CENTRAL/mozilla/configure.py", line 181, in 


    sys.exit(main(sys.argv))
  File "/NEW-SSD/NREF-COMM-CENTRAL/mozilla/configure.py", line 52, in main
    sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))
  File 
"/NEW-SSD/NREF-COMM-CENTRAL/mozilla/python/mozbuild/mozbuild/configure/__init__.py", 
line 494, in run

    raise InvalidOptionError(msg)
mozbuild.configure.options.InvalidOptionError: Unknown option: 
--disable-libjpeg-turbo

*** Fix above errors and then restart with\
   "./mach build"
make: *** [client.mk:115: configure] Error 1


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: New and improved stack-fixing

[ISHIKAWA,chiaki]

Dear Nick,

Will my replacing fix-stacks binary with a binary to echo the input line 
verbatim into the output

help me to obtain the original stack trace line?

I need to get the ORIGINAL line to see if I can get my interim script 
based on linux-stack-fix.py to work
and understand the  subtle issue about the base address mentioned in the 
github ticket.


Obviously the offset printed in the current  log against split debug 
executable cannot be fed to the original linux-stack-fix.py to obtain 
reasonable
stacktrace from my experiment. I thought it was OK, but it was pointed 
out that the stack trace thus obtained was trash.

(See https://bugzilla.mozilla.org/show_bug.cgi?id=1621274 )
Strange. linux-stack-fix.py  used to work when the output was not 
modified by the recent fix-stack. That is what I used to

for many years.

TIA

Chiaki
On 2020/04/30 3:07, ISHIKAWA,chiaki wrote:

Thank you for the tips.

I will have a go to incorporate the logic of old fix_linux_stack.py 
into fix-stacks.


TIA

Chiaki

On 2020/04/29 13:11, Nicholas Nethercote wrote:
On Wed, 29 Apr 2020 at 02:32, ISHIKAWA,chiaki  
wrote:



Ouch, I am using -gsplit-dwarf to make gdb startup time faster...

-gsplit-dwarf is set manually on my bash script before invoking ./mach
configure and build.


That would be the problem!

Do you think digging github of fix-stack gets me anywhere near the

solution?


Or do I have to do a major surgery in the innards of|used libraries of

symbolic and/or goblin ?


https://github.com/mozilla/fix-stacks/issues/22 is the open issue for
separate debug info. There is some information in that issue about 
what is
necessary. I think copying over the old logic from fix_linux_stack.py 
would

be a moderate-sized task.

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: New and improved stack-fixing

[ISHIKAWA,chiaki]

Thank you for the tips.

I will have a go to incorporate the logic of old fix_linux_stack.py into 
fix-stacks.


TIA

Chiaki

On 2020/04/29 13:11, Nicholas Nethercote wrote:

On Wed, 29 Apr 2020 at 02:32, ISHIKAWA,chiaki  wrote:


Ouch, I am using -gsplit-dwarf to make gdb startup time faster...

-gsplit-dwarf is set manually on my bash script before invoking ./mach
configure and build.


That would be the problem!

Do you think digging github of fix-stack gets me anywhere near the

solution?


Or do I have to do a major surgery in the innards of|used libraries of

symbolic and/or goblin ?


https://github.com/mozilla/fix-stacks/issues/22 is the open issue for
separate debug info. There is some information in that issue about what is
necessary. I think copying over the old logic from fix_linux_stack.py would
be a moderate-sized task.

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: New and improved stack-fixing

[ISHIKAWA,chiaki]

On 2020/04/28 18:13, Nicholas Nethercote wrote:

On Tue, 28 Apr 2020 at 17:50, ISHIKAWA,chiaki  wrote:


   0:28.98 GECKO(727384) #01: ???
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x5ab5921)
   0:28.99 GECKO(727384) #02: ???
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x5ab7f34)
   0:28.99 GECKO(727384) #03: ???
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x590de76)
   0:28.99 GECKO(727384) #04: ???
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x5a94c2f)
   0:28.99 GECKO(727384) #05: ???
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x5ab1589)

[...]
The file is  mozilla/tools/rb/fix_stacks.py.:

# Matches lines produced by MozFormatCodeAddress(), e.g.
# `#01: ???[tests/example +0x43a0]`.
line_re = re.compile("#\d+: .+\[.+ \+0x[0-9A-Fa-f]+\]")

You see the re (Regular Expression) does not match the line in mochitest
log produced by NS_ASSERTION().

What gives?


The example at the top of https://github.com/mozilla/fix-stacks/ explains
this.
```
For example, a line like this in the input:

#01: ???[tests/example +0x43a0]

is changed to something like this in the output:

#01: main (/home/njn/moz/fix-stacks/tests/example.c:24)

```

So the output has definitely gone through `fix-stacks`, which is what I
would expect for mochitest output, because all mochitest output is fed
through `fix-stacks`.


I see.



I don't know why it was unable to find the function name, filename and line
number. If something was wrong with the files I would expect `fix-stacks`
to emit an error message. Are the files built with debug info?


It is built with debug info.

But the link above has this.:



  <https://github.com/mozilla/fix-stacks/#shortcomings>Shortcomings

On Linux, use with debuginfo sections in separate files is untested 
and probably does not work.



Ouch, I am using -gsplit-dwarf to make gdb startup time faster...

-gsplit-dwarf is set manually on my bash script before invoking ./mach 
configure and build.


```
export CFLAGS="$CFLAGS $MEMORYMODEL $ASAN -fno-builtin-strlen 
$SPLITDWARF -Dfdatasync=fdatasync  -DDEBUG_4GB_CHECK -DUSEHELGRIND=1 
-DUSEVALGRIND=1 -DDEBUG  $NULLPTRCHK -g -gsplit-dwarf ${WARNFLAGS}"
export CXXFLAGS="$CXXFLAGS $MEMORYMODEL $ASAN  -fno-builtin-strlen 
$SPLITDWARF -Dfdatasync=fdatasync -DDEBUG_4GB_CHECK -DUSEHELGRIND=1 
-DUSEVALGRIND=1 -DDEBUG $NULLPTRCHK  -g -gsplit-dwarf ${WARNFLAGS}"

```

Do you think digging github of fix-stack gets me anywhere near the solution?
Or do I have to do a major surgery in the innards of|used libraries of
smbolic and/or goblin ?

TIA|

|Chiaki|



Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: New and improved stack-fixing

[ISHIKAWA,chiaki]

I used to run the fix_linux_stack.py script AFTER the test was run and 
log was recorded.
I fed the log to the script as its standard input and the script spit 
out the decoded stack trace.
It no longer works this way even if I change line_re to match the 
stack trace in mochitest log. Hmm...




I hasten to add that I did this under linux.

Also, there has been a python dependency issues related to repackaging 
of python modules under Debian. So that may affect the operaton. But in 
that case, I would see an error/warning message somewhere. I did not see 
one.


TIA

Chiaki

On 2020/04/28 16:50, ISHIKAWA,chiaki wrote:

Dear Nick,

I must have been doing something very stupid, but can't figure out 
exactly.


I think after this change of fix-stack script(s), the local TB's 
mochitest log's NS_ASSERTION (or is it MOZ_ASSERT?) stack trace cannot 
be decoded.


I scratched my head and figured out that there is something wrong in 
the stack trace of mochitest.

They look like this. I pick up the first stacktrace in the log.

 0:21.30 GECKO(727384) {debug}:ASSERTION aISizeType=2, BTLS_FINALSIZE=2
 0:21.30 GECKO(727384) aISize=0, guess_min=10203, guess_pref=10203
 0:21.30 GECKO(727384) [727384, Main Thread] ###!!! ASSERTION: Table 
inline-size is less than the sum of its columns' min inline-sizes: 
'!(aISizeType == BTLS_FINAL_ISIZE && aISize < guess_min)', file 
/NEW-SSD/NREF-COMM-CENTRAL/mozilla/layout/tables/BasicTableLayoutStrategy.cpp, 
line 767
Initializing stack-fixing for the first stack frame, this may take a 
while...

 0:28.61 INFO runtests.py | Waiting for browser...
 0:28.98 GECKO(727384) #01: ??? 
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x5ab5921)
 0:28.99 GECKO(727384) #02: ??? 
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x5ab7f34)
 0:28.99 GECKO(727384) #03: ??? 
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x590de76)
 0:28.99 GECKO(727384) #04: ??? 
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x5a94c2f)
 0:28.99 GECKO(727384) #05: ??? 
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x5ab1589)


You can see the telltale sign of fix-stack being invoked.: 
"Initializing stack-fixing for the first stack frame, this may take a 
while..."


However, the stack is not decoded as you can see in the log.

The stack dump comes from the NS_ASSERTION() in the following.
I print out the strange value for debugging before NS_ASSERTION().

#ifdef DEBUG
    if((aISizeType == BTLS_FINAL_ISIZE && aISize < guess_min)) {
  fprintf(stderr,
 "{debug}:ASSERTION aISizeType=%d, BTLS_FINALSIZE=%d\n",
 aISizeType, BTLS_FINAL_ISIZE);
  fprintf(stderr,"aISize=%d, guess_min=%d, guess_pref=%d\n",
 aISize, guess_min, guess_pref);
    }
#endif
    NS_ASSERTION(!(aISizeType == BTLS_FINAL_ISIZE && aISize < guess_min),
 "Table inline-size is less than the "
 "sum of its columns' min inline-sizes");
    if (aISize < guess_min_pct) {


Now, I look at fix_stacks.py and immediately noticed something strange 
there.

The file is  mozilla/tools/rb/fix_stacks.py.:

# Matches lines produced by MozFormatCodeAddress(), e.g.
# `#01: ???[tests/example +0x43a0]`.
line_re = re.compile("#\d+: .+\[.+ \+0x[0-9A-Fa-f]+\]")

You see the re (Regular Expression) does not match the line in 
mochitest log produced by NS_ASSERTION().


What gives?

Also, another point is this.
I used to run the fix_linux_stack.py script AFTER the test was run and 
log was recorded.
I fed the log to the script as its standard input and the script spit 
out the decoded stack trace.
It no longer works this way even if I change line_re to match the 
stack trace in mochitest log. Hmm...


Any idea re what went south?

TIA

Chiaki

On 2020/03/06 7:52, Nicholas Nethercote wrote:

Hi all,

I have written a new stack-fixing tool, called `fix-stacks`, which
symbolizes stack traces produced by Firefox.

`fix-stacks` is intended to replace our existing stack-fixing scripts,
`fix_{linux,macosx}_stack.py` and `fix_stack_using_bpsyms.py`, which are
used (a) on many test outputs, both locally and on automation, and 
(b) by

DMD, Firefox's built-in heap profiler.

`fix-stacks` is now installed by `mach bootstrap` on Windows, Mac, and
Linux. It is usable from Python code via the `fix_stacks.py` wrapper
script. Its code is at https://github.com/mozilla/fix-stacks/.

In bug 1604095 I replaced the use of `fix_{linux,macosx}_stack.py` with
`fix_stacks.py` for DMD, with the following benefits.

* On Linux, stack-fixing of DMD output files (which occurs when you run
`dmd.py`) is roughly 100x faster. On my Linux box I saw reductions from
20-something minutes to ~13 seconds.

* On Mac, stack-fixing of DMD output files on Mac is roughly 10x 
faster. On

my Mac laptop I saw reductions from ~7 minutes to ~45 seconds.

* On Windows, stack-fixing of DMD output files now occurs. 

Re: New and improved stack-fixing

[ISHIKAWA,chiaki]

Dear Nick,

I must have been doing something very stupid, but can't figure out exactly.

I think after this change of fix-stack script(s), the local TB's 
mochitest log's NS_ASSERTION (or is it MOZ_ASSERT?) stack trace cannot 
be decoded.


I scratched my head and figured out that there is something wrong in the 
stack trace of mochitest.

They look like this. I pick up the first stacktrace in the log.

 0:21.30 GECKO(727384) {debug}:ASSERTION aISizeType=2, BTLS_FINALSIZE=2
 0:21.30 GECKO(727384) aISize=0, guess_min=10203, guess_pref=10203
 0:21.30 GECKO(727384) [727384, Main Thread] ###!!! ASSERTION: Table 
inline-size is less than the sum of its columns' min inline-sizes: 
'!(aISizeType == BTLS_FINAL_ISIZE && aISize < guess_min)', file 
/NEW-SSD/NREF-COMM-CENTRAL/mozilla/layout/tables/BasicTableLayoutStrategy.cpp, 
line 767
Initializing stack-fixing for the first stack frame, this may take a 
while...

 0:28.61 INFO runtests.py | Waiting for browser...
 0:28.98 GECKO(727384) #01: ??? 
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x5ab5921)
 0:28.99 GECKO(727384) #02: ??? 
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x5ab7f34)
 0:28.99 GECKO(727384) #03: ??? 
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x590de76)
 0:28.99 GECKO(727384) #04: ??? 
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x5a94c2f)
 0:28.99 GECKO(727384) #05: ??? 
(/NEW-SSD/moz-obj-dir/objdir-tb3/dist/bin/libxul.so + 0x5ab1589)


You can see the telltale sign of fix-stack being invoked.: "Initializing 
stack-fixing for the first stack frame, this may take a while..."


However, the stack is not decoded as you can see in the log.

The stack dump comes from the NS_ASSERTION() in the following.
I print out the strange value for debugging before NS_ASSERTION().

#ifdef DEBUG
    if((aISizeType == BTLS_FINAL_ISIZE && aISize < guess_min)) {
  fprintf(stderr,
 "{debug}:ASSERTION aISizeType=%d, BTLS_FINALSIZE=%d\n",
 aISizeType, BTLS_FINAL_ISIZE);
  fprintf(stderr,"aISize=%d, guess_min=%d, guess_pref=%d\n",
 aISize, guess_min, guess_pref);
    }
#endif
    NS_ASSERTION(!(aISizeType == BTLS_FINAL_ISIZE && aISize < guess_min),
 "Table inline-size is less than the "
 "sum of its columns' min inline-sizes");
    if (aISize < guess_min_pct) {


Now, I look at fix_stacks.py and immediately noticed something strange 
there.

The file is  mozilla/tools/rb/fix_stacks.py.:

# Matches lines produced by MozFormatCodeAddress(), e.g.
# `#01: ???[tests/example +0x43a0]`.
line_re = re.compile("#\d+: .+\[.+ \+0x[0-9A-Fa-f]+\]")

You see the re (Regular Expression) does not match the line in mochitest 
log produced by NS_ASSERTION().


What gives?

Also, another point is this.
I used to run the fix_linux_stack.py script AFTER the test was run and 
log was recorded.
I fed the log to the script as its standard input and the script spit 
out the decoded stack trace.
It no longer works this way even if I change line_re to match the stack 
trace in mochitest log. Hmm...


Any idea re what went south?

TIA

Chiaki

On 2020/03/06 7:52, Nicholas Nethercote wrote:

Hi all,

I have written a new stack-fixing tool, called `fix-stacks`, which
symbolizes stack traces produced by Firefox.

`fix-stacks` is intended to replace our existing stack-fixing scripts,
`fix_{linux,macosx}_stack.py` and `fix_stack_using_bpsyms.py`, which are
used (a) on many test outputs, both locally and on automation, and (b) by
DMD, Firefox's built-in heap profiler.

`fix-stacks` is now installed by `mach bootstrap` on Windows, Mac, and
Linux. It is usable from Python code via the `fix_stacks.py` wrapper
script. Its code is at https://github.com/mozilla/fix-stacks/.

In bug 1604095 I replaced the use of `fix_{linux,macosx}_stack.py` with
`fix_stacks.py` for DMD, with the following benefits.

* On Linux, stack-fixing of DMD output files (which occurs when you run
`dmd.py`) is roughly 100x faster. On my Linux box I saw reductions from
20-something minutes to ~13 seconds.

* On Mac, stack-fixing of DMD output files on Mac is roughly 10x faster. On
my Mac laptop I saw reductions from ~7 minutes to ~45 seconds.

* On Windows, stack-fixing of DMD output files now occurs. (It previously
did not because there is no `fix_window_stacks.py` script.) This means that
DMD is now realistically usable on Windows without jumping through hoops to
use breakpad symbols.

There is more work to be done. Soon, I plan to:

* use `fix-stacks` on test outputs (in `utils.py` and `automation.py.in`);

* re-enable stack fixing on Mac test runs on local builds, which is
currently disabled because it is so slow;

* add breakpad symbol support to `fix-stacks`;

* remove the old scripts.

The tree of relevant bugs can be seen at
https://bugzilla.mozilla.org/showdependencytree.cgi?id=1596292_resolved=1
.

The stack traces produced by `fix-stacks` are sometimes different to those
produced by the old stack-fixers. In my 

Re: Deprecation notice - change to Treeherder's jobdetail API

[ISHIKAWA,chiaki]

On 2020/04/11 3:03, Steve Fink wrote:

On 4/10/20 10:06 AM, Andrew McCreight wrote:
On Thu, Apr 9, 2020 at 6:51 PM Sarah Clements  
wrote:



Hello,

As part of maintenance work and performance improvements to Treeherder,
we're making changes to some of the data that's stored in our 
database. On *April

30th*, we will no longer be retrieving uploaded artifacts for jobs and
storing them in the JobDetail table.

Uploaded artifacts can be retrieved via a taskcluster API, with the
appropriate rootUrl, taskId and runId. TaskId and runId (synonymous 
with
retryId in Treeherder) can be surfaced from any of our /jobs/ API's. 
See

this
 


for implementation details.

As somebody who has dumped files to MOZ_UPLOAD_DIR and used the job 
detail

tab to download those files (just this week, in fact!), but has only a
vague idea of what the "taskcluster API" even is, let alone a 
"rootURL" or
the rest of it, is there any introductory documentation about how I 
might
retrieve files written to MOZ_UPLOAD_DIR? Am I going to have to write 
some
kind of Python script to retrieve my files in the future? That GitHub 
issue

seems to require a lot of background information that I do not have.

Hi,

As an occasional patch contributor, I do a lot of local builds and tests 
to fix TB's bugs and deficiencies.


One thing I noticed lately is the reference to "MOZ_UPLOAD_DIR" during 
local build/test.
(I only started to notice this January. Considering that TB switched to 
mochitest from mozmill-based test completely in December, this is 
natural, I suppose. Mochitest must have used this variable for a long 
time while mozmill test did not.)


The variable is referenced during mochitest of TB and in the following 
warning during test immediately before ASSERT macro is invoked, or at 
the end of the whole test series.:


Failed to retrieve MOZ_UPLOAD_DIR env var

I hope not setting a value to MOZ_UPLOAD_DIR does not impact LOCAL test 
in any adverse way.


Thank you in advance for your attention.

Chiaki



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Bugzilla posting failure.

[ISHIKAWA, Chiaki]

Dear Emma,

Thank you very much for your e-mail.

I see.

- Now I understand "resolved as dup" hides the post.
(Just an idea: We may want to consider an option to see the original 
post instead of our own posts [duped to the original] if the original 
post is not resolved yet.)


Bugzilla is a very helpful tool, but from time to time, I see strange 
symptoms. But it is so rare that is why I was not quite sure what to do 
when the error screen showed up regarding the attachment is not posted, 
etc. I would have read the message very carefully and followed the 
instructions, but basically the error screen said that there was a 
problem of uploading the attachment, and it created an entry without the 
attachment (aside from the complete entry.).


To be honest, I could not grok what it said exactly regarding clicking 
the link in the error screen. I could not fathom what it exactly wanted 
to tell me. I hope BMO team can find the original source of that screen 
and make it easier for mere mortals to understand.


That is why I said the error screen was confusing and a bit misleading IMHO.

Should I create a bug entry in Bugzilla regarding this recent incident?
(Or maybe BMO team can figure this out quickly.)

TIA

Chiaki



On 2019/09/24 14:53, Emma Humphries wrote:

Moving dev-platform to bcc.

Thank you for your email.

You had posted a bug, https://bugzilla.mozilla.org/show_bug.cgi?id=1574787,
which was resolved as a duplicate of
https://bugzilla.mozilla.org/show_bug.cgi?id=1472757, so that would not
show up in My Dashboard.

I'll ask the BMO team to see if they can can find the error logged from
this most recent incident where you tried to create an attachment, got an
error message, but the attachment had been created.

For future reference, the best place to file bugs related to creating bugs
is
https://bugzilla.mozilla.org/enter_bug.cgi?product=bugzilla.mozilla.org=Bug+Creation%2FEditing
.

E C Humphries, Bugmaster


On Mon, Sep 23, 2019 at 8:10 PM ishikawa  wrote:


Hi,

Where is the proper forum to post a bug regarding bugzilla.mozilla.org?

I *THINK* I posted a bug regarding bugzilla failure a couple of months ago,
but for some reason, I could not find it in "My Dashboard" and so I am
posting the question here.

I experienced a bugzilla problem today and it resulted in a pair of
duplicated entry.
Problem was that
- bugzilla showed a message after initial posting that an attachment could
not be uploaded or something ( I wish I had recorded the message), but
- then I realize that after I added an attachment to the
   resulting page WITHOUT attachment that a complete entry had already been
created ...

1. Completely entry created despite the confusing error(?) message from
bugzilla.
https://bugzilla.mozilla.org/show_bug.cgi?id=1583415

2. After I clear the error message, I got a link to the following entry
that
lacked the attachment, and so I  added the attachment. See comment 1 there.
https://bugzilla.mozilla.org/show_bug.cgi?id=1583416

I only learned the existence of the URL 1 above AFTER I added the
attachment
since bugzilla said something about the failure of upload (?).

I have not deleted the duplicate entries yet since the existence of the
entry and the log on the bugzilla server may help whoever wants to debug
the
issue to find the cause of the strange bogus error message and what
happened
after I clicked a link the error message screen.
(I DID, however, mark 1583416 as a dup of 1583415)

At least, the error message screen, which I failed to capture,  ought to be
rewritten IMHO to inform the poster what has happened correctly in an
easy-to-comprehend manner.

Thank you for your attention in advance.

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Bug 1578296 JavaScript error: resource://gre/modules/SessionStoreFunctions.jsm, line 392: NS_ERROR_FILE_NOT_FOUND

[ISHIKAWA,chiaki]

Hi,

I filed the following entry to bugzilla:
https://bugzilla.mozilla.org/show_bug.cgi?id=1578296

Bug 1578296
Many errors of the form JavaScript error: 
resource://gre/modules/SessionStoreFunctions.jsm, line 392: 
NS_ERROR_FILE_NOT_FOUND during |make mozmill| testing of Full Debug of 
C-C TB.


This error started to appear in mid-July during the local test of |make 
mozmill| with the FULL DEBUG version of C-C TB.

The number of such error lines is large: more than 400.

Problem is that there is no session-related component for Product 
Thunderbird.
I also noticed a couple of bugs that seem to be related to this issue in 
Product Firefox Session Restore component.

So I filed the bug to Product Firefox Session Restore component.

But if this is not appropriate,  please change the assignment to a more 
proper component.

It may be that we may have to create Session Restore component in TB (?)
[However, I am not sure what the session would be for TB. Maybe the last 
visited message in each folder and what folder was last visible in the 
three pane layout?]


TIA

Chiaki


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Help: why won't Thunderbird display mail in mochitest conditions?

[ISHIKAWA,chiaki]

Hi, those familiar with test infrastructure

I am sorry that I am piggybacking my question/comment on a not directly 
related topic.


During xpcshell test of TB, no display (of X11, say, under linux) is 
shown. Is this a design decision and is to be expected?


If TB tries to show an error message using X11 dialog when it encounters 
an error condition,

it fails miserably during xpcshell test since no display is available.
I had to disable a code to show an error using a dialog because of this 
issue with xpcshell test  some years ago.


So I am just curious if the decision to inhibit any graphical output 
during xpcshell test is a conscious decision or not.
If there is a global variable/function that can tell whether TB is 
running under xpcshell test, I could
have conditionally enable the graphical error dialog and still pass the 
xpcshell test.


TIA

Chiaki

On 2019/08/27 7:40, Geoff Lankow wrote:

Hi

Over the past year or so, I've been adding mochitests for new 
Thunderbird features. It's recently occurred to me that in a 
mochitest, Thunderbird does not display mail messages. Not even the 
message header list, just a blank rectangle where the message should be.


Obviously this is quite important as displaying messages is 
Thunderbird's primary function. But I don't understand the reason.


I expect that it has something to do with message URLs, which are of 
the form mailbox:///path/to/mailbox?number=1234.


I know that mochitest does things to network access to prevent tests 
from accessing the internet, but that doesn't seem to be the reason as 
I can load the URL using fetch.


Is there some logic in docshell that behaves differently in a test? As 
far as I can work out, this code [1] is a part of the loading process, 
and if docShell->LoadURI fails that would explain why nothing further 
happens. (IANA core hacker, excuse my ignorance!)


GL

[1] 
https://searchfox.org/comm-central/rev/753e6d9df9d7b9a989d735b01c8b280eef388bab/mailnews/local/src/nsMailboxService.cpp#205

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Must we rebuild all our rust code constantly?

[ISHIKAWA,chiaki]

Here is my own answer to clean up the mess of version mismatches, etc.

I did the following.

1. I removed ~/.mozbuild and ~/.cargo directory and
re-ran |make bootstrap|.
(I am not sure if removing ~/.cargo directory was a good idea.)

2. I revered my local changes before I update the M-C and C-C (./comm).

3. Then I noticed mozilla/Cargo.lock has been modified when I tried to 
reapply my local changes.

This file was presumably changed when I did my own sccache installation.

So I reverted the change to Cargo.lock, and now it seems that the 
compilation is proceeding... (Not sure if the result is OK.)



4. When I did this, I noticed there is sccache 0.2.11-Alpha.0 under 
~/.mozbuild/sccache directory.
Obviously, I was using my own installed older version of sccache which I 
installed many months ago.
So I added the path to this sccache in my script for building TB and it 
seems to be compiling TB.


I added the following in my mozconfig for TB, but I am not sure if this 
is the right way to disable incremental compilation.


mk_add_options "export CARGO_INCREMENTAL=0"

TIA

On 2019/08/22 11:22, ISHIKAWA,chiaki wrote:
Well, I have a problem now after trying to update sccache just in case 
I need a new version in the future.


I did the following:

cargo install --force sccache

(I was not so sure of what the proper update procedure of already 
installed package. sccache 2.0.8-alpha-something was already installed.)


Then when I try to compile TB using the command script which worked 
for more than a few years suddenly printed the following and failed. I 
think I have messed up my cargo metadata somehow by the above command.


I have run |mach bootstrap| to see if it fixes the issue. It did not.

ERROR: Couldn't execute `cargo metadata` with manifest 
"/NREF-COMM-CENTRAL/mozilla/toolkit/library/rust/Cargo.toml": 
Metadata(Output { status: ExitStatus(ExitStatus(25856)), stdout: "", 
stderr: "error: failed to select a version for the requirement `libc = 
\"= 0.2.62\"`\n  candidate versions found which didn\'t match: 
0.2.60\n  location searched: directory source 
`/NREF-COMM-CENTRAL/mozilla/third_party/rust` (which is replacing 
registry `https://github.com/rust-lang/crates.io-index`)\nrequired by 
package `mozjs_sys v0.0.0 
(/NREF-COMM-CENTRAL/mozilla/js/src)`\nperhaps a crate was updated and 
forgotten to be re-vendored?\n" })
ERROR: Couldn't generate bindings for 
/NREF-COMM-CENTRAL/mozilla/toolkit/library/rust.

make[4]: *** [backend.mk:72: .deps/ServoStyleConsts.h.stub] Error 1
make[3]: *** [/NREF-COMM-CENTRAL/mozilla/config/recurse.mk:101: 
layout/style/export] Error 2

make[3]: *** Waiting for unfinished jobs
ERROR: Couldn't execute `cargo metadata` with manifest 
"/NREF-COMM-CENTRAL/mozilla/toolkit/library/rust/Cargo.toml": 
Metadata(Output { status: ExitStatus(ExitStatus(25856)), stdout: "", 
stderr: "    Blocking waiting for file lock on package cache 
lock\nerror: failed to select a version for the requirement `libc = 
\"= 0.2.62\"`\n  candidate versions found which didn\'t match: 
0.2.60\n  location searched: directory source 
`/NREF-COMM-CENTRAL/mozilla/third_party/rust` (which is replacing 
registry `https://github.com/rust-lang/crates.io-index`)\nrequired by 
package `mozjs_sys v0.0.0 
(/NREF-COMM-CENTRAL/mozilla/js/src)`\nperhaps a crate was updated and 
forgotten to be re-vendored?\n" })
ERROR: Couldn't generate bindings for 
/NREF-COMM-CENTRAL/mozilla/toolkit/library/rust.
make[4]: *** [backend.mk:11: .deps/webrender_ffi_generated.h.stub] 
Error 1
make[3]: *** [/NREF-COMM-CENTRAL/mozilla/config/recurse.mk:101: 
gfx/webrender_bindings/export] Error 2


Does anyone have a clue how to restore order here?

TIA

Chiaki


On 2019/08/21 4:33, Markus Stange wrote:

On 2019-08-19 8:11 p.m., Dave Townsend wrote:
Thanks to a tip I've tracked this down. This seems to only be the 
case when
I have sccache enabled. Disabling it gives me nice quick incremental 
builds

again.


What's your sccache version? I think you may be hitting the following 
sccache bug which has been fixed already:

https://github.com/mozilla/sccache/issues/436

I'm using sccache 0.2.9 and I don't see rust code rebuilding constantly.

Markus
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform






___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Must we rebuild all our rust code constantly?

[ISHIKAWA,chiaki]

Well, I have a problem now after trying to update sccache just in case I 
need a new version in the future.


I did the following:

cargo install --force sccache

(I was not so sure of what the proper update procedure of already 
installed package. sccache 2.0.8-alpha-something was already installed.)


Then when I try to compile TB using the command script which worked for 
more than a few years suddenly printed the following and failed. I think 
I have messed up my cargo metadata somehow by the above command.


I have run |mach bootstrap| to see if it fixes the issue. It did not.

ERROR: Couldn't execute `cargo metadata` with manifest 
"/NREF-COMM-CENTRAL/mozilla/toolkit/library/rust/Cargo.toml": 
Metadata(Output { status: ExitStatus(ExitStatus(25856)), stdout: "", 
stderr: "error: failed to select a version for the requirement `libc = 
\"= 0.2.62\"`\n  candidate versions found which didn\'t match: 0.2.60\n  
location searched: directory source 
`/NREF-COMM-CENTRAL/mozilla/third_party/rust` (which is replacing 
registry `https://github.com/rust-lang/crates.io-index`)\nrequired by 
package `mozjs_sys v0.0.0 (/NREF-COMM-CENTRAL/mozilla/js/src)`\nperhaps 
a crate was updated and forgotten to be re-vendored?\n" })
ERROR: Couldn't generate bindings for 
/NREF-COMM-CENTRAL/mozilla/toolkit/library/rust.

make[4]: *** [backend.mk:72: .deps/ServoStyleConsts.h.stub] Error 1
make[3]: *** [/NREF-COMM-CENTRAL/mozilla/config/recurse.mk:101: 
layout/style/export] Error 2

make[3]: *** Waiting for unfinished jobs
ERROR: Couldn't execute `cargo metadata` with manifest 
"/NREF-COMM-CENTRAL/mozilla/toolkit/library/rust/Cargo.toml": 
Metadata(Output { status: ExitStatus(ExitStatus(25856)), stdout: "", 
stderr: "    Blocking waiting for file lock on package cache 
lock\nerror: failed to select a version for the requirement `libc = \"= 
0.2.62\"`\n  candidate versions found which didn\'t match: 0.2.60\n  
location searched: directory source 
`/NREF-COMM-CENTRAL/mozilla/third_party/rust` (which is replacing 
registry `https://github.com/rust-lang/crates.io-index`)\nrequired by 
package `mozjs_sys v0.0.0 (/NREF-COMM-CENTRAL/mozilla/js/src)`\nperhaps 
a crate was updated and forgotten to be re-vendored?\n" })
ERROR: Couldn't generate bindings for 
/NREF-COMM-CENTRAL/mozilla/toolkit/library/rust.

make[4]: *** [backend.mk:11: .deps/webrender_ffi_generated.h.stub] Error 1
make[3]: *** [/NREF-COMM-CENTRAL/mozilla/config/recurse.mk:101: 
gfx/webrender_bindings/export] Error 2


Does anyone have a clue how to restore order here?

TIA

Chiaki


On 2019/08/21 4:33, Markus Stange wrote:

On 2019-08-19 8:11 p.m., Dave Townsend wrote:
Thanks to a tip I've tracked this down. This seems to only be the 
case when
I have sccache enabled. Disabling it gives me nice quick incremental 
builds

again.


What's your sccache version? I think you may be hitting the following 
sccache bug which has been fixed already:

https://github.com/mozilla/sccache/issues/436

I'm using sccache 0.2.9 and I don't see rust code rebuilding constantly.

Markus
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Must we rebuild all our rust code constantly?

[ISHIKAWA, Chiaki]

On 2019/08/21 3:52, Eric Rahm wrote:

mach clobber --full


Thank you for the tips.

I will try this.

At the same time, I have a feeling that the debug symbol that rustc 
generates may be a tad bigger than I would like it to be, but I need to 
investigate more about this.


Chiaki
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Must we rebuild all our rust code constantly?

[ISHIKAWA,chiaki]

On 2019/08/20 14:32, Kris Maglione wrote:

On Tue, Aug 20, 2019 at 02:23:06PM +0900, ISHIKAWA,chiaki wrote:

On 2019/08/20 9:11, Dave Townsend wrote:
Thanks to a tip I've tracked this down. This seems to only be the 
case when
I have sccache enabled. Disabling it gives me nice quick incremental 
builds

again. Of course that isn't an ideal solution but it will do for now.

On Mon, Aug 19, 2019 at 1:55 PM Dave Townsend 
 wrote:



For a couple of weeks now I've seen that any attempt to build Firefox,
even incremental builds seem to rebuild an awful lot of rust code. 
I found

this in the source which seems to suggest why:
https://searchfox.org/mozilla-central/source/config/makefiles/rust.mk#238. 

But, this means that now an incremental build with a couple of code 
change
that have no impact on rust is taking upwards of 4 minutes to 
complete in
comparison to around 40 seconds, and the log file is full of cargo 
output.

I've heard similar comments from other developers.

This is a pretty big increase in the time to compile and test and is
really slowing down my work. Is there any way we can avoid this?






I am using linux for local development and noticed something similar.

So I should disable sccache (!?).


For what it's worth, Rust is now configured to use incremental 
compilation, which has its own cache which isn't cleared after 
clobber, so sccache isn't actually needed anymore. Ordinary ccache 
should be sufficient.





I was not sure where this incremental compilation cache is located.
Sure enough on my computer it seems to live under
MOZ_OBJ/x86_64-unknown-linux-gnu/debug/incremental

I have a serious question: does this incremental cache have a control 
mechanism to purge old entries?


I have been hit with file system overflow errors quite frequently since 
early summer and I figure
this is due to the bloated 
MOZ_OBJ/x86_64-unknown-linux-gnu/debug/incremental directory.


It is now 26.6 GB with 67840 items according baobab file system usage 
monitor.


26.6 GB for this incremental cache alone is way too much number that was 
suggested by mozilla development document before.

In comparsion, ccache I use is usable with max size of 5GB
as set by
CCACHE_SIZE=5.0GB
in my script.

On my local PC, I see that the file directory size (with the sizes of 
subdirectories added to it) is as follows:


MOZ_OBJ is 45.8 GB large. (as of now: I had to stop TB compilation due 
to the 50 GB smallish partition overflow where this directory is located.)

MOZ_OBJ/x86_64-unknown-linux-gnu/ is 33.9 GB large .
MOZ_OBJ/x86_64-unknown-linux-gnu/debug/  is 33.9 GB large.
    I think /debug/ is because I create full debug version of TB locally.
MOZ_OBJ/MOZ_OBJ/x86_64-unknown-linux-gnu/debug/incremental is 26.5 GB large.

I need a way to shrink this incremental cache reasonably.
It seems this incremental directory grows without limit. Or, it does not 
trim old unusable cache files automatically.


TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Must we rebuild all our rust code constantly?

[ISHIKAWA,chiaki]

On 2019/08/20 9:11, Dave Townsend wrote:

Thanks to a tip I've tracked this down. This seems to only be the case when
I have sccache enabled. Disabling it gives me nice quick incremental builds
again. Of course that isn't an ideal solution but it will do for now.

On Mon, Aug 19, 2019 at 1:55 PM Dave Townsend  wrote:


For a couple of weeks now I've seen that any attempt to build Firefox,
even incremental builds seem to rebuild an awful lot of rust code. I found
this in the source which seems to suggest why:
https://searchfox.org/mozilla-central/source/config/makefiles/rust.mk#238.
But, this means that now an incremental build with a couple of code change
that have no impact on rust is taking upwards of 4 minutes to complete in
comparison to around 40 seconds, and the log file is full of cargo output.
I've heard similar comments from other developers.

This is a pretty big increase in the time to compile and test and is
really slowing down my work. Is there any way we can avoid this?






I am using linux for local development and noticed something similar.

So I should disable sccache (!?).



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: clang version on tryserver?

[ISHIKAWA,chiaki]

(sorry for top posting)

Dear Jan and Steve,

Problems solved.

After a few trials and errors, I figured there are TWO similar
constructs that, depending on the order of parameter evaluation, may
not pass what is desired due to unexpected modification during
parameter evaluation.

I changed my local patches as follows and now the errors during test
run on the tryserver (using clang) match the ones I see locally (using
GCC).

Now I can home in the few remaining errors possibly introduced my
patches. (They may be timing-dependent issues uncovered by my patch,
but who knows.)

Thank you again!

Chiaki

PS: I have found that gcc-8 has a more feature-rich format string
checking: it warns if a complete output would be longer than the
buffer's length passed to snprintf(). Amazing. However, since I turn
some warnings including this one into compilation errors (to avoid the
unexpected failure on the tryserver while I can build successfully), I had
to tweak the webrtc code in a few places.


Modified parts:

comm/mailnews/local/src/nsPop3Sink.cpp

  //
  // clang and GCC has a different order of argument evaluation.
  // We have to save m_outFileSream because
  // geter_AddRefs(m_outFileStream) may trash it depending on the
  // order of evaluation of passed arguments!
  //
  nsCOMPtr tmp_p = m_outFileStream;
  rv1 = NS_NewBufferedOutputStream(getter_AddRefs(m_outFileStream),
   tmp_p.forget(),
   64 * 1024 );



A bit overkill.:

/comm/mailnews/local/src/nsLocalMailFolder.cpp:

  nsCOMPtr tmp_in = mCopyState->m_fileStream;
  nsCOMPtr tmp_out;

  nsresult rv2
    = NS_NewBufferedOutputStream(getter_AddRefs(tmp_out),
 tmp_in.forget(),
 64 * 1024 );
  mCopyState->m_fileStream = tmp_out;

On 2019/07/31 5:45, ISHIKAWA,chiaki wrote:

Dear Steve,

Thank you for your analysis and suggestion for moving to gcc-8.

I would change my local GCC compiler to gcc-8, and g++-8.

Also, I would modify the offending code so that
the evaluation order of the arguments does not get in the way.

I still yet to do a few more tasks, but I think the mystery of the 
difference between the clang version and GCC version, which got me 
wondering for the last several days,

has been solved by your and Jan's analysis.

I *think* I was assuming left-to-right argument ordering somehow.
I probably should use clang occasionally to put my implicit 
assumptions gleaned from years of GCC usage to torture test, so to speak.


Thank you very much !!!

Chiaki

PS: Well, for whatever the reason, I also found a problem with cflow-8 
(maybe my particular setting of C compilation time macros, etc.).

That would be an unexpected bonus. :-)


On 2019/07/31 0:15, Steve Fink wrote:

On 7/30/19 6:20 AM, Jan de Mooij wrote:

On Tue, Jul 30, 2019 at 1:51 PM ISHIKAWA,chiaki 
wrote:


    nsresult rv2
  = 
NS_NewBufferedOutputStream(getter_AddRefs(mCopyState->m_fileStream),
mCopyState->m_fileStream.forget(),  <=== It seems this can be 
nullptr in

clang-8 version???
   64 * 1024 );

This looks like it could be caused by Clang evaluating your 
arguments in a
different order from GCC (see also [0]). If Clang evaluates 
left-to-right,
that getter_AddRefs might null out your m_fileStream before we 
evaluate the

m_fileStream.forget().

Note that, if I'm reading the spec correctly, it's not just the order 
of evaluation between arguments. Before c++17, the evaluation of 
different arguments could be interleaved arbitrarily. It looks like 
c++17 changes argument evaluation to be "indeterminately sequenced" 
instead of the previous "unsequenced". Indeterminately sequenced 
means they can happen in any order, but *cannot* be interleaved as 
they are now. I don't think that comes up in your specific example, 
but it's closely related.


https://en.cppreference.com/w/cpp/language/eval_order

I know the interleaving happens in practice because the GC rooting 
hazard analysis ran into this: a Rooted value passed to a function 
was extracted into a temporary, another argument's evaluation 
triggered a GC that moved the value, and then the now-invalid 
temporary was passed into the function. It looks like c++17 will 
prevent this from happening.


I guess that gives me motivation to finish updating the rooting 
analysis to gcc8, which has c++17 support. Which is relevant to you, 
since you're compiling with gcc: the minimum gcc version is going to 
be bumped to gcc7 soon, and it looks like we'll only be testing with 
gcc8 in the continuous integration system, so it would be safest to 
use a minimum of gcc 8. (gcc 7.4.0 has a bug that breaks the rooting 
analysis, so it will go straight to gcc 8.)






___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform



_

Re: clang version on tryserver?

[ISHIKAWA,chiaki]

Dear Steve,

Thank you for your analysis and suggestion for moving to gcc-8.

I would change my local GCC compiler to gcc-8, and g++-8.

Also, I would modify the offending code so that
the evaluation order of the arguments does not get in the way.

I still yet to do a few more tasks, but I think the mystery of the 
difference between the clang version and GCC version, which got me 
wondering for the last several days,

has been solved by your and Jan's analysis.

I *think* I was assuming left-to-right argument ordering somehow.
I probably should use clang occasionally to put my implicit assumptions 
gleaned from years of GCC usage to torture test, so to speak.


Thank you very much !!!

Chiaki

PS: Well, for whatever the reason, I also found a problem with cflow-8 
(maybe my particular setting of C compilation time macros, etc.).

That would be an unexpected bonus. :-)


On 2019/07/31 0:15, Steve Fink wrote:

On 7/30/19 6:20 AM, Jan de Mooij wrote:

On Tue, Jul 30, 2019 at 1:51 PM ISHIKAWA,chiaki 
wrote:


    nsresult rv2
  = 
NS_NewBufferedOutputStream(getter_AddRefs(mCopyState->m_fileStream),
mCopyState->m_fileStream.forget(),  <=== It seems this can be 
nullptr in

clang-8 version???
   64 * 1024 );

This looks like it could be caused by Clang evaluating your arguments 
in a
different order from GCC (see also [0]). If Clang evaluates 
left-to-right,
that getter_AddRefs might null out your m_fileStream before we 
evaluate the

m_fileStream.forget().

Note that, if I'm reading the spec correctly, it's not just the order 
of evaluation between arguments. Before c++17, the evaluation of 
different arguments could be interleaved arbitrarily. It looks like 
c++17 changes argument evaluation to be "indeterminately sequenced" 
instead of the previous "unsequenced". Indeterminately sequenced means 
they can happen in any order, but *cannot* be interleaved as they are 
now. I don't think that comes up in your specific example, but it's 
closely related.


https://en.cppreference.com/w/cpp/language/eval_order

I know the interleaving happens in practice because the GC rooting 
hazard analysis ran into this: a Rooted value passed to a function was 
extracted into a temporary, another argument's evaluation triggered a 
GC that moved the value, and then the now-invalid temporary was passed 
into the function. It looks like c++17 will prevent this from happening.


I guess that gives me motivation to finish updating the rooting 
analysis to gcc8, which has c++17 support. Which is relevant to you, 
since you're compiling with gcc: the minimum gcc version is going to 
be bumped to gcc7 soon, and it looks like we'll only be testing with 
gcc8 in the continuous integration system, so it would be safest to 
use a minimum of gcc 8. (gcc 7.4.0 has a bug that breaks the rooting 
analysis, so it will go straight to gcc 8.)






___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: clang version on tryserver?

[ISHIKAWA,chiaki]

Dear Jan,

I am glad that I included the snippet of code here.

I should have known.

I would try to work around this, well I would try, and see if I can 
avoid the

runtime error on tryserver (!).

Thanks a million!

Chiaki

On 2019/07/30 22:20, Jan de Mooij wrote:

On Tue, Jul 30, 2019 at 1:51 PM ISHIKAWA,chiaki 
wrote:


nsresult rv2
  = NS_NewBufferedOutputStream(getter_AddRefs(mCopyState->m_fileStream),
mCopyState->m_fileStream.forget(),  <=== It seems this can be nullptr in
clang-8 version???
   64 * 1024 );


This looks like it could be caused by Clang evaluating your arguments in a
different order from GCC (see also [0]). If Clang evaluates left-to-right,
that getter_AddRefs might null out your m_fileStream before we evaluate the
m_fileStream.forget().

Hope this helps,
Jan

[0]
https://stackoverflow.com/questions/15440833/g-vs-intel-clang-argument-passing-order

Chiaki


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

clang version on tryserver?

[ISHIKAWA,chiaki]

Hi,

What is the version of clang used on tryserver?

The local clang-8 I have on my linux PC crashes due to internal compiler 
error during optimization pass, and I want to know what version is used 
on tryserver.


I have been using GCC for compiling TB locally and intend to do so just 
because the diversity of compilers would help us catch subtle issues in 
source code.
But recently I noticed a subtle binary behavior difference of the local 
binary during the testing locally and of the tryserver binary during 
testing on the trysever.

I need to figure out what happens using clang compiler locally.

That is why I want to know the clang-xxx version on the tryserver.

Detailed Background:

After recreating my local patches to match the great clang-reformatting 
of the source tree after so many months,
I was happy to see that my patches for TB to enable buffering for 
writing messages to files are almost done.
It passes |make mozmill| with only a couple of errors [this was not the 
case in last July time frame. I saw many, probably 400+ errors then, 
that made me give up for the effort for a few months.], and I now see 
four+ errors of xpcshell-test locally.



Now though, I am stunned to see MANY crashes triggered by MOZ_ASSERT on 
tryserver runs.


For example, you can see such instances in

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=258708674

In the xpcshell test for linux64 debug version, you see failed xpcshell 
test, and when you click on X1,


https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=258708674=try-comm-central=2225

you will see an error such as follows:

20:10:35 WARNING - PROCESS-CRASH | 
comm/mailnews/imap/test/unit/test_imapFilterActions.js | application 
crashed [@ nsBufferedStream::Init(nsISupports*, unsigned int)]


I though great, this will help me to focus on where I reset some stream 
variables to nullptr too early, etc. I am setting stream pointers to 
nullptr after closing it to avoid

double Close() calls on the underlying stream.

EXCEPT FOR one thing. This assert does NOT trigger locally AT ALL.
I am very puzzled.

I am using linux AMD64 as local development platform.

After scratching my head for a while and experimented inserting variable 
dumps and compare the test runs locally and tryserver, I came to a 
conclusion.


I am using GCC for local development (gcc-7 and g++-7 to be exact) and 
tryserver presumably uses clang for compilation.
I am building full debug build.: I even pass -DDEBUG=1 on compiler line 
just in case.


I am afraid that there are probably very subtle difference between the 
two compilers.: GCC on my local PC does not produce code that will 
trigger the MOZ_ASSERT() while

clang on tryserver does.

Why the different behavior?

1. GCC or g++ to be exact and  clang-8 may generate slightly different 
code for a valid piece of code. In this case, one of the compiler must 
be broken.


2. These compilers may handle "undefined behavior" of the source code in 
the parlance of "ISO C++" in a different way.


If the problem of different behavior of binaries locally and on 
tryserver is due to the reason 2, then I think we should be happy.
The difference of two compilers warns us of construct in mozilla source 
tree that results in undefined behavior at an early stage.
If one day, clang starts to handle the undefined behavior differently, 
we may be in for a surprise.

Such constructs won't be noticeable easily if we use only clang.

Anyway, I thought I would switch to clang for local compiling 
temporarily to see where the problem lies.


"./mach boostrap" installed clang-8 under 
/home/ishikawa/.mozbuild/clang/bin/clang-8 and so I thought I would use it.


However, this binary crashed miserably during optimization pass with a 
message that prompts me to report the issue to llvm.org, and I am in the 
process of doing it. (The bugzilla acount of llvm.org cannot be created 
automatically due to excessive spams. So I am waiting for the moderator 
there to open an account for me.)



BTW, the code in question that trigger the behavior is the compilation 
of the code here.

This line is added by my patch.

  // XXX TODO/FIXME should the second parameter needs to be
  //   mCopyState->m_fileStream.forget(), /* original */
  //   mCopyState->m_fileStream (simply?)
  //   The second form produces the following error.
  //   ... comm/mailnews/local/src/nsLocalMailFolder.cpp: In member 
function ‘nsresult nsMsgLocalMailFolder::InitCopyMsgHdrAndFileStream()’:
  //  .../comm/mailnews/local/src/nsLocalMailFolder.cpp:2002:46: error: 
could not convert 
‘((nsMsgLocalMailFolder*)this)->nsMsgLocalMailFolder::mCopyState->nsLocalMailCopyState::m_fileStream’ 
from ‘nsCOMPtr’ to ‘already_AddRefed’

  // mCopyState->m_fileStream,
  // ^~~~

  //   I suspect a subtle compiler issue: clang-c++ vs. g++

  PSTREAM2(mCopyState->m_fileStream, "before NS_NewBufferedOutputStream");
  nsresult rv2
    = 

Re: HG problem: abort: decoding near '9: error: ‘sse2::l': 'ascii' codec can't decode byte 0xe2 in position 213: ordinal not in range(128)

[ISHIKAWA,chiaki]

On 2018/12/03 1:18, Dave Townsend wrote:

I would report this to mercurial.



Thank you very much.

Looking forward to hearing the answer (maybe this could be 
HGENCODING=utf-8 after all. )


I have a suspicion that python's locale handler, especially the one that 
is involved in iconv-like code conversion has become either stricter or 
buggier in the transition from v2 to v3.


This is based on my observation of another program broken in subtle manner:
I have used a very useful visual merge tool called "meld" for more than 
a year until this spring.

That runs under linux and is written in python.
This fall, when I tried to use it again to manage the merge of local 
changes to mozilla TB so that they can be applied to newer TB source 
tree I fetched from comm-central, I notice that "meld" fail to handle a 
few local files that contain non-ASCII character. They are local memo 
files. It is very annoying and again the problem was traced to 
encode/decode in python runtime.


Thank you again.


On Sat, Dec 1, 2018 at 11:53 PM ISHIKAWA,chiaki 
wrote:


Hi,

I am developing mozilla patches locally under my local PC running linux.

I often include the GCC's error/warning message in the mercurial
commit message (in the second and subsequent lines, the first line is
for usage by mozilla's bugzilla procedure.) when my local patch is
created to address build error or suppress compiler warning.

It worked just fine for months/years.

After an upgrade of local tools under Debian GNU/Linux last month or
so, I just noticed that such a patch with GCC warning line could not
be re-applied after it was qpopped. (I use mercurial mqueue
extension.)

This happened yesterday. This is the error message.

note: commit message saved in .hg/last-message.txt
transaction abort!
rollback completed
abort: decoding near '9: error: ‘sse2::l': 'ascii' codec can't decode
byte 0xe2 in position 213: ordinal not in range(128)!

I was puzzled. After looking at the patch and the message above, I
finally figured out that that hg barfed on the proper quote character
just before "sse2::l" above. Can you spot the slightly tilted OPENING
quote character?

So I removed the GCC warning line from the commit message and the
patch could be applied. This is problematic because I have about a few
dozen patches (actually more) and some patches could not be re-applied
due to the error above. I had to temporarily removed the offending
patch(es). Luckily the offending patches are relatively independent
and taken out for the moment.
But this could not be done if the patches are interdependent and
must be applied in a certain order.

Either
- GCC began producing such a non-ascii character lately (it is v7 on
my linux PC)
[but I think I must have seen such characters long time before that],
- the "ascii" codec or whatever used by python has become very strict
(But why ascii? Should we not use utf-8 after all?)
[this could be an issue with setup of hg under my Debian GNU/Linux
distribution?] , or
- something in my locale/character setting under Debian GNU/Linux
broke lately due to the upgrade of local packages.

My point is that I have NOT changed in my personal setting at all
(unless |./mach bootstrap| I occasionally run to make sure that I have
the latest rust and other tools did something funny to my .hgrc).
Only the tools and system-wide setting could have changed mysteriously
from my point of view.

I found a similar bug report in the following open source project:
https://github.com/schacon/hg-git/issues/27

I have not tried the HGENCODING=utf-8 fix mentioned in the above URL
for fear of breaking existing patches against my local copy of the
mozilla-central and comm-central source tree.

I wonder what would be the ultimate correct fix or
procedure to avoid/suppress this error.

I am posting this to warn developers who *MAY* see similar issues,
and to figure out where I should contact and report the issue:
- mercurial developers?
- python developers?
- who else?

I understand that mozilla is a big user of mercurial and so it should have
some clout to figure out the correct answer is quickly.  :-)

TIA

Happy Hacking in the good old sense of the phrase...

from a very perplexed linux user.

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

HG problem: abort: decoding near '9: error: ‘sse2::l': 'ascii' codec can't decode byte 0xe2 in position 213: ordinal not in range(128)

[ISHIKAWA,chiaki]

Hi,

I am developing mozilla patches locally under my local PC running linux.

I often include the GCC's error/warning message in the mercurial
commit message (in the second and subsequent lines, the first line is
for usage by mozilla's bugzilla procedure.) when my local patch is
created to address build error or suppress compiler warning.

It worked just fine for months/years.

After an upgrade of local tools under Debian GNU/Linux last month or
so, I just noticed that such a patch with GCC warning line could not
be re-applied after it was qpopped. (I use mercurial mqueue
extension.)

This happened yesterday. This is the error message.

note: commit message saved in .hg/last-message.txt
transaction abort!
rollback completed
abort: decoding near '9: error: ‘sse2::l': 'ascii' codec can't decode 
byte 0xe2 in position 213: ordinal not in range(128)!


I was puzzled. After looking at the patch and the message above, I
finally figured out that that hg barfed on the proper quote character
just before "sse2::l" above. Can you spot the slightly tilted OPENING 
quote character?


So I removed the GCC warning line from the commit message and the
patch could be applied. This is problematic because I have about a few
dozen patches (actually more) and some patches could not be re-applied
due to the error above. I had to temporarily removed the offending
patch(es). Luckily the offending patches are relatively independent
and taken out for the moment.
But this could not be done if the patches are interdependent and
must be applied in a certain order.

Either
- GCC began producing such a non-ascii character lately (it is v7 on
my linux PC)
  [but I think I must have seen such characters long time before that],
- the "ascii" codec or whatever used by python has become very strict
  (But why ascii? Should we not use utf-8 after all?)
  [this could be an issue with setup of hg under my Debian GNU/Linux 
distribution?] , or

- something in my locale/character setting under Debian GNU/Linux
  broke lately due to the upgrade of local packages.

My point is that I have NOT changed in my personal setting at all 
(unless |./mach bootstrap| I occasionally run to make sure that I have 
the latest rust and other tools did something funny to my .hgrc).
Only the tools and system-wide setting could have changed mysteriously 
from my point of view.


I found a similar bug report in the following open source project:
https://github.com/schacon/hg-git/issues/27

I have not tried the HGENCODING=utf-8 fix mentioned in the above URL
for fear of breaking existing patches against my local copy of the
mozilla-central and comm-central source tree.

I wonder what would be the ultimate correct fix or
procedure to avoid/suppress this error.

I am posting this to warn developers who *MAY* see similar issues,
and to figure out where I should contact and report the issue:
- mercurial developers?
- python developers?
- who else?

I understand that mozilla is a big user of mercurial and so it should have
some clout to figure out the correct answer is quickly.  :-)

TIA

Happy Hacking in the good old sense of the phrase...

from a very perplexed linux user.

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Prefs overhaul

[ISHIKAWA,chiaki]

On 2018/03/07 12:08, Nicholas Nethercote wrote:

Hi,

I've been doing a lot of work to improve libpref recently, and there is
more in the pipeline. I've written a brief roadmap that might be of
interest to people: https://wiki.mozilla.org/Platform/PrefsOverhaul

I'd be happy to hear feedback, via this list or other means. Thanks.

Nick


I have noticed that you mentioned rewrite in Rust.

Does the I/O primitive Rust uses handle the following "short read" case 
niciely?


Suppose, prefs file (user profile) is stored on a remote file server.
Based on the network congestion and/or server load, the remote file 
server may end up returning shorter than requested octet strings.

(This can happen for real.)
Under linux, the I/O primitive does not read the remaining octets.
Under Windows, the I/O primitive seems to automagically handles the 
reading of remaining octets as far as the remote file server is CIFS. (I 
don't know about NFS.)


Does libpres under linux handles such short-read cases?

I have filed a few bugzilla for TB's failure to tackle this situation 
(under linux), but I have noticed the issue is deep within M-C portion 
of the code, too.


Bugzilla:
Bug 1170646 - A few M-C fixes to handle short read in Cache code ( from 
[META] Failure to deal with short read)


and related bugs there.

Bug 1170578 - RFC: Buggy NSS dbm code ought to be updated
1170606  <--- this is C-C TB side noted below.
Bug 1170668 - Failure to handle short read: the case of reading a JSON 
file into JavaScript 'String' type. ( [META] Failure to deal with short 
read)


1172304 <-- fixed.

On the purely C-C TB side:

Bug 1170606 - C-C T-B fixes to take care of short read (part of [META] 
Failure to deal with short read)

and 1170646 mentioned there.

cf. My comment about short write of PR_Write() in the currently last 
comment  of

https://bugzilla.mozilla.org/show_bug.cgi?id=1170564
may be misguided.
I found that PR_Write() is automatically retried even under Linux.

PPS: MS-Windows seems to handle this under the hood: the I/O routines 
are mapped to MS syscalls and they seem to handle the short read issue 
on the OS side very well.



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Dropping remains of support for non-UTF-8 file paths on Gtk platforms (was: Re: Please do not use GetNativePath and GetNativeTarget in XP code and Windows-specific code)

[ISHIKAWA,chiaki]

On 2017/12/06 1:04, Jonathan Kew wrote:

On 05/12/2017 15:16, Henri Sivonen wrote:
On Tue, Dec 5, 2017 at 4:37 PM, ISHIKAWA,chiaki 
<ishik...@yk.rim.or.jp> wrote:

There are other non-ASCII character issues such as
https://bugzilla.mozilla.org/show_bug.cgi?id=1258613


Very weird bug! (Summary for others: decomposed voiced sound mark is
rendered on the wrong base character.)


Not all that weird, really; it's almost certainly due to using a font 
that doesn't support the combining mark. Commented in the bug.


JK


Thank you for the comment in the bug.

But I wonder if there is a clearly discernible property/attribute of a 
font which allows the combining of a mark and the other font that doesn't.


Basically, the issue appears under linux OS (Debian GNU/Linux) which I 
use daily.


Without knowing which font is causing the issue (not supporting the 
combining of mark), I can't fix it.


Since the "normalization" of string into canonical form under linux 
seems to solve the problem, I am inclined to have the OS or OS-supplied 
library do that, but I am not entirely sure where the rendering happens.


Come to think of it, I am not sure whether the iOS mail client handles 
the filename of an attachment that is sent from Windows or from Linux.

The party with whom I exchanged the problematic e-mails mentioned that
there are e-mails with attachments which cannot be saved under the 
original name and a machine-generated filename seemed to be used.

Oh well, I will investigate this a bit during holiday break.

TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Dropping remains of support for non-UTF-8 file paths on Gtk platforms (was: Re: Please do not use GetNativePath and GetNativeTarget in XP code and Windows-specific code)

[ISHIKAWA,chiaki]

On 2017/12/05 22:24, Henri Sivonen wrote:

On Mon, Dec 4, 2017 at 2:04 PM, Masatoshi Kimura  wrote:

* If by any chance a profile path contains non-ASCII characters on
non-UTF-8 UNIX systems, Firefox 57.0.1 must have broken the profile just
like 57.0 broke it on Windows. But we didn't hear any such complaints.


Are you referring to
https://hg.mozilla.org/mozilla-central/rev/345fe119b8cf using
GetPath() on all platforms and not just Windows?

Experimenting in an Ubuntu VM, Firefox 57.0.1 indeed fails to save
prefs and history (but saves the HTTP disk cache and various other
things) if the profile path has an illegal byte in it. Additionally,
on Debian-based systems generally, adduser only allows usernames (and,
thereby in the common case where the home directory matches the user
name, home directories) that conform to the POSIX portable username
rules (subset of ASCII). useradd appears to have no such safeguards.



There are other non-ASCII character issues such as
https://bugzilla.mozilla.org/show_bug.cgi?id=1258613

But the bug I mention occurs because some characters are encoded 
DIFFERENTLY under iOS and the rest of the world when UTF-8 is used.
So I think the bug will be there no matter whether this non-UTF-8 path 
is removed or not.


By mentioning the bug, I just wanted to point out that there *ARE* 
obnoxious bugs regarding non-ASCII character handling in mozilla software.
But majority of the Japanese users probably failed to file the non-ASCII 
character bugs, and just think, "oh, another instance of Japanese 
characters not passed correctly between mozilla applications and the 
external programs, etc.": this type of Japanese character mungling has 
been so common before, so it is simply ignored as one of those bugs, OR 
when the problem happens it is so difficult to figure out WHERE the buck 
stops (i.e., on what program either outputs incorrect character strings 
or what program parses input incorrectly, OR BOTH.) And if the producer 
and consumer seem to disagree on what type of encoding is used, it is 
usually not quite clear even to ordinary programming type people WHAT is 
the correct way on a given platform, and who is to blame, and thus many 
simply failed to analyze the issue thoroughly and give up half way.

Oh well.

I have noted that, in the last three months or so, some Japanese strings 
copied from Mozilla TB or mozilla FF are not parsed correctly when

they are inserted into other programs.
This did not happen before.

But due to exactly the same reason I noted above, I am not sure which 
program is to blame, and have not bothered to

pester mozilla bugzilla with possibly false-positive bug reports.
If the problem persists in the next few months, I may file a bug.

TIA for people's attention.

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: We need better canaries for JS code

[ISHIKAWA,chiaki]

I am for stricter checks and crashing the test when some of the errors 
discussed here are detected.
I have been puzzled when some attributes are undefined as such when I 
ran tests for Thunderbird in the past.

The bugs are found non-Thunderbird-specific code often times.

Crashing the tests would be the only way to force the responsible party 
to fix them.


Also, the mechanism for automatic dumping the stack during testing when 
such errors are encountered will be appreciated. Running gdb under linux 
and do some python magic to look at VM stack is not easy.


However, sending the stack from the user may have a privacy concern, and 
should be evaluated very carefully.
(I mean, VM stack may show the URL which the user was visiting at the 
time exactly, etc., or even some input.)



n 2017/10/18 18:45, Gregory Szorc wrote:

On Wed, Oct 18, 2017 at 10:28 AM, David Teller  wrote:


 Hi everyone,

   Yesterday, Nightly was broken on Linux and MacOS because of a typo in
JS code [1]. If I understand correctly, this triggered the usual
"undefined is not a function", which was

1/ uncaught during testing, as these things often are;
2/ basically impossible to diagnose in the wild, because there was no
error message of any kind.

I remember that we had bugs of this kind lurking for years in our
codebase, in code that was triggered daily and that everybody believed
to be tested.

I'd like to think that there is a better way to handle these bugs,
without waiting for them to explode in our user's face. Opening this
thread to see if we can find a way to somehow "solve" these bugs, either
by making them impossible, or by making them much easier to solve.

I have one proposal. Could we change the behavior of the JS VM as follows?

- The changes affect only Nightly.
- The changes affect only mozilla chrome code (including system add-ons
but not user add-ons or test code).
- Any programmer error (e.g. SyntaxError) causes a crash a crash that
displays (and attaches to the CrashReporter) both the JS stack in and
the native stack.
- Any SyntaxError is a programmer error.
- Any TypeError is a programmer error.

I expect that this will find a number of lurking errorsy, so we may want
to migrate code progressively, using a directive, say "use strict
moz-platform" and static analysis to help encourage using this directive.

What do you think?



I agree that errors like this should have better visibility in order to
help catch bugs.

I'm not sure changing behavior of the JS VM is the proper layer to
accomplish this. I think reporting messages from the JS console is a better
place to start. We could change the test harnesses to fail tests if certain
errors (like SyntaxError or TypeError) are raised. If there is a "hook" in
the JS VM to catch said errors at error time, we could have the test
harnesses run Firefox in a mode that makes said errors more fatal (even
potentially crashing as you suggest) and/or included additional metadata,
such as stacks.

Another idea would be to require all non-log output in the JS console to be
accounted for. Kinda like reftest's expected assertion count. Assuming most
JS errors/warnings are unwanted, this would allow us to fail all tests
reporting JS errors/warnings while allowing wanted/known failures to not
fail the test. A problem though is background services "randomly" injecting
their output during test execution depending on non-deterministic timing.
It could be difficult to roll this out in practice. But it feels like we
should be able to filter out messages or stacks accordingly.

But why stop there? Shouldn't Firefox installs in the wild report JS errors
and warnings in Firefox code back to Mozilla (like we do crashes)? I know
this has been discussed. I'm not sure what we're doing/planning about it
though.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to require `mach try` for submitting to Try

[ISHIKAWA,chiaki]

Does "mozilla/mach try" will work for try-comm-central?

(Well, I know there has been a general talk of moving thunderbird to 
somewhere else, but it is not going to happen overnight.)


TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to ship: Changing default Japanese fonts to modern fonts

[ISHIKAWA,chiaki]

Thank you for the clarification.

The bit about MS Gothic reminds me why we seem to have rather classic 
feel to the monospace font web rendering, I suppose.


On 2017/08/24 11:45, Masayuki Nakano wrote:
Ah, yes, this change is only on Windows (only on Windows, we still use 
legacy Japanese fonts as default settings).
FYI: We still need to keep using legacy "MS Gothic" font for default 
Japanese monospace font since Windows doesn't have modern monospace font.


On 8/24/2017 7:47 AM, ISHIKAWA,chiaki wrote:

Hi,

I use firefox under Windows most of the time, and so it will be very 
nice.


But obviously this is only for Windows platform, and
does not affect linux or OSX versions. Correct?



On 2017/08/23 14:19, Masayuki Nakano wrote:

Hi,

We decided that we should change our default Japanese fonts from 
legacy "MS PGothic" (sans-serif) and "MS PMincho" (serif) which have 
bitmap glyph to modern "Meiryo" or "Yu Gothic" (sans-serif) and "Yu 
Mincho" (serif).


This has been enabled on Nightly and early Beta since 55 [1].

Then, I removed the |#ifdef EARLY_BETA_OR_EARLIER| in Nightly 
yesterday [2].  I.e., the new default settings are enabled on release 
build starting from 57.


I'd like to introduce some backgrounds:

"Meiryo" is installed on Vista or later (but not installed on Win10 
unless you add Japanese in system language settings). "Yu Gothic" and 
"Yu Mincho" is installed on Win8.1 or later.


Both Edge and Chrome already started to use "Meiryo" as their default 
Japanese fonts. So, using "Meiryo" improves the compatibility between 
browsers and just looks better and is easier to read than bitmap glyph.


One of the reason why we didn't use "Meiryo" as default fonts is, 
"Meiryo" has normal style glyph as italic style glyph. E.g., 
|abc| looks like |abc|, not |a/b/c|. The other browsers have 
this issue, but I see many users who complain about this issue. For 
solving this issue, we added a hack to ignore italic style family of 
"Meiryo" at loading fonts [3]. So, now, Gecko is the only one engine 
which supports italic style Japanese text with default fonts!


On the other hand, there are still some problems.

"Meiryo" has too big internal leading for supporting accent marks of 
Western languages. This causes increasing normal line height than 
other fonts. Additionally, Gecko's normal line height computation is 
not same as the other browsers. Therefore, this may cause 
compatibility issue with the other browsers. For example, our  
allows to scroll its content. Therefore, if user drags text in 
 vertically, the text may be scrolled [4].


If we could use "Yu Gothic" as Japanese default font, it'd be really 
nicer except compatibility with the other browsers because it has 
nicer glyph than "Meiryo" ("Meiryo" was designed for UI, not for 
document. Therefore, it was designed as easier to read even if the 
font size is small) and does not have too big leading like "Meiryo". 
However, there is a big problem. Text renderer of Windows has a bug. 
Text of "Yu Gothic" is rendered as too light [5]. So, the contrast 
between text and background color may not be enough for some users. 
Therefore, currently, we use "Yu Gothic" as a fallback font only when 
"Meiryo" isn't available.


Anyway, I believe that this makes 57 look nicer for Japanese users!

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=548311
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=1354004
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=1351332
[4] https://bugzilla.mozilla.org/show_bug.cgi?id=1378065
[5] https://bug548311.bmoattachments.org/attachment.cgi?id=8848439



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform






___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Intent to ship: Changing default Japanese fonts to modern fonts

[ISHIKAWA,chiaki]

Hi,

I use firefox under Windows most of the time, and so it will be very nice.

But obviously this is only for Windows platform, and
does not affect linux or OSX versions. Correct?



On 2017/08/23 14:19, Masayuki Nakano wrote:

Hi,

We decided that we should change our default Japanese fonts from legacy 
"MS PGothic" (sans-serif) and "MS PMincho" (serif) which have bitmap 
glyph to modern "Meiryo" or "Yu Gothic" (sans-serif) and "Yu Mincho" 
(serif).


This has been enabled on Nightly and early Beta since 55 [1].

Then, I removed the |#ifdef EARLY_BETA_OR_EARLIER| in Nightly yesterday 
[2].  I.e., the new default settings are enabled on release build 
starting from 57.


I'd like to introduce some backgrounds:

"Meiryo" is installed on Vista or later (but not installed on Win10 
unless you add Japanese in system language settings). "Yu Gothic" and 
"Yu Mincho" is installed on Win8.1 or later.


Both Edge and Chrome already started to use "Meiryo" as their default 
Japanese fonts. So, using "Meiryo" improves the compatibility between 
browsers and just looks better and is easier to read than bitmap glyph.


One of the reason why we didn't use "Meiryo" as default fonts is, 
"Meiryo" has normal style glyph as italic style glyph. E.g., 
|abc| looks like |abc|, not |a/b/c|. The other browsers have this 
issue, but I see many users who complain about this issue. For solving 
this issue, we added a hack to ignore italic style family of "Meiryo" at 
loading fonts [3]. So, now, Gecko is the only one engine which supports 
italic style Japanese text with default fonts!


On the other hand, there are still some problems.

"Meiryo" has too big internal leading for supporting accent marks of 
Western languages. This causes increasing normal line height than other 
fonts. Additionally, Gecko's normal line height computation is not same 
as the other browsers. Therefore, this may cause compatibility issue 
with the other browsers. For example, our  allows to scroll its 
content. Therefore, if user drags text in  vertically, the text 
may be scrolled [4].


If we could use "Yu Gothic" as Japanese default font, it'd be really 
nicer except compatibility with the other browsers because it has nicer 
glyph than "Meiryo" ("Meiryo" was designed for UI, not for document. 
Therefore, it was designed as easier to read even if the font size is 
small) and does not have too big leading like "Meiryo".  However, there 
is a big problem. Text renderer of Windows has a bug. Text of "Yu 
Gothic" is rendered as too light [5]. So, the contrast between text and 
background color may not be enough for some users. Therefore, currently, 
we use "Yu Gothic" as a fallback font only when "Meiryo" isn't available.


Anyway, I believe that this makes 57 look nicer for Japanese users!

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=548311
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=1354004
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=1351332
[4] https://bugzilla.mozilla.org/show_bug.cgi?id=1378065
[5] https://bug548311.bmoattachments.org/attachment.cgi?id=8848439



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: sccache as ccache

[ISHIKAWA, Chiaki]

On 2017/08/06 2:06, ISHIKAWA,chiaki wrote:

On 2017/08/04 20:46, Simon Sapin wrote:

On 04/08/2017 13:02, Mike Hommey wrote:

On Fri, Aug 04, 2017 at 11:25:52AM +0200, Simon Sapin wrote:

`mk_add_options export RUSTC_WRAPPER=sccache` causes an error:


It's `mk_add_options "export RUSTC_WRAPPER=sccache"`.


That worked, thanks! I now have:

ac_add_options --with-ccache=/usr/bin/ccache
mk_add_options "export CCACHE_PREFIX=icecc"
mk_add_options "export RUSTC_WRAPPER=sccache"
mk_add_options MOZ_MAKE_FLAGS="-j100"



Does using only the two lines below

 > ac_add_options --with-ccache=/usr/bin/ccache

and

 > mk_add_options "export RUSTC_WRAPPER=sccache"

have the effect of using ccache for ordinary c++/cc compilation
and using sccache for rustc compilation?

I have used ccache extensively (with its setting in various scripts and 
configuration files and so if I can introduce sccache in an unobtrusive 
manner, I would use sccache. But come to think of it, if sscacche honors

CCACHE_DIR and other ccache environment variables, then I will have
collisons of the cachefiles, correct?
Hmm... )

TIA



OK, as I further checked, sccache uses SCCACHE_DIR environment variable 
for cache storage on local disk while ccache uses CCACHE_DIR.
So I can avoid conflict and the above two lines in my mozconfig seemed 
to use sscache for rust compilation, and I see the directory

specified by SCCACHE_DIR populated by cache subdirectories.

However, I am not sure if the cache is working correctly.

With ccache, we can specify a log file in the environment variable 
CCACHE_LOGFILE to specify. We can study the log file to see if

the cache is indeed working (hits, etc).

Is there an equivalent of CCACHE_LOGFILE with sccache?
There was no trace of such logfile in the directory specified by 
SCCACHE_DIR.


TIA

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: sccache as ccache

[ISHIKAWA,chiaki]

On 2017/08/04 20:46, Simon Sapin wrote:

On 04/08/2017 13:02, Mike Hommey wrote:

On Fri, Aug 04, 2017 at 11:25:52AM +0200, Simon Sapin wrote:

`mk_add_options export RUSTC_WRAPPER=sccache` causes an error:


It's `mk_add_options "export RUSTC_WRAPPER=sccache"`.


That worked, thanks! I now have:

ac_add_options --with-ccache=/usr/bin/ccache
mk_add_options "export CCACHE_PREFIX=icecc"
mk_add_options "export RUSTC_WRAPPER=sccache"
mk_add_options MOZ_MAKE_FLAGS="-j100"



Does using only the two lines below

> ac_add_options --with-ccache=/usr/bin/ccache

and

> mk_add_options "export RUSTC_WRAPPER=sccache"

have the effect of using ccache for ordinary c++/cc compilation
and using sccache for rustc compilation?

I have used ccache extensively (with its setting in various scripts and 
configuration files and so if I can introduce sccache in an unobtrusive 
manner, I would use sccache. But come to think of it, if sscacche honors

CCACHE_DIR and other ccache environment variables, then I will have
collisons of the cachefiles, correct?
Hmm... )

TIA

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Shipping Headless Firefox on Linux

[ISHIKAWA,chiaki]

On 2017/06/16 6:47, Mike Hommey wrote:

On Thu, Jun 15, 2017 at 04:51:48PM -0400, Ben Kelly wrote:

On Thu, Jun 15, 2017 at 4:37 PM, Nathan Froyd  wrote:


On Thu, Jun 15, 2017 at 2:02 PM, Brendan Dahl  wrote:

Headless will run less of the platform specific widget code and I don't
recommend using it for platform specific testing. It is targeted more at
web developers and testing regular content pages. There definitely will

be

cases where regular pages will need to exercise code that would vary per
platform (such as fullscreen), but hopefully we can provide good enough
emulation in headless and work to have a consistent enough behavior

across

platforms that it won't matter.


Would it be feasible to use headless mode for mochitests (or reftests,
etc. etc.)?  I know there are probably some mochitests which care
about the cases you mention above (e.g. fullscreen), but if that could
be taken care of in the headless code itself or we could annotate the
tests somehow, it would be a huge boon for running mochitests locally,
or even in parallel.  (We already have some support for running
reftests/crashtests in parallel.)



There are some tests which fail if the "screen" is not a particular size.
Those might be a problem as well.


The worse offenders are the tests that want focus.

Mike


So modal dialog is bad.

Now my memory is coming back. I added an error dialog processing to a 
particular error condition which was not explicitly flagged to the user 
at run time. I think the user is best notified that such an error 
condition exists (I think it was an I/O error or something).


The code failed under xpcshell test because the window (or something 
similar) was not available when it tried to produce the modal dialog on 
encountering the error condition created artificially under xpcshell test.
So I changed the code to put something in the error console instead of 
producing the error dialog, which definitely is not easy to handle in 
headless mode.
Then the code failed the xpcshell test because, it seems to me, that 
xpcshell test suite framework fails a unit test if it detects something 
is put into error console. Oh well.


So finally I changed the code to print something to stderr under linux 
in full DEBUG build (won work under Windows.). Not ideal but at least
I could check the error processing worked internally, but the user was 
kept in the dark because no visible feedback was produced via GUI when a 
serious I/O error occurred.


However, now that I know I can suppress the modal error dialog in 
headless mode by checking a global flag, I have a chance of passing 
xpcshell test. Interesting.


TIA

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Shipping Headless Firefox on Linux

[ISHIKAWA,chiaki]

On 2017/06/16 5:33, Andrew Sutherland wrote:

On Thu, Jun 15, 2017, at 03:27 AM, ishikawa wrote:

On local machine, when I added a modal error dailog as a response to an
error condition, which happens to be  artificially created and tested in
xpcshell test, the test fails because the screen or whatever is not
available. Fair enough, but it was rather disappointing to see such a
feature needed to be disabled due to xpcshell test failing.


You may want to check out
https://searchfox.org/comm-central/source/mailnews/test/resources/alertTestUtils.js
which provides for intercepting various modal prompts in Thunderbird
xpcshell tests.  Whatever the test is testing likely wants to also be
verifying the dialog pops up, which that helper library provides.



Interesting.
But this covers only modal prompts generated by/in JavaScript modules, 
and not C++ code?
If so, maybe I should re-think my previous error/warning dialog to see 
if the generation can be moved to JavaScript code.

It was generated in C++ code.


Andrew


Thank you again for the tips.

Chiaki
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Shipping Headless Firefox on Linux

[ISHIKAWA,chiaki]

On 2017/06/16 0:52, Brendan Dahl wrote:

Depending on how the dialog was triggered, it may just work in headless
mode. To run the test in headless mode, add "headless = true" to the test
in your xpcshell.ini. Also, if you're in headless mode you can check it by
calling gfxPlatform::IsHeadless() in c++.



Thank you for the tips. I think I will resurrect the warning/error 
dialog somehow.




On Thu, Jun 15, 2017 at 12:27 AM, ishikawa  wrote:


On 2017年06月15日 08:51, Brendan Dahl wrote:

Hello All,


As of Firefox 55 I intend to ship headless Linux support (Firefox

without a

GUI and X11 server connection). Headless mode is enabled via the

--headless

command line flag for Firefox and does not affect Firefox when running in
normal mode or on Windows and macOS



For those unfamiliar with the project, the main goal of headless browsing
is to make web developer workflow and testing with Firefox easier. There
are currently two ways to interact with headless Firefox by using either
SlimerJS or Marionette (WebDriver/Selenium).



Testing:

The Marionette test suite is now run in headless mode alongside the

normal

mode as a tier 2 test on try. There are also some basic xpcshell tests

that

use a simple headless browser. If the Marionette tests remain reliable, I
intend to bump them up to tier 1 in a few weeks [1].



In the near future, I'll also be enabling headless mode on Windows and

then

will start work on macOS. We also are going to investigate some possible
performance improvements for headless mode.



If you run into any issues please file a bug in the new headless

component

[2].



This sounds interesting.

On local machine, when I added a modal error dailog as a response to an
error condition, which happens to be  artificially created and tested in
xpcshell test, the test fails because the screen or whatever is not
available. Fair enough, but it was rather disappointing to see such a
feature needed to be disabled due to xpcshell test failing.

Does this new approach take care of such an issue in a more or less
developer-friendly manner? Maybe we have a global flag to see if
the program runs in headless mode?
(Then I can do "if(!headless_mode) show the dialog on the screen", etc.)

TIA



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Improving visibility of compiler warnings

[ISHIKAWA,chiaki]

Hi,

On 2017/05/20 19:36, Martin Thomson wrote:

On Sat, May 20, 2017 at 4:55 AM, Kris Maglione  wrote:

Can we make some effort to get clean warnings output at the end of standard
builds? A huge chunk of the warnings come from NSS and NSPR, and should be
easily fixable.


Hmm, these are all -Wsign-compare issues bar one, which is fixed
upstream.  We have an open bug tracking the warnings
(https://bugzilla.mozilla.org/show_bug.cgi?id=1307958 and specifically
https://bugzilla.mozilla.org/show_bug.cgi?id=1212199 for NSS).


I am surprised that
https://bugzilla.mozilla.org/show_bug.cgi?id=1307958
was mentioned and when I looked at it, it was filed by me. :-)

The patches there were filed about four months ago, and recently I 
noticed that I need to add a few more changes presumably due to code 
changes, AND I found that I must have disabled a few modules from 
compiling to build C-C TB, and a full M-C FF build with -Wsign-compare 
and warning as error setting needs still a few more changes (not that many).


I will upload the latest patches later.

I agree that disabling this check is a little uncomfortable for crypto 
code.

My patch is pasting over the mismatched comparison by
casting one type to the other. In most of the cases, I see that the 
value should not go that high for wraparound near 2^31, etc. but still
I would have inserted an assert() to detect a problematic situation if I 
have infinite amount of time to spend reading the code.
My patches would alert astute coders to look at the places where the 
problem may arise and insert assert() appropriately.


TIA






NSS is built with -Werror separately, but disables errors on
-Wsign-compare.  Disabling those warnings for a Firefox build of NSS
wouldn't be so bad now that we share gyp config.  Based on a recent
build, that's 139 messages (add 36 if you want to add nspr).

I've spent a little time looking into the real issues.  Fixing
requires some care, since it touches ABI compat in many places.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Should &&/|| really be at the end of lines?

[ISHIKAWA,chiaki]

On 2017/02/18 7:28, Bobby Holley wrote:

On Fri, Feb 17, 2017 at 2:18 PM,  wrote:


Hi again Nick,

Someone made me realize that I didn't fully read your message, sorry for
that.

I now see that as well as &&/||, you have grepped for other operators, and
shown that the overwhelming usage is to put all of them at the end of lines!

In light of this, and from what others here have discussed, I'm
now thinking that we probably should indeed just update our coding style
with whatever is used the most out there, and model our "Official
Formatter" on it.


Another thought, if technically possible:
If our main repository is going to always be under the control of some
official clang-format style, it should be possible for anybody to pull the
repository, and use a different formatter locally with their favorite
style. And when pushing, their modified code could be automatically
reformatted with the official formatter -- Everybody feels good when
programming! :-)



Please no. That would make reviewing a nightmare.



I think the idea is to produce the patch after (virtually) reformatting 
the local tree into the officially adopted format and create the patch

against the tree.
The reviewers will get into

checkout
Central repo
  -> local repo in the official format
  check out a file or files to be modified.
  Use a clang-format or whatever by passing options to it
  to produce one's preferred format and check it (them) into
  a working repo.
 -> local repo in the preferred format: local mods take place
 We work on the files in one's preferred format.

checkin/creating a patch

 local repo in the preferred format: local mods take place
 Locally use clang-format or whatever to mutate the changed
 file into official format and check into the local repo in the
 official format.
 <--
  local repo in the official format
  Either create a patch (in the official format source)
  or check into the central repo.
  <--
Central repo

Point 1:
I think creating a wrapper for conversion between the two local repos
as above should not be that difficult.

Point 2:
I prefer all the operators including "&&" and "||"
at the beginning since such a format makes the tree-like structure of 
multi-line easier to understand and edit, too.

Adding a condition or removing a condition is much easier in this form.
I forgot where the rational was explained well. Superficially it is 
covered in GNU coding starnds:

https://www.gnu.org/prep/standards/html_node/Formatting.html

Point 3: Having a tool that does such conversion AND having an editor 
support such formating style is very important (In my case, it is Emacs.)


Point 4: Have no fear. Such conversion tools have existed at least for 
C, Fortran, Cobol and other languages

and been used extensively.
For example, more than a dozen years ago,
in the days of version 2.1.1xx, the whole SCSI subsystem source tree of 
linux kernel were converted using "indent" program.

The only change of SCSI subsystem in that revision was formating change.
This was to untangle the spaghetti code of SCSI subsystem.

Point 5: We should set up a "Flag Day" to convert the source tree into 
the official format THAT IS SUPPORTED by the mechanical 
converter/formater, and change the source code in  one sweep.

For example, the data can be 6 months from now. August 18, 2017.
(Such a flag day / intention was announced for SCSI subsystem format 
change of linux kernel, and the "Flag Day" moniker was used when Multics 
changed its internal character code from what I read in the glossary 
file which later became Hacker's Dictionary.)


Mechanical conversion/formating is nice: we can do this locally before 
producing an official patch, and there will be no disagreements among 
the reviewers :-)



expressions





Cheers,
Gerald


On Friday, February 17, 2017 at 5:16:42 PM UTC+11, Nicholas Nethercote
wrote:

I personally have a strong preference for operators at the end of lines.
The codebase seems to agree with me, judging by some rough grepping

('fff'

is an alias I have that's roughly equivalent to rgrep):

$ fff "&&$" | wc -l
  28907
$ fff "^ *&&" | wc -l
   3751

$ fff "||$" | wc -l
  26429
$ fff "^ *||" | wc -l
   2977

$ fff " =$" | wc -l
  39379
$ fff "^ *= " | wc -l
629

$ fff " +$" | wc -l
  31909
$ fff "^ *+$" | wc -l
491

$ fff " -$" | wc -l
   2083
$ fff "^ *-$" | wc -l
 52

$ fff " ==$" | wc -l
  1501
$ fff "^ *== " | wc -l
  161

$ fff " !=$" | wc -l
  699
$ fff "^ *!= " | wc -l
  129

They are rough regexps and probably have some false positives, but the
numbers aren't even close; operators at the end of the line clearly
dominate.

I will conform for the greater good but silently weep inside every time I
see it.

Nick

On Fri, Feb 17, 2017 at 8:47 AM,  wrote:


Question of the day:
When breaking overlong expressions, should &&/|| go at the end or the
beginning of the line?

TL;DR: Coding style says 'end', I think we 

Re: Changing the representation of rectangles in platform code

[ISHIKAWA,chiaki]

On 2017/02/14 8:37, gsquel...@mozilla.com wrote:

On Tuesday, February 14, 2017 at 3:25:05 AM UTC+11, Ehsan Akhgari wrote:

On 2017-02-11 3:27 AM, ISHIKAWA,chiaki wrote:

On 2017/02/11 7:07, Botond Ballo wrote:

On Fri, Feb 10, 2017 at 3:45 PM, Milan Sreckovic
<msrec...@mozilla.com> wrote:

First step needs to happen completely before the second step does, so I
guess the danger is that we start and give up before we get to step
2.  I
don't think that will happen, but it is something we should always think
about.


It may be possible to automate most if not all of step 1 using an
automated refactoring based on clang's AST matchers (the same thing
our static analyses are based on). Then we could do step 1 all at
once, and even do before/after performance comparisons before
committing to it. Worth exploring, I think.



Is there any document about the AST matchers, especially about the
analyzer that does "static analyses" for mozilla code?

For example, I found the following URL:
https://clang.llvm.org/docs/LibASTMatchersTutorial.html


This reference documents all of the built-in matchers:
<http://clang.llvm.org/docs/LibASTMatchersReference.html>.  The page
above is a good place to start.  I also have a blog post that explains
how to create a simple static analysis check where I talk about AST
matchers among other things:
<https://ehsanakhgari.org/blog/2015-12-07/c-static-analysis-using-clang>.
 There are other good blog posts and code samples to be found by doing a
web/github search.

Cheers,
Ehsan


And clang-query is your best friend! It saved me days of debugging-by-Try for 
my second matcher. :-)
More info there:
http://eli.thegreenplace.net/2014/07/29/ast-matchers-and-clang-refactoring-tools

Also http://gcc.godbolt.org/ using clang with compiler option: '-Xclang 
-ast-dump', to quickly work out how some code translates to ASTs.

Cheers,
Gerald



Hi, thank you for the pointers.

I will study them to see if I can use the matcher to do something useful.


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: A reminder about MOZ_MUST_USE and [must_use]

[ISHIKAWA,chiaki]

On 2017/01/20 8:10, Nicholas Nethercote wrote:

There are lots
of functions where not checking the return value is reasonable, such as
close().


A file opened for writing and is buffered will flush pending data to 
disk upon Close() and may encounter the error such as disk full AT THAT 
POINT, and so the return value of

Close() MUST BE CHECKED for those cases.
[I am fighting to fix the issue since C-C TB fails to use buffered write 
which causes so much I/O overhead, but then I found the return value of 
Close() are not checked at all. Ugh... So I have to fix C-C TB to make 
it check the return value of Close() and take appropriate error 
reporting/recovery action before I can enable buffering. Very 
disappointing...]


A blanket statement above can put bad ideas in many people's mind :-(

Of course, if the "close()" above refers to POSIX close(), fine.

TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: GCC 4.9 now required to build on Linux/Android

[ISHIKAWA,chiaki]

On 2016/12/24 1:54, Nathan Froyd wrote:

On Fri, Dec 23, 2016 at 11:37 AM, Mike Hoye  wrote:

On 2016-12-23 11:08 AM, Nathan Froyd wrote:

Bug 1322792 has landed on inbound, which changes configure to require
GCC 4.9 to build; our automation switched over to GCC 4.9 for our
Linux builds earlier this week.  (Android builds have been using GCC
4.9 for some time.)


I happened to be poking at the MDN docs when this came in, so I'll update
them to reflect this.


Thank you!


I haven't tested our minimum hardware recommendations on Linux - 2GB ram,
30GB free space - recently, but I'll test them in the new year.


For 64-bit Linux, I think you need 4GB at the absolute minimum (build
may start swapping), and 8GB would be better.  My rule of thumb is
that you need a minimum of 2GB/thread that you're compiling with on
64-bit Linux (e.g. 16GB on a 4 core/8 thread machine); some of our
autogenerated files take lots of RAM to compile, and you want to have
a little bit left over for applications.


Anyone know offhand if it's still possible to build on a 32-bit Linux box?
We haven't been able to build on 32-bit Windows for a while now.


I suspect it's possible, based on halving the RAM from a 64-bit build,
but I haven't tried it.

-Nathan


I used to compile C-C TB in 32-bit environment, but it required GNU 
gold, ld.gold linker. This is a must if someone tries 32-bit build.


This is because, circa three years ago or so, ordinary ld linker started 
to run out of memory space during linking (its internal table became too 
large for 32-bit memory space). I had switched to GNU gold linker 
because it was faster, but it also used much smaller memory area. (Going 
over 2GB or not was crucial for 32-bit memory space build.)
When I noticed the ordinary linker started to blew up due to 32-bit 
limit, switching to GNU gold linker kept me building C-C TB on 32-bit 
linux. I did this for a year or so.


Also, you may have to run with -gsplit-dwarf so that debug symbol 
generation is somewhat different from the default method, but this I am 
not sure. Again, for -gsplit-dwarf to work effectively, you need to use 
GNU ld linker, I think.


I noticed the memory space issue for 32-bit linux around the time when 
compiling and linking for linux binary began using 64-bit linux in the 
compilation farm. It was not easy for me to switch my desktop linux PC 
to 64-bit, so I stuck to 32-bit linux for a while.

But I needed to switch to GNU gold linker due to the reasons above.

Since then, though, I have switched to 64-bit linux now.
The 32-bit machine's power unit died and when new installation in a new 
box was necessary, I decided to use 64-bit.


I am using Debian GNU/linux 64-bit and so far it works fine. I think the 
compilation farm of mozilla uses CentOS, but as far as the toolchain 
goes, there does not seem to be much difference today. (Debian's g++ was 
broken for mozilla software for half a year about 4 years ago. It could 
not grok subtle name scope issues: Debian's g++ had its own original 
patches. Also, Debian tended to stick to very old stable versions.
But recent GCC development has a very quick turnaround in contrast to 
the old days, and Debian now releases the GCC following the regular GCC 
updates. So users may encounter less such problems now. I have not 
noticed Debian-specific issues for building for quite a while. Debian's 
iconv seemed to have a subtle issue for a long time, but that is 
separate from build issues.)




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: hg.mozilla.org certificate fingerprint changed?

[ISHIKAWA,chiaki]

On 2016/10/01 4:23, Ralph Giles wrote:

The change was announced here and on firefox-dev a few weeks ago. See
for example 
https://groups.google.com/d/msg/mozilla.dev.platform/LOC83qKUPfk/cZtmaEbOAwAJ


Obviously I missed it during a busy trip. Thank you.
(It would be nice to have the announcement on dev-apps-thunderbird and 
dev-builds ML as well. I would have noticed this if all these MLs have 
the announcement...)



It might be nice if `mach mercurial-setup` did this kind of update?


As Gregory Szorc <g...@mozilla.com> already noted, it is an egg and 
chicken problem. As a matter of fact, I tried "mach mercurial-setup" 
just in case, and bumped into the already outdated cert issue since

the data/code used by mach mercurial-setup is taken from hg.mozilla.org.

My |hg| is 3.9.1, but I am not sure if my Python is new enough so that 
the security handling mentioned by Gregory works or not.


Short of automation and one-time announcement,
it may be a good idea to have a secure web page that lists the latest 
fingerprint of certs used by major mozilla servers that users interact 
DIRECTLY (bugzilla and hg come to my mind.)
Then many of us can calmly check the fingerprints for the servers when 
some mismatch is reported by tools like ssh/https-related tools, and 
decide to update the local check/verification data assuming that they 
have missed the update announcements.


TIA



 -r

On Fri, Sep 30, 2016 at 12:18 PM, ISHIKAWA,chiaki <ishik...@yk.rim.or.jp> wrote:

In the last few days, hg.mozilla.org certificate fingerprint seems to have
changed.
I noticed this because the trial to update local copy of mozilla-central
repository within C-C repository failed due to

m-central/mozilla', 'https://hg.mozilla.org/mozilla-central/']
pulling from https://hg.mozilla.org/mozilla-central/
abort: certificate for hg.mozilla.org has unexpected fingerprint
73:7f:ef:ab:68:0f:49:3f:88:91:f0:b7:06:69:fd:8f:f2:55:c9:56
(check hostfingerprint configuration)

But I did not see any announcement of this change.
(It is possible that I missed it during a hectic schedule during a trip).

However, it is great to see a posting of such major infra change in advace,
especially security-related one.

Finally, I bit the bullet and changed it, but checked bugzilla
just in case, and found
https://bugzilla.mozilla.org/show_bug.cgi?id=1305909
which seems to be related.

Automation is nice, but I still would like to see an announcement of server
certificate change in advance.

TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

hg.mozilla.org certificate fingerprint changed?

[ISHIKAWA,chiaki]

In the last few days, hg.mozilla.org certificate fingerprint seems to  
have changed.
I noticed this because the trial to update local copy of mozilla-central  
repository within C-C repository failed due to


m-central/mozilla', 'https://hg.mozilla.org/mozilla-central/']
pulling from https://hg.mozilla.org/mozilla-central/
abort: certificate for hg.mozilla.org has unexpected fingerprint  
73:7f:ef:ab:68:0f:49:3f:88:91:f0:b7:06:69:fd:8f:f2:55:c9:56

(check hostfingerprint configuration)

But I did not see any announcement of this change.
(It is possible that I missed it during a hectic schedule during a trip).

However, it is great to see a posting of such major infra change in  
advace, especially security-related one.


Finally, I bit the bullet and changed it, but checked bugzilla
just in case, and found
https://bugzilla.mozilla.org/show_bug.cgi?id=1305909
which seems to be related.

Automation is nice, but I still would like to see an announcement of  
server certificate change in advance.


TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Strange error message on try-comm-central: Error: Invalid addon ID: expected addon ID specialpow...@mozilla.org, found special-pow...@mozilla.org in manifest

[ISHIKAWA,chiaki]

While perusing error log of try-comm-central run of
Windows 7 debug version run of |make mozmill| test on try-comm-central 
treeherder, I noticed a strange error message.


3:07:56 INFO -  1474488476446	addons.xpi-utils	WARN	addMetadata: 
Add-on specialpow...@mozilla.org is invalid: Error: Invalid addon ID: 
expected addon ID specialpow...@mozilla.org, found 
special-pow...@mozilla.org in manifest  ...



I am not sure if this comes from the intended test.
Rather it looks a configuration error.
The error line seems to appear at beginning of the selected run of each 
test directory.


The error lines surrounding the "special-pow...@mozilla.org in manifest" 
are as follows.

I quote  from one instance of such error lines.


13:07:52 INFO - 
['C:\\slave\\test\\build\\venv\\Scripts\\python.exe', 'runtest.py', 
'-t', 'keyboard', '--binary', 
'C:\\slave\\test\\build\\application\\thunderbird\\thunderbird.exe', 
'--symbols-path', 'C:\\slave\\test\\build\\symbols', '--plugins-path', 
'C:\\slave\\test\\build\\application\\thunderbird\\plugins', 
'--testing-modules-dir', 'C:\\slave\\test\\build\\test\\modules']
13:07:54 INFO -  Using profile dir: 
C:\slave\test\build\tests\mozmill\mozmillprofile
13:07:54 INFO -  [1244] WARNING: Failed to load startupcache file 
correctly, removing!: file 
c:/builds/moz2_slave/tb-try-c-cen-w32-d-000/build/mozilla/startupcache/StartupCache.cpp, 
line 219
13:07:54 INFO -  (debug) 
nsWindowWatcher::SetWindowCreator(nsIWindowCreator* aCreator) called
13:07:54 INFO -  [1244] WARNING: CheckLinkStatus called on main 
thread! No check performed. Assuming link is up, status is unknown.: 
file 
c:/builds/moz2_slave/tb-try-c-cen-w32-d-000/build/mozilla/netwerk/system/win32/nsNotifyAddrListener.cpp, 
line 707
13:07:54 INFO -  [1244] WARNING: This method is lossy. Use 
GetCanonicalPath !: file 
c:/builds/moz2_slave/tb-try-c-cen-w32-d-000/build/mozilla/xpcom/io/nsLocalFileWin.cpp, 
line 3477
13:07:56 INFO -  1474488476446	addons.xpi-utils	WARN	addMetadata: 
Add-on specialpow...@mozilla.org is invalid: Error: Invalid addon ID: 
expected addon ID specialpow...@mozilla.org, found 
special-pow...@mozilla.org in manifest 
(resource://gre/modules/addons/XPIProvider.jsm -> 
resource://gre/modules/addons/XPIProviderUtils.js:1658:15) JS Stack 
trace: addmetad...@xpiproviderutils.js:1658:15 < 
processfilechan...@xpiproviderutils.js:2018:23 < 
this.xpiprovider.checkforchan...@xpiprovider.jsm:3808:34 < 
this.xpiprovider.star...@xpiprovider.jsm:2794:25 < 
callprovi...@addonmanager.jsm:236:12 < 
_startprovi...@addonmanager.jsm:789:5 < 
addonmanagerinternal.star...@addonmanager.jsm:973:9 < 
this.addonmanagerprivate.star...@addonmanager.jsm:3017:5 < 
ammanager.prototype.obse...@addonmanager.js:71:9
13:07:56 INFO -  1474488476452	addons.xpi-utils	WARN	Could not 
uninstall invalid item from locked install location
13:07:57 INFO -  [1244] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed 
with result 0x80520012: file 
c:/builds/moz2_slave/tb-try-c-cen-w32-d-000/build/mozilla/extensions/cookie/nsPermissionManager.cpp, 
line 2675
13:07:57 INFO -  [calBackendLoader] Using icaljs backend at 
C:\slave\test\build\application\thunderbird\extensions\{e2fda1a4-762b-4020-b5ad-a41df1933103}\components\icaljs-manifest
13:07:57 INFO -  [1244] WARNING: 'NS_FAILED(rv)', file 
c:/builds/moz2_slave/tb-try-c-cen-w32-d-000/build/mozilla/dom/plugins/base/nsPluginHost.cpp, 
line 2364


OBSERVATION:
As it turns out, try-comm-central log for mozmill run of linux64 debug 
version also has a similar error.

So it is not windows specific.

*HOWEVER*, I don't see a similar error line on my local PC's test log.
I am testing linux64 debug version on local PC.

Something is incorrectly set up on try-comm-central?

Using DXR, I found 8 occurrences of special-pow...@mozilla.org, and
three occurrences of specialpow...@mozilla.org.

I suspect they should be consistently named.

TIA
cf. 3 results from the comm-central tree:
mozilla/testing/specialpowers/Makefile.in
♢7 XPI_PKGNAME = specialpow...@mozilla.org

mozilla/testing/testsuite-targets.mk
♢ 356 specialpow...@mozilla.org.xpi \

obj-x86_64-pc-linux-gnu/testing/specialpowers/Makefile
♢ 15 XPI_PKGNAME = specialpow...@mozilla.org


windows debug version raw log is:
https://archive.mozilla.org/pub/thunderbird/try-builds/ishik...@yk.rim.or.jp-8fa70f3401d6/try-comm-central-win32-debug/try-comm-central_win7_ix-debug_test-mozmill-bm126-tests1-windows-build3.txt.gz

https://treeherder.mozilla.org/logviewer.html#?job_id=21663=try-comm-central
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: NS_WARN_IF_FALSE has been renamed NS_WARNING_ASSERTION

[ISHIKAWA,chiaki]

On 2016/09/22 5:56, Daniel Holbert wrote:

On 09/21/2016 12:48 PM, ISHIKAWA,chiaki wrote:

In the following URL about coding style,
https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Coding_Style
--- begin quote ---
if (NS_WARN_IF(somethingthatshouldbetrue)) {
  return NS_ERROR_INVALID_ARG;
}

if (NS_WARN_IF(NS_FAILED(rv))) {
  return rv;
}

--- end quote 

I am not a native speaker of English, but shouldn't the variable name in
the first if-statement example be
|somethingthatshouldNOTbetrue| instead of |somethingthatshouldbetrue|?


You're very right!  I've fixed it to say "somethingthatshouldbefalse"
(s/true/false/):
https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Coding_Style$compare?locale=en-US=1122081=1086401



Thank you for fixing the issue so quickly.


If so, Japanese, Chinese and Russian translation ought to be changed as
well.


I am not qualified to fix those. I'm hoping you & others can take care
of that, perhaps! :)


I am not sure how to alert the translation contributors. Does anyone 
know how?



I may have found a few code fragments that may have been miswritten due
to misunderstanding.


Yikes! Please file bugs on those. Hopefully they are just cases of of us
accidentally warning in the wrong condition, rather than us actually
behaving incorrectly.


The particular issue I was struggling with turned out to be NOT the 
warning in the wrong condition, but the failure to use the return value 
of NS_WARN_IF, and I am not sure if I got it right now that I re-read 
the developer page :-(
Bug 1304504 - Check the return value of GetIntPref() properly or an 
initialized bogus value may be used

https://bugzilla.mozilla.org/show_bug.cgi?id=1304504

I was confused to no end when I read the original coding guideline page.
Thank you again for the quick fix. I hope future developers won't be 
confused any longer :-)




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: NS_WARN_IF_FALSE has been renamed NS_WARNING_ASSERTION

[ISHIKAWA,chiaki]

On 2016/09/02 16:02, Nicholas Nethercote wrote:

Greetings,

NS_WARN_IF_FALSE has been renamed NS_WARNING_ASSERTION.

See https://bugzilla.mozilla.org/show_bug.cgi?id=1299727 for details,
including the justification.

See https://bugzilla.mozilla.org/show_bug.cgi?id=137 for a bug
that this change exposed, thus adding weight to the justification.

See https://bugzilla.mozilla.org/show_bug.cgi?id=132 for a
comm-central patch.

Nick



I think it is a good time to ask about a confusing issue.

In the following URL about coding style,
https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Coding_Style

there is a section about NS_WARN_IF.
I quote it below.

--- begin quote ---

Use the NS_WARN_IF macro when errors are unexpected.

The NS_WARN_IF macro can be used to issue a console warning in debug 
builds if the condition fails. This should only be used when the failure 
is unexpected and cannot be caused by normal web content.


If you are writing code that wants to issue warnings when methods fail, 
please either use NS_WARNING directly or use the new NS_WARN_IF macro.


if (NS_WARN_IF(somethingthatshouldbetrue)) {
  return NS_ERROR_INVALID_ARG;
}

if (NS_WARN_IF(NS_FAILED(rv))) {
  return rv;
}

--- end quote 

I am not a native speaker of English, but shouldn't the variable name in
the first if-statement example be
|somethingthatshouldNOTbetrue| instead of |somethingthatshouldbetrue|?

If so, Japanese, Chinese and Russian translation ought to be changed as 
well.


I may have found a few code fragments that may have been miswritten due 
to misunderstanding.


TIA

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: All profile settings lost in new version

[ISHIKAWA,chiaki]

On 2016/09/02 20:15, Wayne wrote:

I'm wondering, under what circumstances would a user have a prefs.js-1
and prefs.js be missing?

ref https://support.mozilla.org/en-US/questions/1136992
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




I am curious if the affected user(s) use remotely mounted file system to 
store pref.js.


This is because I have  noticed in so many places the return value of 
close() and its wrapper functions are not checked in both C-C and even M-C.



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Heads up: valgrind/memcheck issues

[ISHIKAWA,chiaki]

Hi,

I did at least for the memory access after free issue.
Other issues may have to wait unless someone checks the issues.

I wish these are false positives, but they seem too numerous this time 
around.


TIA

CI

On 2016/08/26 3:17, Andrew McCreight wrote:

We regularly run AddressSanitizer and Valgrind on mozilla-central, so it
seems likely this is Thunderbird specific. Please file security bugs in
bugzilla with any reports you have found.

Andrew

On Thu, Aug 25, 2016 at 10:59 AM, ISHIKAWA,chiaki <ishik...@yk.rim.or.jp>
wrote:


Hi,

I occasionally check C-C TB by running |make mozmill| test suite under
valgrind/memcheck for detecting memory issues.

I ran such a test the day before and saw a flurry of memcheck warnings
which I did not see in the test that was run back on May 12th.

So in three months between, some memory errors were introduced.

I checked them briefly and saw many errors related to wrappers for
getenv().
But I also saw some other issues which seemed rather disturbing.
I am tied with a big project with a day job and not sure if I can follow
up on these issues in a timely manner before it is too late.

If anyone occasionally checks C-C TB or M-C FF using valgrind/memcheck by
running xpcshell tests and mozmill test in the case of C-C TB), I urge such
people to do the tests NOW so that we can figure out which patch sets
introduced such memory issues before it becomes difficult to figure out the
culprit.

To wit, the frequency and types of warnings from a partial log (partial
since it takes more than 24 hours to run the test, I had to quit it
halfway).

 12Memcheck:Addr1
284Memcheck:Addr2
  4Memcheck:Addr8
 39Memcheck:Cond
  1Memcheck:Param

The number at the beginning of the line is the frequency of appearance in
the log. (But do note that there are different errors reported under the
same heading, Memcheck:Addr2, for example. This is disturbing enough since
usually a known false positive often reports only the same code path.

Yes, valgrind/memcheck can report false failures sometimes.
But this time, the warnings are varied enough and some did look
a problem like the access AFTER free (!) in one case. That should not be
overlooked!!!

TIA

CI

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Heads up: valgrind/memcheck issues

[ISHIKAWA,chiaki]

Hi,

I occasionally check C-C TB by running |make mozmill| test suite under  
valgrind/memcheck for detecting memory issues.


I ran such a test the day before and saw a flurry of memcheck warnings  
which I did not see in the test that was run back on May 12th.


So in three months between, some memory errors were introduced.

I checked them briefly and saw many errors related to wrappers for getenv().
But I also saw some other issues which seemed rather disturbing.
I am tied with a big project with a day job and not sure if I can follow  
up on these issues in a timely manner before it is too late.


If anyone occasionally checks C-C TB or M-C FF using valgrind/memcheck  
by running xpcshell tests and mozmill test in the case of C-C TB), I  
urge such people to do the tests NOW so that we can figure out which  
patch sets introduced such memory issues before it becomes difficult to  
figure out the culprit.


To wit, the frequency and types of warnings from a partial log (partial  
since it takes more than 24 hours to run the test, I had to quit it  
halfway).


 12Memcheck:Addr1
284Memcheck:Addr2
  4Memcheck:Addr8
 39Memcheck:Cond
  1Memcheck:Param

The number at the beginning of the line is the frequency of appearance  
in the log. (But do note that there are different errors reported under  
the same heading, Memcheck:Addr2, for example. This is disturbing enough  
since usually a known false positive often reports only the same code path.


Yes, valgrind/memcheck can report false failures sometimes.
But this time, the warnings are varied enough and some did look
a problem like the access AFTER free (!) in one case. That should not be  
overlooked!!!


TIA

CI

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: New [must_use] property in XPIDL

[ISHIKAWA,chiaki]

On 2016/08/25 6:38, Mike Hommey wrote:

On Mon, Aug 22, 2016 at 04:39:15PM -0700, R Kent James wrote:

On 8/21/2016 9:14 PM, Nicholas Nethercote wrote:

I strongly encourage people to do likewise on
any IDL files with which they are familiar. Adding appropriate checks isn't
always easy


Exactly, and I hope that you and others restrain your exuberance a
little bit for this reason. A warning would be one thing, but a hard
failure that forces developers to drop what they are doing and think
hard about an appropriate check is just having you set YOUR priorities
for people rather than letting people do what might be much more
important work.


If you feel so strongly that you don't want to spend your time chasing
those new errors as they pop up, you're also free to drop what you are
doing and add -Wno-error=unused-result to your build flags.

Mike


But that means we have to add -Wno-error=unused-result to compiler flag 
that produces the released binary in the server farm. Correct?

(At least this would be the case of C-C TB a couple of days ago...)

In the long run, I would DFINITELY like to see this type of compile-time 
warning (maybe not the compiler failure) so that we can fix the 
non-checking of should-be-checked return value of low primitives.
There are simply TOO MANY such omission of checks in C-C and M-C code to 
my taste. I was disgusted to read the code (and still am) to read C-C 
code to fix a minor bug (and that is not limited to C++ code, but to JS 
code either), and lo and behold, when I trace the return value 
processing, I often end up seeing that M-C code also fails to do the 
proper checking of return value. (It is a problem of legacy code without 
such built-in checks at the early stage.)


I have tried to add such checks in many places when I produced the patch 
to enable buffering for file I/O (mostly output) in C-C TB.
Without such checks, the supposed transaction processing of e-mail 
messages in the face of download/saving failure of the message does not 
work, and I have found out there are places where proper error checking 
that should have been there in the face of network file system failures 
due to networking issues.


I have said this several times before, but if the code in mozilla source 
tree is handed in as a course project to a systems programming 401 or 
something like that, I would have no qualm to give D to the submitter as 
the TA or lecturer.


There ought to be a long-term plan for C-C and M-C to introduce such 
missing checks gradually.


Just two my cents worth.

CI

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: mach and ccache failure?

[ISHIKAWA,chiaki]

On 2016/08/20 10:03, Jan Varga wrote:

I see the same problem on my mac, but I didn't investigate why it's not
working anymore.




Did ccacge ever work on the Mac to support two directories?
(I mean the compilation result in one object directory could be
substituted for the other object directory if the source file did not 
change?)


TIA



On 20/08/16 08:48, zbranie...@mozilla.com wrote:

On Friday, August 19, 2016 at 4:12:12 PM UTC-7, Xidorn Quan wrote:

On Sat, Aug 20, 2016, at 08:45 AM, zbranie...@mozilla.com wrote:

Both builds take around 43-46 minutes, with ccache hit rate 0.8-1.0%.

This is the same source - mozilla-central from today.

What am I doing wrong?

Probably your ccache cache is too small? What size did you set?

- Xidorn

It fills the ccache:

cache directory /home/zbraniecki/.ccache
primary config  /home/zbraniecki/.ccache/ccache.conf
secondary config  (readonly)/etc/ccache.conf
cache hit (direct) 39320
cache hit (preprocessed)2983
cache miss 43686
called for link 2809
called for preprocessing2156
compile failed   512
preprocessor error   914
bad compiler arguments   121
unsupported source language  191
autoconf compile/link   3074
no input file  21162
files in cache 21388
cache size   8.9 GB
max cache size  10.0 GB

zb.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Using rr chaos mode to find intermittent bugs

[ISHIKAWA,chiaki]

On 2016/02/11 5:47, Robert O'Callahan wrote:

On Thu, Feb 11, 2016 at 9:32 AM, Ted Mielczarek  wrote:


BenWa tried doing some work on this but kept getting hung up
on hitting test failures unrelated to the ones we see in production,

possibly due to environment issues.
Yes. In this vein, it's possible that in some cases rr chaos mode might
trigger bugs that don't normally happen, that one way or another block you
from finding the bug you care about.

However, bugs found by rr chaos mode should all be "real bugs". I'd
certainly love to hear about any cases where that's not true.

Rob


This scheduling change causing rare to reproduce bugs to occur more 
often sounds interesting.


I have found that running C-C TB (sorry it is not the browser here)
under valgrind/memcheck which slows down the operation dramatically
have helped me to find a few issues.
From the top of my head:
 - incremental GC gets re-entered before it finishes the previous 
invocation.

   This was not handled properly until I noticed the issue, but it is
   now handled OK.
 - there are some issues in threading.
   For one, at start up, some threads incorrectly assume that window as 
on screen is

   already there, but due to the slowdown, it is not created yet.
   I see some disturbing warning messages printed on the invoking tty 
window.
   I have not filed a bug yet since this is relatively new. I don't 
think I saw

   such messages early last year.

   For the other, at shutdown, C-C TB has a problem of incorrect 
ordering of

   thread shutdown: some threads seem to request services during shutdown
   from service providers, but threads that provide the services have 
already
   shutdown. So proper shutdown does not happen. There may even be a 
cyclic

   dependency. Who knows?
   With the slowdown due to valgrind/memcheck, the issue
   gets more pronounced. Well, right now, though, there is
   a timer that monitors the shtudown process and the prolonged timeout of
   some operations due to the thread missing and the slowdown caused by
   valgrind/memcheck automatically triggers the assertion of permanent 
hung at
   shutdown and so it is difficult to figure out what are going on. But 
one can

   hope that the check for permanent hung gets removed temporarily to
   investigate the issue further.
   Crashes at C-C TB are something I experienced several times in the last
   couple of years in real life.


Another thing this rr framework or similar approach will be useful for 
C-C TB xpcshell testing (and I think it is useful for FF xpcshell 
testing as well.)


There seem to be a few intermittent test failures in xpcshell tests.
This rr approach may make the test fail more often.

*HOWEVER*, I am going to file a bugzilla about
OVEREAGER ASYNC approach of the current test xpcshell script introducing
spurious errors at least under Windows (a previous test which still have 
some files open has not completely shut down before the next test that 
seems to use

THOSE files get started. Under windows, opening such a file may result in
file locked error (under linux/OSX, I think it is OK to open such files 
unless the first program explicitly calls |flock| or something.)


So whether ALL the intermittent failures in C-C TB xpcshell tests are 
something that can be investigated better with rr approach is anyone's 
guess, but

I think it does have a potential to trigger more dormant bugs just as
valgrind/memcheck uncovered a few timing issues.

But one other post suggested that it is not applicable right now outside 
Gecko, meaning C-C TB xpcshell testing cannot directly benefit from rr?

(The approach, of course, can be emulated, I suppose.)

TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Using rr chaos mode to find intermittent bugs

[ISHIKAWA,chiaki]

On 2016/02/11 7:04, Robert O'Callahan wrote:

rr should work fine with c-c xpcshell tests (and most other Linux programs).

This sounds great!

CI

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Strange errors n OS X platform

[ISHIKAWA, Chiaki]

On 2015/12/15 23:40, Philip Chee wrote:

On 13/12/2015 16:26, ISHIKAWA, Chiaki wrote:

{"fun":"getCurrentDirectory","id":2}
  14:54:20 INFO -  PROCESS | 5963 | OS Controller Message posted
  14:54:20 INFO -  PROCESS | 5963 | OS Controller Expecting reply
  14:54:20 INFO -  PROCESS | 5963 | OS Controller Received message
from worker
{"id":1,"timeStamps":{"entered":1449960860763,"loaded":1449960860788}}
  14:54:20 INFO -  PROCESS | 5963 | OS Controller Received uncaught
error from worker Error: couldn't find function symbol in library


The tests appear to be failing at the "getCurrentDirectory" line.

Suspect:
<http://mxr.mozilla.org/comm-central/source/mozilla/toolkit/components/osfile/modules/osfile_unix_front.jsm?rev=b777166f71b0=1064-1064#1052>

Phil


It seems
Bug 994951 - Activate OS.File xpcshell tests on B2G
changed the code somewhat, but probably it is not the culprit.

I am not sure how the code can return null if one of the first if 
conditions are true (and the last else clause makes me a little 
uncomfortable, but we trust system calls. Erh?)


But what perplexes me is that sometimes this bug is not triggered and 
sometimes it is. (Oh maybe it is likely to happen on optimized build 
than on the debug build.I see the error in OSX 10.6 opt and 10.10 opt in 
the particular tryserver run.  Hard to say at this moment.)




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Question: "Chrome file doesn't exist: ..." in |make mozmill| log by running DEBUG versions of C-C TB

[ISHIKAWA,chiaki]

On 2015/11/24 20:21, Philip Chee wrote:

On 22/11/2015 03:20, ISHIKAWA,chiaki wrote:



   35 Chrome file doesn't exist:
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/toolkit/skin/classic/global/icons/information-64.png

information-64.png only exists in the osx theme
http://mxr.mozilla.org/comm-central/source/mozilla/toolkit/themes/osx/global/jar.mn#104
http://mxr.mozilla.org/comm-central/source/mozilla/toolkit/themes/osx/global/global.css#58

Windows should use:
"chrome://global/skin/icons/information-32.png"

Linux should use:
"moz-icon://stock/gtk-dialog-info?size=dialog"

This appears to be a problem only in Thunderbird and Instantbird.
http://mxr.mozilla.org/comm-central/search?string=information-64.png

There appears to be a informationGhosted-64.png that isn't used by
anything. Suggest rm
http://mxr.mozilla.org/comm-central/find?string=informationGhosted-64.png=comm-central=

Phil


Thank you. I will look into this.

Less cluttered log file is much nicer!

CI


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Question: "Chrome file doesn't exist: ..." in |make mozmill| log by running DEBUG versions of C-C TB

[ISHIKAWA,chiaki]

On 2015/11/23 14:00, Philip Chee wrote:

On 22/11/2015 03:20, ISHIKAWA,chiaki wrote:


  100 Chrome file doesn't exist:
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/classic/skin/classic/messenger/icons/dropbox.png

Was removed in Bug 763471 - Convert Dropbox Filelink provider code to addon

Try using box-logo.png instead.




Replacing the said filename in a test javascript source file caused the
message to go away!


Chrome file doesn't exist (checked since Apri 2, 2015)



113 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/messenger/content/messenger/messengercompose/null
 35 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/toolkit/skin/classic/global/icons/information-64.png
  2 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/toolkit/skin/classic/mozapps/plugins/notifyPluginOutdated.png
  2 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/toolkit/skin/classic/mozapps/plugins/notifyPluginBlocked.png
  2 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/classic/skin/classic/messenger/messengercompose/compose-toolbar.png
  2 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/classic/skin/classic/messenger/icons/mail-toolbar-small.png
  1 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/toolkit/skin/classic/mozapps/extensions/stripes-info-negative.png


I will file a bugzilla entry.

Does anybody have an idea of the issue of

> 113 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/messenger/content/messenger/messengercompose/null


"null" can't be right.

TIA

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Question: why an external DTD is accessed (or not)? (svg11.dtd)

[ISHIKAWA,chiaki]

Hi,

I have been puzzled at a pair of strange warning messages over the last 
few years, but can't hold the curiosity any longer.


So here is my question.

During the invocation of TB |make mozmill| test suite by running locally 
produced DEBUG version of C-C test, I have seen the following error 
messages in the log.

Below, 35 is the number of times the error was seen in the latest run.
The next [4482] is a sample of the process ID and should be ignored.

 35 [4482] WARNING: Failed to open external DTD: publicId 
"-//W3C//DTD SVG 1.1//EN" systemId 
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd; base 
"file:///NREF-COMM-CENTRAL/comm-central/mail/themes/linux/mail/tabs/selected-start.svg" 
URL "resource://gre/res/dtd/svg11.dtd": file 
/NREF-COMM-CENTRAL/comm-central/mozilla/parser/htmlparser/nsExpatDriver.cpp, 
line 705


 35 [4482] WARNING: Failed to open external DTD: publicId 
"-//W3C//DTD SVG 1.1//EN" systemId 
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd; base 
"file:///NREF-COMM-CENTRAL/comm-central/mail/themes/linux/mail/tabs/selected-end.svg" 
URL "resource://gre/res/dtd/svg11.dtd": file 
/NREF-COMM-CENTRAL/comm-central/mozilla/parser/htmlparser/nsExpatDriver.cpp, 
line 705


It seems that the  TB binary seems to try to access external DTD and 
failed(?).


From what I read in the past, W3C's position seems to be that
the web resource of W3C should not be accessed during the runtime for 
obvious performance reasons, and the application is encouraged to have

a local copy anyway.
But if my reading of the code was correct, it was as if TB by way of 
mozilla library was doing just that (?)
Or is it simply TB's packaging forgets to place svg11.dtd in the 
expected place?


Any sensible explanation why this is happening (or not) and why we have
this pair of warning messages printed during the execution of the DEBUG 
version of C-C TB is appreciated.


These messages were visible in January 2013 and possibly earlier.
I have deleted the log due to space reasons :-(

Actually, I see another failure of opening external DTD, but it occurs 
just once, so it is not THAT disturbing.

I can wait for the explanation for *THAT*: xhtml11.dtd.
(Maybe the explanation for the pair of messages would solve the puzzle 
for this one, too.)


  1 [5300] WARNING: Failed to open external DTD: publicId "" 
systemId "resource:///res/dtd/xhtml11.dtd" base 
"moz-nullprincipal:{e2a85fc4-28dc-412e-befa-2feaea7c9705}" URL 
"resource://gre/res/dtd/xhtml11.dtd": file 
/NREF-COMM-CENTRAL/comm-central/mozilla/parser/htmlparser/nsExpatDriver.cpp, 
line 705


The last message started to appear in my local log in March 2013 for the 
first time. Maybe it might have something to do with moz-nullprincipal 
thingy (for security hardening).



TIA



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Question: why an external DTD is accessed (or not)? (svg11.dtd)

[ISHIKAWA,chiaki]

Thank you for the clarification.

It is assuring to know that some people are aware of the issue.


On 2015/11/22 3:59, Kyle Huey wrote:

Bug 994305 is on file for this.

- Kyle

On Sat, Nov 21, 2015 at 10:57 AM, ISHIKAWA,chiaki <ishik...@yk.rim.or.jp> wrote:

Hi,

I have been puzzled at a pair of strange warning messages over the last few
years, but can't hold the curiosity any longer.

So here is my question.

During the invocation of TB |make mozmill| test suite by running locally
produced DEBUG version of C-C test, I have seen the following error messages
in the log.
Below, 35 is the number of times the error was seen in the latest run.
The next [4482] is a sample of the process ID and should be ignored.

  35 [4482] WARNING: Failed to open external DTD: publicId "-//W3C//DTD
SVG 1.1//EN" systemId "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd;
base
"file:///NREF-COMM-CENTRAL/comm-central/mail/themes/linux/mail/tabs/selected-start.svg"
URL "resource://gre/res/dtd/svg11.dtd": file
/NREF-COMM-CENTRAL/comm-central/mozilla/parser/htmlparser/nsExpatDriver.cpp,
line 705

  35 [4482] WARNING: Failed to open external DTD: publicId "-//W3C//DTD
SVG 1.1//EN" systemId "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd;
base
"file:///NREF-COMM-CENTRAL/comm-central/mail/themes/linux/mail/tabs/selected-end.svg"
URL "resource://gre/res/dtd/svg11.dtd": file
/NREF-COMM-CENTRAL/comm-central/mozilla/parser/htmlparser/nsExpatDriver.cpp,
line 705

It seems that the  TB binary seems to try to access external DTD and
failed(?).

 From what I read in the past, W3C's position seems to be that
the web resource of W3C should not be accessed during the runtime for
obvious performance reasons, and the application is encouraged to have
a local copy anyway.
But if my reading of the code was correct, it was as if TB by way of mozilla
library was doing just that (?)
Or is it simply TB's packaging forgets to place svg11.dtd in the expected
place?

Any sensible explanation why this is happening (or not) and why we have
this pair of warning messages printed during the execution of the DEBUG
version of C-C TB is appreciated.

These messages were visible in January 2013 and possibly earlier.
I have deleted the log due to space reasons :-(

Actually, I see another failure of opening external DTD, but it occurs just
once, so it is not THAT disturbing.
I can wait for the explanation for *THAT*: xhtml11.dtd.
(Maybe the explanation for the pair of messages would solve the puzzle for
this one, too.)

   1 [5300] WARNING: Failed to open external DTD: publicId "" systemId
"resource:///res/dtd/xhtml11.dtd" base
"moz-nullprincipal:{e2a85fc4-28dc-412e-befa-2feaea7c9705}" URL
"resource://gre/res/dtd/xhtml11.dtd": file
/NREF-COMM-CENTRAL/comm-central/mozilla/parser/htmlparser/nsExpatDriver.cpp,
line 705

The last message started to appear in my local log in March 2013 for the
first time. Maybe it might have something to do with moz-nullprincipal
thingy (for security hardening).


TIA



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Question: "Chrome file doesn't exist: ..." in |make mozmill| log by running DEBUG versions of C-C TB

[ISHIKAWA,chiaki]

Here is another question of mine that has bothered me for a long time.

During the invocation of TB |make mozmill| test suite by running locally 
produced DEBUG version of C-C test, I have seen the following WARNING 
messages in the log regarding "Chrome file doesn't exist".


I checked the bugzilla this time around, but
I don't think anybody mentioned the following messages I observed.

Found mentions in bugzilla:
612796  Chrome file doesn't exist: 
$OBJDIR/_tests/reftest/reftest/chrome/reftest/content/autoconf.js 
2010-11-16

1105713 "Chrome file doesn't exist head.js" in tests 2014-12-23
762842 Chrome file doesn't exist: mozapps/update/warning.gif 2012-06-09
954129 Chrome file doesn't exist: branding/icon48.png 2013-12-29
1133261Chrome file doesn't exist: …/mailCore.js 2015-02-14
1191778Chrome file doesn't exist: 
.../NightlyDebug.app/Contents/Resources/chrome/toolkit/content/mozapps/preferences/preferences.css 
2015-08-24

6 bugs found.

From the summary in my log:
Again the number at the beginning of each line indicates the frequency 
of occurences.


Please note that the most frequent (132)WARNING message had "null" as 
its last component of

the file path. I suspect there is something wrong in the code.
I suspect a null pointer is passed and "null" is produced as the result 
in the pathname, but I am not sure.


The second most frequent message refers to dropbox.png (!?).
Yers, I have dropbox installed, but how does TB know about it and why it 
wants to access dropbox icon or something?


Some names such as notifyPluginBlocked.png
suggests we are missing an important icon for notifying the user that 
some plugin update status (this case "blocked")?




Chrome file doesn't exist


132 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/messenger/content/messenger/messengercompose/null
100 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/classic/skin/classic/messenger/icons/dropbox.png
 35 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/toolkit/skin/classic/global/icons/information-64.png
  2 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/toolkit/skin/classic/mozapps/plugins/notifyPluginOutdated.png
  2 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/toolkit/skin/classic/mozapps/plugins/notifyPluginBlocked.png
  2 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/classic/skin/classic/messenger/messengercompose/compose-toolbar.png
  2 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/classic/skin/classic/messenger/icons/mail-toolbar-small.png
  1 Chrome file doesn't exist: 
/NREF-COMM-CENTRAL/objdir-tb3/dist/bin/chrome/toolkit/skin/classic/mozapps/extensions/stripes-info-negative.png


Again, any sensible explanation why these are accessed (especially the 
dubious "null" file name at the top) is appreciated.


This is tested under Debian GNU/Linux 64-bit (aka amd64) version.

If I can get rid of 200- warning lines, so much the better.

TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Error of OS X build (c-c TB)

[ISHIKAWA,chiaki]

On 2015/11/16 17:03, ISHIKAWA,chiaki wrote:

On 2015/11/15 14:28, Xidorn Quan wrote:
On Sun, Nov 15, 2015 at 1:05 PM, ISHIKAWA,chiaki 
<ishik...@yk.rim.or.jp> wrote:

On 2015/11/14 12:31, Xidorn Quan wrote:

Have you checked your log here:

https://treeherder.mozilla.org/logviewer.html#?job_id=11934=try-comm-central 



- Xidorn


Yes, I have looked at it.

The information there is sparse. The error indicates there is ambiguity
resolving TextArea symbol.

The information from the log looks fairly enough to me. The content of
AERegistry.h around the location actually has nothing interesting:

struct TextRange {
  SInt32  fStart;
  SInt32  fEnd;
  SInt16  fHiliteStyle;
};
typedef struct TextRangeTextRange;
typedef TextRange * TextRangePtr;
typedef TextRangePtr *  TextRangeHandle;
But, since the same ambiguous error does not occur in linux build,
I can only guess that there are similarly named data type or 
something in
OS X specific headers which is confusing the compiler (clang?), and 
thus
I want to know if someone with OS X developer source files can 
figure this

out.
I am not sure why others don't bump into this issue under OS X.
My patch does NOT touch anything related to "TextArea", etc.

It seems you added some #include which did not exist before which
causes this issue. You may want to reorder the #includes in
nsStreamUtils.cpp to workaround. I don't have particularly good
suggestion here.

- Xidorn


Thank you for your help.

I modified the source so that the include is taken out.

Problem is that although my patches will build linux and Windows 
binaries, but under OS X, there seems to be a configure error, even. 
Now I can't proceed to the compilation stage even. Hmm...


https://treeherder.mozilla.org/#/jobs?repo=try-comm-central=8f2de14e9fd2 




ValueError: Item already in manifest: 
chrome/classic/skin/classic/messenger/addressbook/icons/abcard.png

make[2]: *** [configure] Error 1

1047760 Intermittent build failures with "ccache: failed to create 
/builds/ccache/tmp (No such file or directory)" ("make[3]: *** 
[configure] Error 1", "make[2]: *** [configure] Error 1" or "make: *** 
[configure] Error 1")


make[1]: *** 
[/builds/slave/tb-try-c-cen-m64-d-000/build/objdir-tb/Makefile] Error 
2



The particular error does not seem to be correct. (bug 1047760).



I removed the newly introduced include as suggested AND
after removing the duplication (of two entries)
in theme/osx/jar.mn (this has also been
addressed by a new patch in bugzilla),
I could build C-C TB under OS X on Try Server (!)

https://treeherder.mozilla.org/#/jobs?repo=try-comm-central=c374265b8011

Thank you for your suggestion.

Now, though, the following bug busts testing of all the builds.
It is hard to check the sanity of any patches :-(

*Bug 1224840* <https://bugzilla.mozilla.org/show_bug.cgi?id=1224840> 
-mozmill run: Assertion failure: IsOuterWindow(), at 
/builds/slave/tb-c-cen-lx-d-/build/mozilla/dom/base/nsGlobalWindow.cpp:7678


One more to go.

Thank you again.

CI
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Error of OS X build (c-c TB)

[ISHIKAWA,chiaki]

On 2015/11/15 14:28, Xidorn Quan wrote:

On Sun, Nov 15, 2015 at 1:05 PM, ISHIKAWA,chiaki <ishik...@yk.rim.or.jp> wrote:

On 2015/11/14 12:31, Xidorn Quan wrote:

Have you checked your log here:

https://treeherder.mozilla.org/logviewer.html#?job_id=11934=try-comm-central

- Xidorn


Yes, I have looked at it.

The information there is sparse. The error indicates there is ambiguity
resolving TextArea symbol.

The information from the log looks fairly enough to me. The content of
AERegistry.h around the location actually has nothing interesting:

struct TextRange {
  SInt32  fStart;
  SInt32  fEnd;
  SInt16  fHiliteStyle;
};
typedef struct TextRangeTextRange;
typedef TextRange * TextRangePtr;
typedef TextRangePtr *  TextRangeHandle;
But, since the same ambiguous error does not occur in linux build,
I can only guess that there are similarly named data type or something in
OS X specific headers which is confusing the compiler (clang?), and thus
I want to know if someone with OS X developer source files can figure this
out.
I am not sure why others don't bump into this issue under OS X.
My patch does NOT touch anything related to "TextArea", etc.

It seems you added some #include which did not exist before which
causes this issue. You may want to reorder the #includes in
nsStreamUtils.cpp to workaround. I don't have particularly good
suggestion here.

- Xidorn


Thank you for your help.

I modified the source so that the include is taken out.

Problem is that although my patches will build linux and Windows 
binaries, but under OS X, there seems to be a configure error, even. Now 
I can't proceed to the compilation stage even. Hmm...


https://treeherder.mozilla.org/#/jobs?repo=try-comm-central=8f2de14e9fd2


ValueError: Item already in manifest: 
chrome/classic/skin/classic/messenger/addressbook/icons/abcard.png

make[2]: *** [configure] Error 1

1047760 Intermittent build failures with "ccache: failed to create 
/builds/ccache/tmp (No such file or directory)" ("make[3]: *** 
[configure] Error 1", "make[2]: *** [configure] Error 1" or "make: *** 
[configure] Error 1")


make[1]: *** 
[/builds/slave/tb-try-c-cen-m64-d-000/build/objdir-tb/Makefile] 
Error 2



The particular error does not seem to be correct. (bug 1047760).

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Error of OS X build (c-c TB)

[ISHIKAWA,chiaki]

On 2015/11/15 11:05, ISHIKAWA,chiaki wrote:

On 2015/11/14 12:31, Xidorn Quan wrote:

Have you checked your log here:
https://treeherder.mozilla.org/logviewer.html#?job_id=11934=try-comm-central 



- Xidorn


Yes, I have looked at it.

The information there is sparse. The error indicates there is ambiguity
resolving TextArea symbol.

But, since the same ambiguous error does not occur in linux build,
I can only guess that there are similarly named data type or something in
OS X specific headers which is confusing the compiler (clang?), and thus
I want to know if someone with OS X developer source files can figure 
this out.

I am not sure why others don't bump into this issue under OS X.
My patch does NOT touch anything related to "TextArea", etc.



It may be that my patch defines DEBUG (and a few other symbols which I 
am not aware of due to preprocessor magic?) and that my conditionally 
define other symbols in OS X header files, and thus caused the 
compilation issue.

But again, without the source, I have no idea.

TIA

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Error of OS X build (c-c TB)

[ISHIKAWA,chiaki]

On 2015/11/14 12:31, Xidorn Quan wrote:

Have you checked your log here:
https://treeherder.mozilla.org/logviewer.html#?job_id=11934=try-comm-central

- Xidorn


Yes, I have looked at it.

The information there is sparse. The error indicates there is ambiguity
resolving TextArea symbol.

But, since the same ambiguous error does not occur in linux build,
I can only guess that there are similarly named data type or something in
OS X specific headers which is confusing the compiler (clang?), and thus
I want to know if someone with OS X developer source files can figure 
this out.

I am not sure why others don't bump into this issue under OS X.
My patch does NOT touch anything related to "TextArea", etc.

TIA

On Sat, Nov 14, 2015 at 1:05 PM, ISHIKAWA,chiaki <ishik...@yk.rim.or.jp> wrote:

Hi,

I am trying to test my patches by building C-C TB under OS X using try
server.
However, the build fails with the following compilation error.
Since I don't have OS X (my local PC runs linux), I have no idea why the
error occurs under OS X.
BTW, I can build C-C TB under linux on my local PC, and have been
testing the patches locally.

Can someone who has OS X show me what the source line near AERegistry.h
(near line 693 et al) are like and even better suggest what the possible
problems are?
(I don't know if different OS X versions have very different source
files, but at least if someone looks for |TextRange|, we can home in the
problem.)

System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AERegistry.h:693:16:
error: reference to 'TextRange' is ambiguous
/System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AERegistry.h:694:9:
error: reference to 'TextRange' is ambiguous
/System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AERegistry.h:698:3:
error: reference to 'TextRange' is ambiguous
/System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AERegistry.h:698:23:
error: field has incomplete type 'mozilla::TextRange'
/System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AERegistry.h:700:16:
error: reference to 'TextRangeArray' is ambiguous
/System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AERegistry.h:701:9:
error: reference to 'TextRangeArray' is ambiguous
make[5]: *** [Unified_cpp_xpcom_io1.o] Error 1
make[5]: *** Waiting for unfinished jobs
make[4]: *** [xpcom/io/target] Error 2
make[4]: *** Waiting for unfinished jobs
make[3]: *** [compile] Error 2
make[2]: *** [default] Error 2
make[1]: *** [build] Error 2
make: *** [build] Error 2

I suspect maybe some macros I define may have unexpected mutation of
some symbols used by
OS X or that some scoping issues are introduced.

TIA

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Error of OS X build (c-c TB)

[ISHIKAWA,chiaki]

Hi,

I am trying to test my patches by building C-C TB under OS X using try
server.
However, the build fails with the following compilation error.
Since I don't have OS X (my local PC runs linux), I have no idea why the
error occurs under OS X.
BTW, I can build C-C TB under linux on my local PC, and have been
testing the patches locally.

Can someone who has OS X show me what the source line near AERegistry.h
(near line 693 et al) are like and even better suggest what the possible
problems are?
(I don't know if different OS X versions have very different source
files, but at least if someone looks for |TextRange|, we can home in the
problem.)

System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AERegistry.h:693:16:
error: reference to 'TextRange' is ambiguous
/System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AERegistry.h:694:9:
error: reference to 'TextRange' is ambiguous
/System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AERegistry.h:698:3:
error: reference to 'TextRange' is ambiguous
/System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AERegistry.h:698:23:
error: field has incomplete type 'mozilla::TextRange'
/System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AERegistry.h:700:16:
error: reference to 'TextRangeArray' is ambiguous
/System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AERegistry.h:701:9:
error: reference to 'TextRangeArray' is ambiguous
make[5]: *** [Unified_cpp_xpcom_io1.o] Error 1
make[5]: *** Waiting for unfinished jobs
make[4]: *** [xpcom/io/target] Error 2
make[4]: *** Waiting for unfinished jobs
make[3]: *** [compile] Error 2
make[2]: *** [default] Error 2
make[1]: *** [build] Error 2
make: *** [build] Error 2

I suspect maybe some macros I define may have unexpected mutation of
some symbols used by
OS X or that some scoping issues are introduced.

TIA

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Merging comm-central into mozilla-central (summary v1)

[ISHIKAWA,chiaki]

On 2015/10/27 9:06, Philipp Kewisch wrote:

Hi all,

I'd love to see if we can move towards an agreement. For those of you
that would prefer not to merge, I'd love to hear what your absolute
minimum requirements would be that you'd accept a merge with. Changes to
hg? Changes to dxr? A policy chanage? If we can establish clear
requirements, maybe we can see they are not so hard to fulfill and we
can come to an agreement quickly. Ideally, I'd like to find a middle
ground here that is acceptable to everyone.

Let me summarize requirements I have read:

* Thunderbird bustages should not close Firefox trees
* A version of dxr that does not show comm projects
* A version of dxr that shows all projects
* Treeherder remains separate
* Normal m-c try pushes don't automatically build Thunderbird
* There should be a commitment that comm projects will be regularly
maintained at least in the foreseeable future

* Thunderbird builds should not run on every m-c/m-i commit
* The default checkout should not contain comm projects
* It does not make sense if Thunderbird will be separate short term
* There should be least amount of pressure to fix Thunderbird things for
Firefox contributors


If these are the requirements, with the exception of narrow clones which
as I've understood are not stable yet, I believe they are all fairly
easy to fulfill.

Are there any other requirements I've missed?

Philipp



I think the above summarizes the discussions and counter-points raised 
rather well.


Although I don't hear from many volunteers who send in TB patches 
occasionally
in this thread, I believe many of them would work hard to make the 
following a reality:


> There should be a commitment that comm projects will be regularly
> maintained at least in the foreseeable future

[Of course, there is NO contract by the user community, so "commitment" is a 
rather
vague term here. I can't speak for others.]

I, for one, use TB for regular work at the office and at home regularly, and
have come to depend on it. So any serious bugs that I find (and there are many
unfortunately) *HAVE* to be fixed eventually and so I try to send patches.
Some may send in patches to fix these bugs, and others may send in patches
to bring in new features. But if there are a million users out there,
I bet there are a couple hundred volunteers who can program. (I think there was 
a statistics
compiled by someone to show the user community contribution in terms of patches 
sent in
by individuals in the last few years after Mozilla foundation put TB at the 
mercy of
user community.)

And to make this maintenance easier (less breakage of tree herder compilation, 
less duplication of slightly different similar documents etc.), the merge is 
preferable from the standpoint of developers IMHO.

TIA

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Strange Error: Reentrancy error: some client, attempted to display a message to the console while in a console,listener

[ISHIKAWA,chiaki]

On 2015/07/02 4:30, Andrew Sutherland wrote:

This is probably due to
https://dxr.mozilla.org/comm-central/source/mailnews/test/resources/logHelper.js
that registers itself as a console listener and its low-tech feedback
loop prevention.  (NB: The quippy file-level comment should be
s/aweswome/Andrew/ for the third instance of awesome.)


Thank you for the comment (I mean the post to this mailing list).
I understand such recursion-prevention is necessary.

Many years ago, I recall Emacs blew up when it reached low-memory 
condition. It detected this low-memory condition early on to warn the 
users in advance. But when it tried to print out a warning about 
low-memory condition and the user might want to save the buffers and 
start over, the print routine which created a small buffer to store the 
lengthy warning message and show it on the screen kicked invoked ANOTHER 
invocation of memory allocation function, which in turn detected the 
low-memory condition and ... ad infinitum. Today, to avoid blow-up in 
the particular execution path, Emacs pre-allocates the necessary 
low-level data structure to avoid blowup (e.g., error message constant 
string is pre-created as Emacs Lisp string, etc.)


So that part is understandable.

See

http://www.visophyte.org/blog/tag/logsploder/ for context on the various
logsploder references and the purpose of most of the machinery in there.
  It might be easier to remove the functionality than to try and make it
smarter since I rather doubt anyone uses any of the fancy logging stuff
anymore.


Hmm, it seems that C-C TB's mozmill test suite uses it still.



Andrew


Thank you again for the background on the machineary of log handling.

I am still wondering why on earth Components is not defined in 
head.js. Something is screwed up there.


TIA







___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Strange Error: Reentrancy error: some client, attempted to display a message to the console while in a console,listener

[ISHIKAWA,chiaki]

On 2015/07/01 15:53, Robert Strong wrote:

I've only seen those when anything tries to print to the console after a
test has called do_test_finish and in that case I suspect it is due to
xpcshell shutting down, etc.

Robert



Thank you for your observation.
I will investigate if this happens towards the test termination.

I am still at a loss about  why the strange syntax error (missing symbol 
definition) occurs in the first place. Missing Components is

really strange.

TIA


On Tue, Jun 30, 2015 at 11:35 PM, ishikawa ishik...@yk.rim.or.jp wrote:


Hi,

I am not sure where to ask and so ask three mailing lists.

First off, TB 38.0.1 after many months of waiting  solved many outstanding
issues,
Great.
I hope we can correct many remaining issues in the months ahead.

I am testing full debug build of C-C TB locally.
This is done by running
|make xpcshell-tests| to run xpcshell unit tests.

I have noticed a few new errors in the last week or so after the code
has been refreshed.

These errors/warnings are new additions and I am trying to eliminate them.

A case in point. A strange error I have never seen before.

PROCESS | 14289 | [14289] WARNING: Reentrancy error: some client
attempted to display a message to the console while in a console
listener. The following message was discarded: [JavaScript Error:
TypeError: do_print is not a function {file:
/NREF-COMM-CENTRAL/comm-central/mozilla/testing/xpcshell/head.js line:
130}]: file
/NREF-COMM-CENTRAL/comm-central/mozilla/xpcom/base/nsConsoleService.cpp,
line 198


I looked at nsConsoleService.cpp:
comm-central/mozilla/xpcom/base/nsConsoleService.cpp, line 198


193   if (NS_IsMainThread()  mDeliveringMessage) {
194 nsCString msg;
195 aMessage-ToString(msg);
196 NS_WARNING(nsPrintfCString(Reentrancy error: some client
attempted 
197   to display a message to the console while in a console
listener. 
198   The following message was discarded: \%s\, msg.get()).get());
199 return NS_ERROR_FAILURE;
200   }

My understanding: It looks that when a thread tries to print
something to the console while it is in a console listener, the
code, in this case, C-C TB suppresses/discards the new message because
such printing may lead to an infinite recursion as in this case.

Basically, here a warning is going to be printed by the routine that
was invoked, but then since |do_print| is not deemed a function in
JS-side of execution, and the thread would have called the display
routine again, and during new invocation, there probably this warning
about missing function
definition of |do_print| is regenerated, ... ad infinitum. (This is a
possibility and the code tries to avoid invoking output to the console
recursively.)


Now, this warning about suppressing/discarding error message is only
printed for DEBUG BUILD (|NS_WARNING| is used).

I think that is why not many people seem to have noticed this issue.

I am afraid this is a serious issue since a proper error message about
printing mechanism is getting lost.

So anyway, then I checked head.js. That is where mysterious errors are
produced.

comm-central/ mozilla/ testing/ xpcshell/ head.js

I found the situation strange because |do_print| is declared as a
function at the top level if I am not mistaken.

Maybe the error is generated before |do_print()| is defined?


621 /**
622  * Prints a message to the output log.
623  */
624 function do_print(msg, data) {
625   msg = _wrap_with_quotes_if_necessary(msg);
626   data = data ? data : null;
627   _testLogger.info(msg, data);
628 }

And it was referenced in |observe|

129 observe : function (msg) {
130   do_print(CONSOLE_MESSAGE: ( + levelNames[msg.logLevel] + )
 + msg.toString());
131 }


Funny, thought I. This is because |do_print| ought to be visible to
|observe| (*IIF* it has been defined at runtime before |observe| is
called.)
I moved the definition of |do_print| to an earlier position, but
It did not seem to make any difference.

But I thought, OK, here is a listener, and maybe the runtime
context/closure/whatever under which |observe| may operate is not
quite the same as that of the definition time, and I copied the
|do_print| definition to |observe| body so that it *IS* always visible.

This was done by adding the following inside|observe| body just before
|do_print| is called.

   let do_print = function (msg, data) {
 msg = _wrap_with_quotes_if_necessary(msg);
 data = data ? data : null;
 _testLogger.info(msg, data);
   }

Of course, this shut up the warning/error about |do_print|.
But then I have complaints about |_wrap_with_quotes_if_necessary| and
|_testLogger| naturally.

When I copied the definition of other functions such as
|_wrap_with_quotes_if_necessary| and |_testLogger|, using id =
function (arg) {...} construct one by one as the error message
suggests |observe| is missing, eventually I could eliminate all the
warnings except for one about |Components|

PROCESS | 3996 | JavaScript error:

Re: The War on Warnings

[ISHIKAWA,chiaki]

On 2015/06/17 8:29, Eric Rahm wrote:

An update on progress. I've landed bugs which should clean up ~180,000 
warnings. I have bugs pending for another ~26,000.

The latest top 40:

*Note: I improved my normalization a bit so it might look a bit different



Are there documents explaining why
these warnings are generated?
I mean, say, I see the first warning during the run of C-C TB's |make 
mozmill| test and I wonder, what C-C TB is doing wrong to see this 
Subdocument container has no frame?


Maybe each month, we should pick up the top-10 warnings (without the 
description given below) in the frequency, and ask the author of the 
patch that inserted the warning to
explain the following: this will help the testers to asses the 
situation, and more importantly, help people maintain the legacy code 
such C-C TB to figure out what can be possibly wrong in the existing 
code and how we can fix them.:


 - severity of the message: INFO, DEBUG, WARNING, etc.
 - explain why there is a warning (background),
 - is there going to be a bad thing happening soon after
   the situation occurs? what bad symptoms should we expect?
 - what can a caller do to prevent the bad thing happening (I mean
   the caller of the invocation chain that eventually leads to the
   particular warning.)

Without such information, warnings are not so useful in the log.

I see about 25 of the following warnings in C-C TB |make mozmill| run.
And I have been wondering OK, if Subdocument container has no frame, 
is this bad?, is there going to be something bad?, is this message 
printed because C-C TB code invokes document API incorrectly?, if so, 
what is the correct procedure, and such.


As a few mentioned that there are simply so many warnings in the log and 
we are overwhelmed, but anything that can help us to cope with the tidal 
wave of warnings ought to be implemented and tried.


And among the warnings, there *ARE* warnings that lead to real bugs.
e.g.:
	 1627 [N] WARNING: Called close() before start()!: 'mStarted', 
file dom/workers/Message

Obviously, the program is sloppy and calling
some functions in incorrect order. We should fix this.

 1149 [N] WARNING: RemoveObserver() called for 
unregistered observer: file hal/Hal.cpp, line 205

Again, the program is sloppy and is not keeping track of
its resources and thus calling non-existing, phantom observer
to be removed. This ought to be fixed.

Another real bug is this.
Although it did not seem to be in the top 40, I see a few types of 
messages that make me thing that the shutting down of thread are not

properly synchronized at the end of the program termination.
Obviously some threads try to dump data while the subsystem/module that 
handles the writing of the data is already closed/shutdown and the 
writing fails, etc. This ought to be fixed.


These high volume warnings can be very useful if only the description 
about them can be found easily.


I am not claiming the description for EVERY warning, but
we can start with top-most 10 warnings (without such description).


18012 [N] WARNING: Subdocument container has no frame: file 
layout/base/nsDocumentViewer.cpp, line 2520
14816 [N] WARNING: NS_ENSURE_SUCCESS(EnsureScriptEnvironment(), nullptr) 
failed with result 0x: file docshell/base/nsDocShell.cpp, line 4605
9809 [N] WARNING: No docshells for remote frames!: file 
dom/base/nsFrameLoader.cpp, line 459
8929 [N] WARNING: Someone passed native anonymous content directly into 
frame construction.  Stop doing that!: file 
layout/base/nsCSSFrameConstructor.cpp, line 6559
8062 [N] WARNING: Suboptimal indexes for the SQL statement 0x 
(http://mzl.la/1FuID0j).: file storage/mozStoragePrivateHelpers.cpp, line 109
7760 [N] WARNING: NS_ENSURE_TRUE(mDocShell) failed: file 
embedding/browser/nsWebBrowser.cpp, line 363
7454 [N] WARNING: anonymous nodes should not be in child lists (bug 
439258): file layout/base/RestyleManager.cpp, line 1439
6294 [N] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x: 
file netwerk/base/nsFileStreams.cpp, line 492
6294 [N] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x: 
file netwerk/base/nsFileStreams.cpp, line 205
6131 [N] WARNING: No outer window available!: file 
dom/base/nsGlobalWindow.cpp, line 3920
5207 [N] WARNING: NS_ENSURE_TRUE(domWindow) failed: file 
embedding/browser/nsDocShellTreeOwner.cpp, line 83
5193 [N] WARNING: NS_ENSURE_TRUE(aInBrowser) failed: file 
embedding/browser/nsDocShellTreeOwner.cpp, line 79
5073 [N] WARNING: zero axis length: file dom/svg/nsSVGLength2.cpp, line 124
4606 [N] WARNING: No widget found in TabParent::UpdateDimensions: file 
dom/ipc/TabParent.cpp, line 974
4574 [N] WARNING: Shouldn't call SchedulePaint in a detached pres context: 
file layout/generic/nsFrame.cpp, line 5181
3891 [N] WARNING: Please do not use mouseenter/leave events in chrome. They 

Re: Revisiting modelines in source files

[ISHIKAWA,chiaki]

Assessing style compliance as == clang-format(whole file) is by far the
easiest to implement. I'd be in favor of a flag day (or set of mini flag
days) where we mass rewrite the tree with clang-format's output.



To be honest, I don't care WHICH particular style we follow, but
please set the flag day soonish so that
mass rewrite is done.

It is really bothering to realize that some existing files do
not follow the supposed mozilla coding style and even
occasionally conflict
with the modeline at the beginning of that file after all
these discussions.

Finding that I have to tweek the indentation every
time I hit the enter key to fix something in a file
is annoying.

Either a flag day for the ENTIRE tree, or by top-most directories, ought 
to be implemented. Maybe someone can check the statistics and
the month where the number of patch submissions is lowest ought to be 
picked up and the flag day can be set there.


Just a thought.

BTW, I use emacs and it can handle a lot of different styles.
So emacs users should have no problem except for one thing.

Unfortunately, it does not handle the particular coding style of
#if,#else,#endif in JavaScript source files since
use of C-style macro preprocessor is not quite standard.

Nobody seems to have written a emacs mode for this C-style macro use in 
JavaScript code. Correct me if I am wrong on this point.

(When I tried to indent some broken JavaScript source files
it could not handle #if,#else,#endif, etc. very well.)

I wonder if there is an automatic formatter for JavaScript
that handles this particular issue of non-standard C-preprocessor usage.

If not, we may be forced to rewrite such macro usage into

BEFORE:
#if defined(WIN32)
blabla
#else
gazonk-foobar
#endif

AFTER
if(C.system_win32) {
 blabla
} else {
 gazonk-foobar
}

or something like that (modulo the correct positioning of {, }, 
else) by introducing system-wide constant that defines
system types. Run-time overhead should be minimal given the modern JS 
interpreter and optimizing JIT compiler.


Just a thought.



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: The War on Warnings

[ISHIKAWA,chiaki]

Hi,

On 2015/06/06 2:39, Eric Rahm wrote:

On Friday, June 5, 2015 at 8:23:53 AM UTC-7, ISHIKAWA,chiaki wrote:

After coping with voluminous messages in C-C TB |make mozmill| test
suite log [much smaller volume than full FF logs],
I think we should have NS_INFORMATION() macro that
prints out an information for someone who is developing code and
just wants to know how the code is doing.


I suppose we could add yet another macro, but really this sounds like a use 
case for:

   MOZ_LOG(myModule, LogLevel::Info, Dev message!);

which could then be enabled with something like:

   |NSPR_MODULES=myModule:3 ./mach run|



Thank you for the tips.

(I was referring to *NS*_INFORMATION() since
when I tried to use MOZ_ABORT() or something in some test codes,
I could not compile and link them and learned some MOZ_* macros and NS_* 
macros don't mix very well during linking(?).)


Anyway, I am a little confused since MOZ_LOG() is very new.

What is the first argument to the following code?
|MOZ_LOG(myModule, LogLevel::Info, Dev message!);|


Looking at how PR_LOG() was called with PRLogModuleInfo,

typedef struct PRLogModuleInfo {
   const char *name;
   PRLogModuleLevel level;
   struct PRLogModuleInfo *next;
} PRLogModuleInfo;

Is |myModule| it declared as

PRLogModuleInfo myModule;

is passed to as the first argument of MOZ_LOG().?
But it sounds as if we should stay away from PRLog* data structure.

Or is it simply anything (and stringified internally) that
will be specified in environmental variable later?



It might be useful to eventually add a |MOZ_LOG_IF| to make things easier.



Yes, for readability, we need some creative macros.


 1576 [10262] WARNING: '!mMainThread', file
/REF-COMM-CENTRAL/comm-central/mozilla/xpcom/threads/nsThreadManager.cpp, line


We're working on this in bug 1171716, it's related to cycle collection on 
shutdown.


Anyway, I agree that we should simply try to attack the top  entries
first to figure out whether they are indication of genuine issues/bugs, etc.


Yes! Please file blocking bugs for the top c-c entries against bug 765224.


I will try to do so.




Eventually we could cut down the number to manageable size.
(At near the beginning of 2013, the list became only about a dozen items
long, but started to grow again. In a sense it reflects the desire of
code authors to catch strange conditions, but I agree we must keep on
looking at the logs and do something about the top offenders in the log
list.)


It might be nice to create a very simple start/stop test that just makes sure 
there are no warnings so that we can avoid regressing that in the future.



Sounds a sensible idea.

TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: The War on Warnings

[ISHIKAWA,chiaki]

On 2015/06/04 21:38, Robert O'Callahan wrote:

Usually I use NS_WARNING to mean something weird and unexpected is
happening, e.g. a bug in Web page code, but not necessarily a browser bug.
Sometimes I get useful hints from NS_WARNING spew leading up to a serious
failure, but for those usages could probably be switched to PR_LOG without
losing much.

Rob



After coping with voluminous messages in C-C TB |make mozmill| test 
suite log [much smaller volume than full FF logs],

I think we should have NS_INFORMATION() macro that
prints out an information for someone who is developing code and
just wants to know how the code is doing.

So many WARNING lines such as the ones below can be turned into
INFORMATION messages and we can probably ignore these lines from NG/GO 
decisions of tests.


Hmm. The particular log I picked from two months ago is not
quite representative.  I specifically excluded NS_ENSURE_TRUE summary 
from below, but for this particular log, the messages near the top of 
list (decreasing frequency order) seem too serious to be INFORMATION kind.


 WARNING: Allow one letter C/c as language code.:
The above line below is from my own attempt to
make C-C TB accpet C.UTF-8 as locale: I think cygwin's default locale is 
this. But mozilla code refuses to accept this claiming that the first 
part of the locale needs to be either two or three letters.
And I used dreaded NS_WARNING() instead of something better: if 
NS_INFORMATION() is there, I will use it.


--- begin quote ---
Second without NS_ENSURE macros:

Count/sum is modulo [without the first PID]

   1576 [10262] WARNING: '!mMainThread', file 
/REF-COMM-CENTRAL/comm-central/mozilla/xpcom/threads/nsThreadManager.cpp, line 
395
868 [10262] WARNING: Failed to retarget HTML data delivery to the 
parser thread.: file 
/REF-COMM-CENTRAL/comm-central/mozilla/parser/html/nsHtml5StreamParser.cpp, 
line 951
693 [10262] WARNING: Subdocument container has no frame: file 
/REF-COMM-CENTRAL/comm-central/mozilla/layout/base/nsDocumentViewer.cpp, 
line 2511
100 [21074] WARNING: We should have hit the document element...: 
file /REF-COMM-CENTRAL/comm-central/mozilla/layout/xul/BoxObject.cpp, 
line 183
 58 [10262] WARNING: unable to post continuation event: file 
/REF-COMM-CENTRAL/comm-central/mozilla/xpcom/io/nsStreamUtils.cpp, line 453
 58 [10262] WARNING: An event was posted to a thread that will 
never run it (rejected): file 
/REF-COMM-CENTRAL/comm-central/mozilla/xpcom/threads/nsThread.cpp, line 505
 45 [10553] WARNING: No inner window available!: file 
/REF-COMM-CENTRAL/comm-central/mozilla/dom/base/nsGlobalWindow.cpp, line 
9658
 36 [10262] WARNING: Re-registering a CID?: file 
/REF-COMM-CENTRAL/comm-central/mozilla/xpcom/components/nsComponentManager.cpp, 
line 531
 36 [10262] WARNING: Hardware Vsync support not yet implemented. 
Falling back to software timers
 36 [10262] WARNING: Failed to open external DTD: publicId 
-//W3C//DTD SVG 1.1//EN systemId 
http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd; base 
file:///REF-COMM-CENTRAL/comm-central/mail/themes/linux/mail/tabs/selected-start.svg 
URL resource://gre/res/dtd/svg11.dtd: file 
/REF-COMM-CENTRAL/comm-central/mozilla/parser/htmlparser/nsExpatDriver.cpp, 
line 703
 36 [10262] WARNING: Failed to open external DTD: publicId 
-//W3C//DTD SVG 1.1//EN systemId 
http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd; base 
file:///REF-COMM-CENTRAL/comm-central/mail/themes/linux/mail/tabs/selected-end.svg 
URL resource://gre/res/dtd/svg11.dtd: file 
/REF-COMM-CENTRAL/comm-central/mozilla/parser/htmlparser/nsExpatDriver.cpp, 
line 703
 36 [10262] WARNING: Allow one letter C/c as language code.: file 
/REF-COMM-CENTRAL/comm-central/mozilla/intl/locale/unix/nsPosixLocale.cpp, 
line 128
 35 [10262] WARNING: unable to Flush() dirty datasource during 
XPCOM shutdown: file 
/REF-COMM-CENTRAL/comm-central/mozilla/rdf/base/nsRDFXMLDataSource.cpp, 
line 754
 35 WARNING: YOU ARE LEAKING THE WORLD (at least one JSRuntime and 
everything alive inside it, that is) AT JS_ShutDown TIME.  FIX THIS!
 32 [21074] WARNING: Must complete empty transaction when 
compositing!: file 
/REF-COMM-CENTRAL/comm-central/mozilla/layout/base/nsPresShell.cpp, line 
6257
 30 [21074] WARNING: Someone passed native anonymous content 
directly into frame construction.  Stop doing that!: file 
/REF-COMM-CENTRAL/comm-central/mozilla/layout/base/nsCSSFrameConstructor.cpp, 
line 6550
 28 [21074] WARNING: nsMsgProtocol::SetContentCharset() not 
implemented: file 
/REF-COMM-CENTRAL/comm-central/mailnews/base/util/nsMsgProtocol.cpp, 
line 580
 20 [21074] WARNING: anonymous nodes should not be in child lists 
(bug 439258): file 
/REF-COMM-CENTRAL/comm-central/mozilla/layout/base/RestyleManager.cpp, 
line 1441
 18 [8115] WARNING: There is no observer for invalidformsubmit. 
One should be implemented!: file 
/REF-COMM-CENTRAL/comm-central/mozilla/dom/html/HTMLFormElement.cpp, 
line 

DBM code too old (Re: mozmill test: Hightail ???)

[ISHIKAWA, Chiaki]

I changed the subject:
Followup-to: mozilla.dev.platform

On 2015/05t/29 16:41, ishikawa wrote:
 Hi,
 
 I have been testing locally built FULL DEBUG C-C thunderbird using |make
 mozmill| test suite.
 
 When I tinkered with low-level I/O testing by simulating short read,
 I saw about a couple of dozen new errors in |make mozmill|.
 I already fixed a few to take care of the unhandled short read situations.
 
 (Turns out, short write is handled (at least on Posix), by pt_Write by
 retrying automatically.)
 
 Among the new errors that puzzled me is Hightail test.
 When I simulated short reads, I began experiencing errors from
 test-cloudfile-backend-hightail.js.
 
 Well, I thought that would be another
 debugging session, but I had no idea where to start.
 
 But then, I noticed a posting in
 mozilla-support-thunderb...@lists.mozilla.org
 (or mozilla.support.thunderbird )
 message ID:
 mailman.123.1432754945.30853.support-thunderb...@lists.mozilla.org
 Subject: Re: Have we lost You Send It / Hightail?
 From: WaltS48 thalion...@removeaim.com
 
 That says:

 We have reached an agreement with Box to integrate their services directly 
 into Thunderbird. The agreement with Hightail is no longer valid.

 REF: 
 https://support.mozilla.org/en-US/kb/filelink-large-attachments#w_filelink-faq

 There is a list of extensions for other providers on that page.
 
 Could that mean I don't have to pay much attention to hightail test any 
 more?
 (But |make mozmill| suggests that
 C-C TB seems to talk to the fake simulated hightail server(?) rather well
 if I don't introduce simulated short read, and that puzzles me to no end.)
 
 TIA
 

Hi,

I figured out the particular cause of
test-cloudfile-backend-hightail.js.
experienced during |make mozmill| when I simulated short read and
short write to files
under the profile directory used by the test was that
*.db files managed by Berkeley DB 1.85 (presumably by the code below
mozilla/security/nss/lib/dbm/src/) experienced failures.

That is, the code in mozilla/security/nss/lib/dbm/src/
does not handle the case such as files in remote mount may experience
short read (read() system call returning shorter than expected
characters although enough octets are there to read), or write() system
call returning before all the characters are written.

As a result of my injection of short read/write, the data in these files
during test set up got messed up, and
the intended test did not run successfully.

The DB files used during tests were three files as follows:
My MOZOBJ is /REF-OBJ-DIR/objdir-tb3:

_tests/mozmill/mozmillprofile/cert8.db,
_tests/mozmill/mozmillprofile/key3.db,
_tests/mozmill/mozmillprofile/secmod.db

below my MOZOBJ directory.

After exempting these *.db files from my simulated short read/write,
the tests passed (!).

I noticed the strange I/O to secomod.db first, and figured out it can't
cope with short read/write at all
after reading the source code. So I exempted cert8.db and key3.db, too.

I suspect that anyone who has mounted the profile directory from remote
file system may experience such transient short read/wrte failures.
I now recall seeing such bugzilla entries from people who seem to have a
corrupt db(!) Hmm, but I can find a relevant entry now.

I will check if exempting these *.db files cause OTHER new failures,
which were introduced after short read/write emulation, to disappear.

If so, this is a compelling reason to update the dbm (or db) code
to a newer one. ( I noticed we only use HASH version of db.)

However, now that Sleepycat has been bought by Oracle, I am not sure
what is the good option left.

http://www.oracle.com/us/products/database/berkeley-db/overview/index.html

At least, Oracle seems to give us the source code, but the license-wise,
I am not sure if its fits mozilla's needs.

In the meantime, the failure to handle short read/write which may occur
due to remotely mounted file system is very bad considering the effect
it has on |make mozmill| test.

TIA
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Replacing Mozmill driven tests for Firefox with Marionette

[ISHIKAWA, Chiaki]

Ouch, there was a line near the end.


Successfully installed jsbridge-2.4.14 mozmill-1.5.16


This is from a local C-C TB build.

CI

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

A question about do_QueryInterface()

[ISHIKAWA, Chiaki]

Lately, I refreshed comm-central thunderbird code
and tested my local modification to enable buffering of writing
downloaded message to a local mail store.

(This is about when one uses POP3. Please bear this in mind.
Imap testing is further along.)

I noticed a couple of things:

(1) File API semantic change? : unexpected file pointer position.

First, this could be my mixup of refreshing the source tree
and merging the local modification to create a new set of hg patches,
but has there been a change to file-handling function code so that
a newly opened file's seek position (or file pointer to write or read
for the next operation in other words) be placed at the beginning (i.e.,
0 offset) even if the said file existed and has non-zero value and the
intention is to append?
I didn't write the original code, so am unsure, but the intention was to
append to the file. I don't think I touched that part of the code.

It seems to me that the file pointer was at the end of the file for
writing to Berkely style mbox-format Inbox (so we append to it)
previously before the refresh of source tree.
But after the refresh, I realized it is no longer positioned at the end,
but at 0-th offset, and so a call to |Seek()| before appending is now
indeed necessary.

I have been attempting to remove a call to |seek()| in a busy loop to
write to the Inbox: I don't think this |seek()| is needed.
[1]
The offending |Seek()| nullifies our attempt to write to Inbox with
buffering. This slows down I/O.

After a couple of months testing and debugging on and off, even
monitoring the checking of the file positions before and after |seek|,
I thought I can remove the offending |seek()| safely at least from my
limited testing.

Well what about other people's environment?
Screwing up one's mailbox is something that needs to be avoided at all cost.

Based on the suggestion of aceman, I was about to upload a proposed
patch for Beta to test that it is true that we can remove the |seek()|
and still be safe in other people's environment: the patch is meant to
check that the |seek()| has no effect, i.e., the file pointer position
does not change before and after the offending |seek()|. There are a few
preferences that can change the behavior of TB and so I wanted to make
sure. Once we know for sure that it is safe to remove |Seek()|, I intend
to upload the final patch to remove the |Seek()|.

Above slight change in the file pointer position after opening
mbox-style Inbox, I now need to insert a |Seek| once after we begin
writing to it. Just once.

All is well now.

But I am curious if anyone is aware of such file API change.
I tried to read through the changes to M-C tree for the last week, but
it was long and I could not figure out if the change is in the last week
or not. Thus I have to ask experts here.


(2)  This is more like C++ semantic issue and the way XPCOM is built.

(After I wrote down the following, I found out from system trace, that
maybe my change does not buffer the writing to temporary file still :-(.
I will follow up when the dust settles.)

With a user preference setting of mailnews.downloadToTempFile to true,
thunderbird mail client can be taught to download a POP3 message
first to a temporary file and close it, and then
read the content of this file and append it to mbox-style Inbox.

I think this is meant to deal with arcane very spartan anti-virus
software that would REMOVE/QUARANTINE a file when a virus or malware is
detected. If you receive a spam e-mail with malware, and your whole
Inbox is removed/quarantined, it is a disaster.

I think today's AV software is more benign and handle the situation more
elegantly (I don't know the details.)

Although mailnews.downloadToTempFile does not even exist by default,
and you need to manually set it to test the feature,
the code path to handle this feature DOES exist, and I
needed to test the path in my quest for better I/O error handling in
TB.[2][3][4][5].

When I enabled buffering of writing to a local file during
downloading as in [1], and the local file is temporary (by setting
mailnews.downloadToTempFile to true), I hit a snug.

There is a code in the mailnews.downloadToTempFile is true  path.
http://mxr.mozilla.org/comm-central/source/mailnews/local/src/nsPop3Sink.cpp#787


*   787   nsCOMPtr nsIInputStream inboxInputStream =
do_QueryInterface(m_outFileStream);
788   rv = MsgReopenFileStream(m_tmpDownloadFile, inboxInputStream);

Before, as in the current release, m_outFileStream is not buffered.
And the code on line 787 produces non-null inboxInputStream.

However, once m_outFileStream is turned into a buffered output stream
using, say,

  m_outFileStream = NS_BufferOutputStream(m_outFileStream, 64 * 1024 );

the code on line 787 produces nullptr.

Is this to be expected?

Up until now, I thought of do_QueryInterface() as mere sugar-coating for
certain type-mutation or something. But I now know I am wrong.

I read a page about do_QueryInterface() but it does not

Re: A question about do_QueryInterface()

[ISHIKAWA, Chiaki]

Thank you for the clarification.

On 2015/05/01 3:38, Boris Zbarsky wrote:
 On 4/30/15 2:25 PM, ISHIKAWA, Chiaki wrote:
 Is this to be expected?
 
 Sure.  You're taking an _output_ stream and QIing it to
 nsI_Input_Stream.
 

Yes, that is how the original code was written.

 It might happen that some objects implement both interfaces (and looks
 like nsMsgFileStream  does).  The object returned by
 NS_BufferOutputStream does not: it's an output stream, but not an input
 stream.

Oh, I see. So non-buffered version was OK, but now that I introduce
buffering, it is no longer possible.

 I recommend keeping track of both the input _and_ output stream in
 members, and buffering only the output, or buffering them separately, as
 you prefer.
 

I will try to do something sensible, but I do not want to meddle with
the original code as much as possible, too.

 I read a page about do_QueryInterface() but it does not
 explain the principle very much.
 
 It's simple:  You pass the object a 128-bit IID, it returns a pointer to
 something that implements that interface if that's an IID it implements,
 and null otherwise.
 
X = do_QeuryInterface(Y) is possible only when X is the direct or
 indirect  descendant of Y?
 
 No, that has nothing to do with it.  It's all about what the object
 claims to implement.

Now I see. I thought of do_QueryInterface() has something to do with
class hierarchy, but it has nothing to do with.
It is about the object's claim that it implements certain interface.

Thank you again.

BTW, the seeming lack of buffering was that I failed to
remove the offencing |Seek()| that negates buffering.
Once I disabled it in my local builds, everything seems AOK!



 -Boris

Chiaki Ishikawa

 

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Excessive inbound bustage

[ISHIKAWA,chiaki]

On 2015/04/22 22:15, Ehsan Akhgari wrote:


Note that it's not just the turn around time that can become an issue.
For me personally the reasons I choose to not use the try server are:

1) If I have ~20 unlanded patches in my queue (which is normal for my
workflow on an average day), I need to juggle 20 open tabs for 20 try
pushes if I wanted to send each one of them to the try server, which
means that I would need to spend the time to look at each result, go
through the intermittents on them, retrigger the runs for the
intermittents that have not been filed yet (I get them surprisingly
often), etc.



I gave up on using tryserver when my hg patch queues
became around 20 ~ 30 for a similar reason.

If code is very mature and I only realize a few bugs a year, it is OK.

But code like C-C TB has so many hidden issues, accumulated over the 
years, when one starts to investigate a real-world issue, one can easily 
end up with 20 patch queue entries over time :-(

To me, it is a mystery how one can cope with the situation with
hg queue feature. If a single bug is tackled fine. But when multiple 
bugs are noticed and handled at the same time, it needs improvement (I 
found git is much more amenable in this regard by making different 
branch creation very easy, but I digress.)


Back to my main topic:

I use my local PC to test the patch that it builds and passes tests.
Only when I am not quite confident that my patch tested locally on linux
would not break Windows binaries, I ask someone to push it to try.
(I am mainly submitting patches for C-C TB, my main mail client.)

Oh I remember now:
Another practical reason I have not used try for C-C TB for sometime
is this.
In C-C TB, we have a subtree of a clone of M-C under ./mozilla 
subdirectory.

Submitting patches to the main C-C portion is easy.

*IF*, however, one needs to patch code under M-C subtree also,
we have an issue.
We have to go through a contortion of creating a tentative copy of patch 
IN the top-level directory of C-C which tryserver cleverly understands 
as a patch for M-C portion,
and changes it into the patch for M-C portion of the tree and apply it 
before running the build and test jobs.
I am sure many TB developers have tried to automate this process of 
creating the tentative copy of M-C patch at the top-level directory
by creating a script on their own. It is a tricky procedure prone to 
errors if done manually.


Now, such a script needs to look at
the patch queue as a whole so that one does not misses a patch to be 
submitted.
However, this means that, if like many developers, one has a lot of 
unrelated patches, there is a problem.

I work on multiple bugs and mutiple set of patches at the same time,
When I need to submit a job to tryserver in order to test the particular 
patch set to fix a bug, I need to clear from the patch queue of other 
unrelated patches so that my hand-crafted script for patch job 
submission to tryserver

does not pick them up.

This clearing the patch queues before try server submission forced me to 
give up the use of try server. I can't have so many

different source trees just for try server job submission.
(git makes it easy to create branches easily.)

On my local test, I can barely manage the testing of intended patches 
only by re-ordering the patches in the queue and only applying the first 
N patches and test. Only when this works, I submit the patches to bugzilla.
(git makes it easy to create many branches easily and I can test a 
particular bug fixes there. Maybe using a queue of patch queues(?) is a 
way to go with HG/mercurium locally, but
I have no idea how such approach can work with try server job 
submission. Remember the M-C patch handling? If I have multiple queues,

I don't think a simple script can work it out.
BTW, This complexity is one reason many developer wish for C-C merge 
with M-C. If C-C subtree can sit below the main M-C tree as a mere 
sub-directory, NOT an independent repository,

I think some practical issues will be gone.)

So I am trying to test my paches with tryserver or local test although 
there is practical difficulty of using tryserver under the circumstances.


Well, less workload on tryserver is a good thing :-)

However, looking at the discussion, although I thought that everyone 
checks the patches locally if try is not used,

this may not have been the case always.
That at least ought to be clearly documented (!?).

TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running C-C TB under valgrind during mozmill test: extending timeout values (but where?)

[ISHIKAWA, Chiaki]

(Just this once, I post to m.d.platform and m.d.a.firefox)

I found out that firefox's mochitest
can be run under valgrind
according to How Do I Run A Mochitest Under Valgrind?
in https://developer.mozilla.org/en-US/docs/Mozilla/Testing/Valgrind

e.g.:
./mach mochitest-plain --debugger=valgrind
--debugger-args=--smc-check=all-non-file
--vex-iropt-register-updates=allregs-at-mem-access relative/path/to/tests

[not tested yet.]

Maybe I should try to see if some timeout values are extended during the
execution of the above command (and if so, how.)

However, I have a nagging feeling that mozmill test of TB seems to run
a TB as a DOM object that can be manipulated from an external driver
program via XPCOM to simulate user-interaction, and the framework may be
very different from mochitest framework.
Even if the framework may be similar, but the initial setup of running
TB as a daemon that can be manipulated by external program (setting up
proxy called a bridge in test framework) seems to take a very long time
in the case of TB. That is the problem.

Oh, I hasten to say that running C-C TB under valgrind as a standalone
program to be manipulated by a user using mouse and keyboard runs fine
under suitable linux kernel revision. (I have an issue with 3.16.0 under
Debian GNU/Linux.)

Testing manually is not exhaustive and thus I want to run TB under
valgrind through mazmill test and xpshell-tests,
and that is where the timeout issues raise ugly head!

Short of good idea, I will create a monstrous bundle of editor scripts
to change many timeout values found in .py files under MOZOBJ directory
used for testing to a much larger fixed value. I wonder if it will work.
[But this is not desirable in principle.
I want to extend only the relevant timeout values to handle the
slowdown by the execution under valgrind. Network delay observed between
a remote deamon should not be handled with extended timeout, for
example. I may have to wait unduly long for such cases if all the
timeout values are extended.]

Anyway, I will appreciate feedback, tips, comments, etc.

Again, followup-to m.d.a.thunderbird.

TIA



On 2015/04/16 13:41, ishikawa wrote:
 I wonder if someone has any idea about the following issues:
 
 I used to be able to run C-C TB under valgrind by creating a wrapper binary
 during mozmill test.
 This wrapper is named as .../dist/bin/thunderbird and invokes
 thunderbird-bin under valgrind.
 
 By extending, timeout values in a few places in mozmill script files, I
 could run this valgrind + TB combination successfully
 during mozmill test and this helped me uncover about a dozen uninitialized
 memory references
 and one out of bound memory reference which was caused by badly crafted
 Date: line, etc.
 
 But after the major build infrastructure change, which eventually led to the
 use of mozharness, etc.,
 the timeout value changes that I performed no longer are enough to prevent
 the test framework to timeout prematurely.
 I could not get valgrind+TB run before the timeout kicks in.
 
 I wonder if anyone has an idea of how to extend the timeout value(s)
 successfully.
 
 I checked the source files recently, but there are so many timeout values
 scattered around in many files,
 I don't know exactly which ones I should change.
 (The first timeout I hit is related to connect/setup bridge to running TB, I
 think.)
 
 Another thing is that the comments suggest there is a mixture of timeout
 values that are
 in milliseonds and seconds used in the code.
  From the experience I had until last spring/summer, I am quite sure that the
 timeout values
 used for socket timeout, etc. are in milliseconds.
 However, browsing through a few .py files at the top-level, I noticed that
 they are commented as seconds and not milliseconds, and so got confused.
 Maybe the timeout values needs to be multipled by 1000 to make sure
 that valgrind can run. I have no idea.
 
 Any hint / tips regarding how to extend the timeout values globally
 will be appreciated.
 
 (One other thing, these values are embedded inside .py files that are
 expanded/created during
 the build process from archive under the source directory, and so I have to
 modify the .py files JUST BEFORE running tests.
 If there is a way to change the values in the archive so that they will
 persist, that will be great.)
 
 I need to make sure TB won't crash due to these memory issues during normal
 usage and that is why
 I want to test it throughly.
 
 TIA
 
 
 

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Using address sanitizer with GCC 4.9 to compile C-C TB

[ISHIKAWA, Chiaki]

It just so happens that I have been using GCC on my linux PC, and
I have never run clang there.

Maybe I should.

In any case, GCC 4.9 + ASAN worked and helped me to fix a memory issue 
(referencing a heap after it is freed.)


Another thing is that on a small device based on ARM, etc.,
there *may* be only GCC cross compiler and not CLANG yet, and
in that case, someone who may be porting mozilla software with their 
addon, etc. to such

environment may find my experience useful.

TIA


On 2015/03/25 21:36, Ehsan Akhgari wrote:

Why wouldn't you use clang instead?  ASAN builds using clang on Linux
are a supported and tested Firefox configuration so there is a good
chance that it works well for Thunderbird as well.  ASAN builds using
gcc is neither tested nor supported as far as I know.

On 2015-03-24 10:15 PM, ISHIKAWA,chiaki wrote:

Hi,

I filed the following bugzilla to share my experience of
using address sanitizer with GCC 4.9 (*NOT CLANG*)
to compile C-C TB.

Bug 1147248 - GCC 4.9 + Address Sanitizer + C-C TB: Compiling
comm-central thunderbird using address sanitizer, -fsanitize=address
using GCC 4.9

I wonder what I did is optimal or not.
Comments welcome, especially the patch I created for mozalloc.h

I believe similar setup will make it compile FF with GCC + ASAN.

I needed to use ASAN since valgrind/memcheck did not work due to recent
changes in OS, libraries, or the mozilla software itself.

TIA

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform





___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Using address sanitizer with GCC 4.9 to compile C-C TB

[ISHIKAWA,chiaki]

Hi,

I filed the following bugzilla to share my experience of
using address sanitizer with GCC 4.9 (*NOT CLANG*)
to compile C-C TB.

Bug 1147248 - GCC 4.9 + Address Sanitizer + C-C TB: Compiling
comm-central thunderbird using address sanitizer, -fsanitize=address
using GCC 4.9

I wonder what I did is optimal or not.
Comments welcome, especially the patch I created for mozalloc.h

I believe similar setup will make it compile FF with GCC + ASAN.

I needed to use ASAN since valgrind/memcheck did not work due to recent
changes in OS, libraries, or the mozilla software itself.

TIA

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: PSA: Non-unified builds no longer occurring on central/inbound and friends

[ISHIKAWA,chiaki]

On 2015/02/03 15:24, Mike Hommey wrote:
 On Tue, Feb 03, 2015 at 02:27:52PM +0900, ISHIKAWA,chiaki wrote:
 I did a non-unified build and saw the expected failure.
 This is a summary of what I saw.

 Background:

 I may need to modify and debug basic I/O routines on local PC, and so
 want to avoid unnecessary compilation. I use ccache locally to make
 sure I can avoid re-compilation of touched but not modified C++ source
 files (files get touched and remain unmodified when I execute
 hg qpop and hg qpush in successions to work on different patches.
 Without ccache, I have to compile many files. ccache helps a lot.)

 There is a different perspective on unified compilation.

 Compiler farm users:
 One time fast compilation　is very important.
 So unified compilation is a win.
 (I suspect precompiled headers, -pch, would be a good win, too.)

 Developers who repeats edit a small set of files, compile and link
 many times on local PC:

 He/she may modify only a few files and want quick
 turn around of the compile of a few files and link time.

 Unified compilation actually compiles more lines than he/she wants
 (because of the extra source lines included in unified source files
 in which his/her modified files are also included.
 (Correct? Am I missing something here?)
 So he/she may not like unified compilation in such scenario.
 
 Here's my take on this: yes, we should optimize for build times when
 code is modified.
 
 But here's the thing: in most directories, unified compilation shouldn't
 be making a huge difference. That is, compiling one unified source vs.
 compiling one source shouldn't make a big difference. If it does (and it
 does in some directories like js/src), then the number of unified
 sources in the directory where it's a problem should be adjusted.
 
 Mike

Mike, thank you for the comment.
I suspect this is indeed the case in many directories.
(I mean unless a change of a single file caused 20 or 30 files to be
included into a unified source, then it is an overhead certainly. But so
far, the upper-bound of single change of a file is less than a couple of
minutes including the link with -gsplit-dwarf.)

I will report if I find a file, when touched, causes  an extraordinarily
long compilation time (by including many of the source files during
unified compilation).

By the way, I saw Unified_binding_*.cpp files during compilation, and I
suspect they are different types of unified compilation since this
unified_binding compilation seems to occur no matter what the setting
of FILES_PER_UNIFIED_FILE.

TIA

CI


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Memory management in C programs

[ISHIKAWA, Chiaki]

On 2015/01/12 22:46, Philip Chee wrote:


One large difference between C and most other programming languages is
that in C, you have to handle memory yourself rather than having a
garbage collector do it for you. Ensuring that memory is allocated at
the correct moment is not very difficult (and something that needs to be
done manually in pretty much every language); the hard part is to ensure
that enough memory is allocated, and to ensure that the memory is
deallocated when it is no longer in use.

There are several techniques available for memory management in C. Many
of them are used in NetHack 3.4.3; and even more are used somewhere in
NetHack 4. In this blog post, I'd like to look at them and discuss their
advantages and disadvantages. I'm mostly concerned about correctness,
rather than efficiency, here; that means that unless the performance
difference is very large, I care more about clean code than I do about
fast code.


http://nethack4.org/blog/memory.html

Phil



Nethack? That nethack?
I looked at the web page.
Yes, *that* nethack.

Do people still play it? It seems so
and on a UTF-8 capable terminal (?)
We have come a long way from vt100.

Anyway, a surprise that there are so memory allocation issues under the 
hood even for nethack! It now even has an interface for JSON library (?)


Written on Sun Mar 16 19:10:57 UTC 2014 it is a good summary, and 
something good old DDJ would carry.


But after all, the author of the page states in a section
titled 'Avoiding allocations in the first place':
As can be seen above, memory allocation in C is something of a mess ...

So we are better off with more modern languages although at
low-level we DO need to deal with issues mentioned in the blog.

Hmm, the author seems to be rewriting nethack using some modern memory 
handling approach

using contorted C. Long live nethack.

I am not sure if he or anybody will rewrite nethack
in more modern languages. Nethack in haskel may be an interesting Google 
summer intern programming topic (?)


Moral of the story:
This blog has a bearing on mozilla software. too.

I am struggling to clean up a mess of stream handling in TB's message 
download code: it is not entirely clear who owns the stream in the sense 
that who should close an open stream. Because of the confusion, TB 
currently closes already closed stream left and right.
Also, it must have broken subtle handling regarding stream and buffering 
cannot be enabled without causing a strange message corruption.


So topic of who frees the allocated memory hits home (hard to done 
correctly in C.)


At least we use C++ for mozilla software, so we are better off. I notice 
there have been tricks to make sure
some allocated objects not released accidentally, etc. to make memory 
management more robust automatically.


Unfortunately, stream handling in the current TB code
suffers from the similar issue suffered from C programs. Maybe we can 
tweak the stream class more robust, but I think in the current case, TB 
ought to be re-coded somewhat to show us its handling of stream in an 
explicit accountable manner.)



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: PSA: Non-unified builds no longer occurring on central/inbound and friends

[ISHIKAWA,chiaki]

On 2015/01/15 10:37, Steve Fink wrote:

On 01/14/2015 11:26 AM, Ehsan Akhgari wrote:

 From now on, the only supported build mode is unified compilation.  I
am planning to follow-up with removing support for the
--disable-unified-compilation configure option altogether in bug 1121000.


I commented in the bug, but I guess this is probably a better forum.

Why is the configure option being removed? I understand always building
unified in automation, but not having a straightforward way at all to
see if your code is buggy seems... suboptimal. If someone wants to go
through occasionally and make our codebase valid C++, how should they do
it after this change?

Perhaps the previous dev-platform thread would provide enlightenment and
I ought to go back and reread it :( , but I do not offhand recall it
deciding to remove the configure option altogether.



I have an issue here, too.

Debugging using gdb will be very difficult when the unified build 
creates a source file on the fly (but it is removed, correct?).


No sane compiler/debugger combination can help me do
the source level debugging if the source code that the compiler compiled 
is gone by the time debugger tries to find it...


Maybe I am missing something...

TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

C-C TB performance issue: unbuffered write where it can use buffering.

[ISHIKAWA, Chiaki]

(I posted this to
https://bugzilla.mozilla.org/show_bug.cgi?id=558528#c46 as a permanent
record, but I am posting this to a couple of newsgroups with a few typo
fixes and added comments to
solicit a solution re how to fix this performance issue from a wider
audience.)

I have spotted a very conspicuous offender of not using buffered
output where it can in comm-central thunderbird.

My questions to the list is as follows.

I am not familiar with I/O code within mozilla code base.
Is there a way to set the buffer-mode of |m_fileStream-Write| inside
CopyDate [see below] (just in case there are other wayward callers)?
Or can we simply replace this |Write| with a buffered output version,
but what that would be?

I think as long as a buffer of reasonable size, even if it is 4KB or
16KB, is used, I/O performance will  be much better and reasonably good.
(especially for a remote file system, or imap case.)
It does not have to be 128KB or much larger something, I think,
although it would help for a local disk write and read.

However, here I report that I have found a routine is copying message
data WITHOUT ANY BUFFERING at all when it WRITES the output LINE BY
LINE.

A REAL PROBLEM:

Imagine if the message has a few MB of binary attachment as mime data
(PDF, JPEG photo, etc.).  Typically, the attachment in the message is
encoded as mime data, each line smaller than 80 octets.  TB seems to
send mime as 74 octets line including the ending.  So, say, for a 2MB
bytes of attachment,
assuming that base64 increases the size of characters to be sent by 1.33,
approximately 1.33*2MB / 72 (subtracting two from 74 for CRLF) = 38739
write system calls are issued by
TB on the receiving side.

38739 |write| system calls.

NO WONDER TB feels slow.

The copying feels slow when my mail is stored on local disk.
Imagine someone who has to use a remote file system in a corporate
setting for mail store. He/she would be really penalized with so
excessive # of write system calls with small chunk of data over the
wire. These write calls ought to be merged by proper buffering!

Even a small 4K buffer would cut down the # of system calls about 55
times, not too bad.

Enough preamble for giving the background information.

The culprit routine I found is

NS_IMETHODIMP nsMsgLocalMailFolder::CopyData(nsIInputStream *aIStream,
int32_t aLength)
in the file
comm-central/mailnews/local/src/nsLocalMailFolder.cpp

And the problem unbuffered write in question is

http://mxr.mozilla.org/comm-central/source/mailnews/local/src/nsLocalMailFolder.cpp#2111

2111 rv = mCopyState-m_fileStream-Write(start, lineLength,
bytesWritten);

This does not seem to use buffering at all.
(Please check the excerpt of system call trace
from strace under linux
attached in the following.).

I agree that, since this function seems to be in need of scanning the
data for From at the beginning, and escaping From at the beginning
of line, etc., it needs to SCAN the INPUT line by line, but there is
NO REASON AT ALL TO USE UNBUFFERED *OUTPUT*!

Repeat-by:

Monitor and record the system calls issued by TB.
I used strace under linux.
strace -o /tmp/t.out TB-BINARY

Start TB, and then copy a message with reasonably large binary
attachment in one folder to another folder.

I found the repetition of small write(fd, bug, len) with length
parameter 74 MANY times.

E.g.: Excerpt from a typical strace record.

I only excerpted the interesting lines.  There are many other system
calls.

The particular message I copied had an attachment of 1.2MB size.

The repeated writes() have the first parameter (file descriptor) as
54.  I searched backward in trace file and found that FD 54 is for the
target folder. (folder-C)

   ...

open(/home/ishikawa/.thunderbird/u1ku1i3y.default/Mail/Local
Folders/folder-C, O_RDWR|O_CREAT, 0664) = 54
   ...

Extraction of the initial part of the writes to this file descriptor:
(I think I created the message using HTML format mail. That is why you
see a few lines of HTML main message body. But that is not of any
concern. Concern is the long repetition of mime data that follows.)
...

write(54, From - Sun Nov 22 11:35:01 2009\r..., 33) = 33
clock_gettime(CLOCK_MONOTONIC, {2572825, 299989570}) = 0
clock_gettime(CLOCK_MONOTONIC, {2572825, 300253008}) = 0
write(54, X-Mozilla-Status: 0001\r\n, 24) = 24
clock_gettime(CLOCK_MONOTONIC, {2572825, 299989570}) = 0
clock_gettime(CLOCK_MONOTONIC, {2572825, 300253008}) = 0
write(54, X-Mozilla-Status2: 1000\r\n, 29) = 29
   I omit clock_gettime() below ...
write(54, X-Mozilla-Keys: $label3 ..., 97) = 97
write(54, FCC: mailbox://nobody@Local%20Fo..., 44) = 44
write(54, X-Identity-Key: id1\r\n, 21) = 21
write(54, Message-ID: 4B08A02E.7080201@ex..., 44) = 44
write(54, Date: Sun, 22 Nov 2009 11:35:01 ..., 39) = 39
write(54, From: ishikawa ishikawa@example..., 39) = 39
write(54, X-Mozilla-Draft-Info: internal/d..., 70) = 70
write(54, User-Agent: Thunderbird 2.0.0.23..., 49) = 49
write(54, MIME-Version: 1.0\r\n, 

Re: Is anyone still using JS strict warnings?

[ISHIKAWA,Chiaki]

(2014/12/20 5:19), Jason Orendorff wrote:

So if you go to about:config and set the javascript.options.strict pref,
you'll get warnings about accessing undefined properties.

 js Math.TAU
 undefined
 /!\ ReferenceError: reference to undefined property Math.TAU

(It says ReferenceError, but your code still runs normally; it really is
just a warning.)

Is anyone using this? Bug 1113380 points out that the rules about what kind
of code can cause a warning are a little weird (on purpose, I think). Maybe
it's time to retire this feature.

 https://bugzilla.mozilla.org/show_bug.cgi?id=1113380

Please speak up now, if you're still using it!


thunderbird relies on many JS source files, and
sometimes a bug creeps in, for example,

 - attributes which were set before, do not get set any more
   after a modification, and

 - some other files are still referencing the now non-existing
   attributes.

JS strict warnings are the only way for me to realize such bugs exists.
Most of the codes are written many years ago, and so frankly nobody owns 
them, so to speak these days.


(I also notice that there are typos that can only be uncovered by
 JS strict warnings, etc.)

From the software engneerng point of view, it is essential to keep the 
large amount of JS source files of TB in shape IMHO.


I use the JS strict warnings that are printed from the DEBUG build of 
C-C TB to find and fix remaining and newly introduced bugs in TB all the 
time. (The number of such warnings and errors printed during |make 
mozmill| tests are staggering, and I created a script to sort them and 
prioritize them. I see probably a few dozen different bugs and attack 
the most frequent or most disturbing-looking bugs/warnings first.)


TIA



-j
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Unified compilation is going to ride the train

[ISHIKAWA,chiaki]

On 2014/11/28 18:26, Mike Hommey wrote:

On Fri, Nov 28, 2014 at 11:57:56AM +0900, ISHIKAWA,chiaki wrote:

On 2014/11/28 10:12, Mike Hommey wrote:

Hi,

A year ago, when unified compilation was introduced to speed up builds,
a couple issues were raised and we conservatively restricted them out
of aurora/beta/release/esr.

A year later, it's time to revisit this decision, and since afaik we
haven't had problems specific to unified compilation on nightlies,
including for crash reports, we can assume the issues are either gone
or didn't exist in the first place (one problem that comes to mind is
bug 943695, and it probably isn't a problem in practice, although weird)

I know a lot of people have burned non-unified builds now and then.
That's an annoyance and a distraction for getting things done. If
unified compilation rides up to beta and we don't see significant
problems, I think we can disable all periodic non-unified builds
and make the few builds that are always non-unified unified again (a few
debug builds are this way).

The downside from doing so, though, is that non-unified build *will*
be broken, and code purity (right includes in the right sources,
mostly) won't be ensured. Do you think this is important enough to keep
non-unified builds around?

Mike



Can we debug unified build binary using gdb under Unixens?

I mean the unified source is gone by the time the binary runs,
and I wonder what the gdb would do to print the source lines.

Of course, if gdb gets confused, we can create a non-unified binary to use
source code debugging, etc. [I am talking about C++ side of the debugging.]

But, if gdb gets confused,
then  not supporting non-unified build for C++ debugging
sounds tough. People's opinion may vary, but I have seen many issues
with C++ binary  and so am not comfortable dropping non-unified build,
if gdb and other debugger aids get confused.

Of course, producing end-user binary wins by using unified compilation. I
agree.


Local builds have defaulted to unified builds for a year. If that was
causing problems with debuggers, we'd have heard about it a thousand
times already.

Mike



Aha, I noticed I have a following line
in my MOZCONFIG file:

ac_add_options --disable-unified-compilation

Hmm... I think I have put it as soon as some trials
with unified compilation occurred.
I will wait and see.

TIA



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Unified compilation is going to ride the train

[ISHIKAWA,chiaki]

On 2014/11/28 10:12, Mike Hommey wrote:

Hi,

A year ago, when unified compilation was introduced to speed up builds,
a couple issues were raised and we conservatively restricted them out
of aurora/beta/release/esr.

A year later, it's time to revisit this decision, and since afaik we
haven't had problems specific to unified compilation on nightlies,
including for crash reports, we can assume the issues are either gone
or didn't exist in the first place (one problem that comes to mind is
bug 943695, and it probably isn't a problem in practice, although weird)

I know a lot of people have burned non-unified builds now and then.
That's an annoyance and a distraction for getting things done. If
unified compilation rides up to beta and we don't see significant
problems, I think we can disable all periodic non-unified builds
and make the few builds that are always non-unified unified again (a few
debug builds are this way).

The downside from doing so, though, is that non-unified build *will*
be broken, and code purity (right includes in the right sources,
mostly) won't be ensured. Do you think this is important enough to keep
non-unified builds around?

Mike



Can we debug unified build binary using gdb under Unixens?

I mean the unified source is gone by the time the binary runs,
and I wonder what the gdb would do to print the source lines.

Of course, if gdb gets confused, we can create a non-unified binary to 
use source code debugging, etc. [I am talking about C++ side of the 
debugging.]


But, if gdb gets confused,
then  not supporting non-unified build for C++ debugging
sounds tough. People's opinion may vary, but I have seen many issues
with C++ binary  and so am not comfortable dropping non-unified build,
if gdb and other debugger aids get confused.

Of course, producing end-user binary wins by using unified compilation. 
I agree.


TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

profiler in TB

[ISHIKAWA,chiaki]

Hi,

I thought I try profiler to see how it works in TB, but
I get the following error.
It looks a module called |panel| is not usable in
TB.
I looked at jetpack-panel-apps web page noted in the message, but
am clueless.

Error Message:
error: geckoprofiler: An exception occurred.
Error: The panel module currently supports only Firefox.  In the future 
we would like it to support other applications, however.  Please see 
https://bugzilla.mozilla.org/show_bug.cgi?id=jetpack-panel-apps for more 
information.
resource://jid0-edalmuivkozlouyij0lpdx548bc-at-jetpack/addon-sdk/lib/sdk/panel.js 
12

Traceback (most recent call last):
  File resource://gre/modules/NetUtil.jsm:123, line 17, in 
NetUtil_asyncOpen/.onStopRequest
  File 
resource://jid0-edalmuivkozlouyij0lpdx548bc-at-jetpack/addon-sdk/lib/sdk/net/url.js:49, 
line 9, in readAsync/

 [...]

Any profiler that is usable for TB?
(Or are we to revert to low-level tools such as GCC's -pg and use
gprof or something like that? I have no idea if it will work
with a complex program like TB or FF with so many shared libraries and 
source files spread over so many directories, etc.)


TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: profiler in TB

[ISHIKAWA,chiaki]

(sorry, I must have responded to an existing thread instead of creating 
a new entry.


Actually, I was testing TB: it seems to have
a bug in that creation of new message is prohibited after I try to 
respond to a message with e-mail address as well as newsgroup in its 
destination. This prohibition itself is fine. But please read on.
TB then complains that from this account I can only send an e-mail 
message when I try to issue the message. So I took out
a newsgroup destination and happily sent out e-mail message, that is 
what I thought. But then create a new mail menu entry is greyed out, 
and I could not create a new message any more.

I had to restart TB. I was checking the issue,
and left an empty composite window lying around, and
I must have used THAT window for this message. Sorry about broken 
threading info.)


On 2014/10/26 11:48, ISHIKAWA,chiaki wrote:

Hi,

I thought I try profiler to see how it works in TB, but
I get the following error.
It looks a module called |panel| is not usable in
TB.
I looked at jetpack-panel-apps web page noted in the message, but
am clueless.

Error Message:
error: geckoprofiler: An exception occurred.
Error: The panel module currently supports only Firefox.  In the future
we would like it to support other applications, however.  Please see
https://bugzilla.mozilla.org/show_bug.cgi?id=jetpack-panel-apps for more
information.
resource://jid0-edalmuivkozlouyij0lpdx548bc-at-jetpack/addon-sdk/lib/sdk/panel.js
12
Traceback (most recent call last):
   File resource://gre/modules/NetUtil.jsm:123, line 17, in
NetUtil_asyncOpen/.onStopRequest
   File
resource://jid0-edalmuivkozlouyij0lpdx548bc-at-jetpack/addon-sdk/lib/sdk/net/url.js:49,
line 9, in readAsync/
  [...]

Any profiler that is usable for TB?
(Or are we to revert to low-level tools such as GCC's -pg and use
gprof or something like that? I have no idea if it will work
with a complex program like TB or FF with so many shared libraries and
source files spread over so many directories, etc.)

TIA


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: MSE WebM/VP9 enabled on nightly

[ISHIKAWA,chiaki]

On 2014/10/24 13:46, Anthony Jones wrote:

I just wanted to give a heads up to everyone that we enabled Media
Source Extensions on nightly for WebM/VP9. This brings Adaptive
Streaming capability to Firefox video playback. The feature is not
complete so the pref will automatically turn off when it gets to
beta/release if we do nothing.

You can check on YouTube by right clicking the playing video and
selecting Stats for nerds which should appear above the About the
HTML5 player option. If you see DASH: yes then you are now living in
the future.

This affects YouTube but may also affect sites that use MSE with
WebM/VP9 but it could also affect sites that use MSE but fail to check
codec compatibility.

Please file any (unfiled) issues you experience as blocking bug 1083588.
Don't expect it to be perfect and if you run into trouble you can set
media.mediasource.enabled to false in your about:config

Anthony



Hi,

Just reporting what I observed after a source refresh half a day ago.

I noticed during C-C TB compilation
a file under common needs to be copied to encoder, another one to 
decoder directory .

I am talking about files below these directories.
mozilla/media/libvpx/vp9/{common,encoder,decoder}

But since C-C was in such a disarray in terms of compilation lately, 
etc., and the source was refreshed just before this compilation effort,

I am not sure if the configuration was quite correct.
I failed to write down a memo exactly which files were
copied. I thought I was logging it using script, but did not.
I clobbered and then tried to see how it would work out, and then was 
side-tracked by bug 1088497


I can compare the directory to report what files were copied
if no such bugs have been filed yet and you are not aware of the issue.
(I tried to see which one by timestamp, but python client.py checkout
seems to give same timestamps to all the files and so I am not sure 
which ones were copied. cp or Emacs's filecopy seems to preserve the 
timestamp. That is good sometimes, but annoying sometimes.)


But then again, I am not entirely sure if it was a temporal hiccup after
the source refresh.

Just an observation.

TIA



TIA

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Has something changed about nsITreeView (nsItreeView.setTree) lately?

[ISHIKAWA,chiaki]

Hi,

I posted the following message to mozilla.dev.builds newsgroup, but
I wonder if there is a mozilla-wide change to nsITreeView lately?
I noticed this while testing a patch to C-C TB locally.

(Considering that COMM-CENTRAL may have a slight delay in getting
M-C changes, it may be that nsITreeView change may have happened a few
weeks earlier or something).

[Exception... Failure arg 0 [nsITreeView.setTree]  nsresult:
0x80004005 (NS_ERROR_FAILURE)

The error is fairly new.
I did not see it on local PC until I updated the local source to build
local DEBUG BUILD of TB.
I thought initially it was my local patch that was causing the error,
but upon investigation, it was hardly likely, and
when I checked TBPL I realized it was causing trouble for everybody's
compilation job.

Any pointers?

[I searched for nsITreeView.setTree in bugzilla but nothing came up. Is
the simple search too simplistic?]

TIA

--- begin quote ---

I think about 1/4 of 1000+ tests in |make mozmill| are failing now
See
https://tbpl.mozilla.org/?tree=Thunderbird-Trunk

This is pretty bad, and
looks to me that something about nsITreeView.setTree are the cause of
major malfunctions.

[Exception... Failure arg 0 [nsITreeView.setTree]  nsresult:
0x80004005 (NS_ERROR_FAILURE)

It is called in many places and resulted in this failure.

Any idea which has caused this  new regression?

Simply stated, no would-be contributor can work on any patches in this
state of affairs.

TIA

--- end quote
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Incorrect line number reported by JS engine for warnings?

[ISHIKAWA, Chiaki]

Lately, the stricter checks from JS engines produced many warnings
during the test suite run (|make mozmill|) of locally built DEBUG BUILD
of C-C thunderbird.

These in themselves are very good to make code better, more robust, etc.

However, today I noticed a few problems of line numbers thus reported in
the warnings.

Has anyone noticed similar issues?

Cases in point:

Please see Bugzilla:

https://bugzilla.mozilla.org/show_bug.cgi?id=1081767

Bug 1081767 JavaScript error:
chrome://messenger/content/nsContextMenu.js, line 693: TypeError: item
is null

In this bugzilla, line 693 is actually out of range for the file reported.
The file is much shorter.
Worse, in another part of the log, I see line 694 reported for this
same error. It looks as though there is an uninitialized variable or
something (that gets referenced when it has  different values when
these warnings are printed.

https://bugzilla.mozilla.org/show_bug.cgi?id=1081765
Bug 1081765 JavaScript strict warning:
chrome://messenger/content/nsContextMenu.js, line 103: ReferenceError:
reference to undefined property this.onEditableArea

When I look at line 103, the line again is an empty line, and I look at
the previous line, but there does not seem to any mention of
this.onEditableArea on that line.

I suspect that the line numbers are incorrect in these cases.

Anyone notice the similar line # issues?

The C-C source tree was refreshed yesterday.
(I know that copy of M-C subtree in C-C is synched, but still lag the
true M-C tree a little bit, and so if this issue is fixed in
true M-C tree already, please disregard this message.)


TIA
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Incorrect line number reported by JS engine for warnings?

[ISHIKAWA, Chiaki]

Sorry for top posting.

Mea Culpa:

I was looking at the wrong file as Boris correctly pointed out.

These two bugs were seen
in the timeout test case mentioned first in
https://bugzilla.mozilla.org/show_bug.cgi?id=1081763
Bug 1081763 - C-C TB: gContextMenu is undefined during test that
eventually times out.

In that bugzilla, it was mailContextMenus.js (as opposed to
nsContextMenus.js).

Sorry about the confusion.

One more bugzilla to file from the same test that times out.:
I have begun to see the following error since I refreshed
the source at the end of September:
JavaScript strict warning: resource://gre/modules/FormHistory.jsm, line
639: ReferenceError: reference to undefined property change.guid

TIA

On 2014/10/13 11:45, Boris Zbarsky wrote:
 On 10/12/14, 9:54 PM, ISHIKAWA, Chiaki wrote:
 In this bugzilla, line 693 is actually out of range for the file reported.
 The file is much shorter.
 
 Is it?  The error is in nsContextMenu.js but your comments in the bug
 are about mailContextMenus.js, which is not at all the same thing.
 
 Worse, in another part of the log, I see line 694 reported for this
 same error.
 
 In the same file?
 
 What I recommend doing is opening the exact chrome:// URI involved in
 view-source and going to the relevant line.  That will tell you what the
 file actually looks like in your install.  This matters, because during
 the build these files are sometimes preprocessed, have comments
 stripped, etc, which will make the line numbers the JS engine sees not
 match what's in the source tree (but match what's actually available if
 you click the error link).
 
 chrome://messenger/content/nsContextMenu.js, line 103: ReferenceError:
 reference to undefined property this.onEditableArea

 When I look at line 103, the line again is an empty line
 
 http://mxr.mozilla.org/comm-central/source/mail/base/content/nsContextMenu.js#103
 sure isn't.
 
 Are you looking at the right file?
 
 I suspect that the line numbers are incorrect in these cases.
 
 I doubt it.
 
 -Boris
 

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Removing ancient and unused code from tools/

[ISHIKAWA, Chiaki]

On 2014/10/13 13:36, Nicholas Nethercote wrote:

- tools/rb/ -- most, if not all, scripts actively used


Thank you for the clean up work.

I am using fix_linux_stack tool under tools/rb/ for cleaning up
non symbolic stack trace in the log of DEBUG BUILD of TB when I am 
testing it locally.


I wonder though, who are using these tools and maintaining them.

This summer or earlier, it was changed from  Ruby/Perl [I forgot which] 
script to python script, now fix_linux_stack.py, and also, the

argument (the logfile to be cleaned up) must be given to the script
now as STDIN stream as opposed to the positional argument $1 to the 
older tool.

I would like to be kept updated of these changes since when I tried to
invoke this script I had to
look at the sudden disappearance of the former script and then
needed figure out the change of the manner how a log file is specified 
to the script.

But thank you for the great tool anyway!

TIA




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Testing framework does not wait for correct dialog sometimes.

[ISHIKAWA,chiaki]

Hi,

I am not sure where to post, but since this affects every platform, here
it goes.

I noticed several months ago that some javascript test programs
that run during |make mozmill| test suite
wait for dialogs during the execution.

But I also realized that there is no distinction for dialogs.

That is, whatever dialog appears at the time when the test program
expects a particular dialog to appear in the test scenario
seems to let   the test programs assume that they are seeing THE dialog
that they expect at the point of  the test execution.

Well, this is not quite correct sometimes.
Program may show unexpected error dialog/popup due to coding error which
the test program does not expect.

While debugging, I noticed a few programs
saw unexpected dialogs, but thought they were seeing the
correct dialogs they expected and ran to the later step of the
test scenario.

This is not quite desirable, and one of the deficiencies of
the current testing framework.

I think the test framework should stop the test execution and print
failure then and there.


I think we need to enhance the wait for dialog to specify which dialog
to wait to avoid this unrelated dialog satisfies the test scenario
expecting a particular dialog to appear.

But then I realize that the current dialog mechanism may not have
such an identifier. (Maybe we can use the string or part of the string
that is shown in the dialog as a key to distinguish different dialogs?
Of course, change to existing dialog mechanism may be in order.
test script can be enhanced later. We can add a wait-for-dialog API with
an identifier argument and gradually change over to the new APIs by
replacing the wait for dialog where problems are noticed in test
scenarios. So the test scripts do not have to change overnight although
the core dialog implementation needs to add such ID and a means to check
the ID of a displayed dialog.)

I noticed such erratic behavior of unexpected dialog fooling the
test program into thinking it is seeing an expected dialog
during |make mozmill| since
one of my PCs somehow failed the access to (remote) mail account
information database during mail account creation, and
TB was showing the failure to access such database, but
the test program(s) thought they saw expected dialogs for dialog
creation and got stuck
later due to missing data, etc.

[If the code has changed in the last several months to support such
IDs for dialogs, please disregard this message.]

TIA

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Fwd: JavaScript (strict) Warning from FF mochitest-plain test.

[ISHIKAWA,chiaki]

I am for this proposal, too.

We need to spend a little man-power in the areas other than
implementing new features :-)

Fixing seemingly innocent warnings, etc. may not look a glorious task,
but we need to do this to ship quality software (not that
today's versions are not good quality, but we can make them better.)

And any automated process to get this started will be appreciated.

TIA
(Sorry, I am not that familiar how newsfeed and mailing list interacts 
for this newsgroup, so I am leaving the two personal addresses of Mike 
and Chris intact. Ehsan is a regular so the mailing list should reach 
him. Correct me if I am wrong.)


On 2014/10/03 18:01, Mike de Boer wrote:

FYI, forwarding Ehsan’s reply:

Begin forwarded message:


From: Ehsan Akhgari ehsan.akhg...@gmail.com
Subject: Re: JavaScript (strict) Warning from FF mochitest-plain test.
Date: 2 Oct 2014 23:09:29 GMT+2
To: Mike de Boer mdeb...@mozilla.com

Great!  I'm all for this if we decide to act on the data.  :-)

(BTW this email was just sent to me, not sure if it went to the list or not -- 
if you sent it through NNTP it probably did...)

On Thu, Oct 2, 2014 at 5:06 PM, Mike de Boer mdeb...@mozilla.com wrote:
On 02 Oct 2014, at 22:55, Ehsan Akhgari ehsan.akhg...@gmail.com wrote:


Are we going to spend the time to fix these, however?


I’d be up for that, certainly!


  For context, for as long as I remember, if you kept the browser console open 
and used the browser, we'd occasionally report an unhandled chrome JS errors 
(for example when attempting to access properties off of null variables). And 
those are not even strict errors.

I've occasionally filed bugs for them, but my impression was that unless I can 
construct a case for those warnings to map to a user visible bug, they would 
not be fixed.


Hence my - perhaps controversial - idea to auto-create and auto-assign bugs. So 
for new warnings that appear in the integration branch and m-c logs we could 
even make those bugs block the bugs that introduced them.

Mike.



--
Ehsan


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform




___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform