[webkit-dev] Is valgrind suppressions.txt unused

2021-04-30 Thread Aakash Jain via webkit-dev
Hi Everyone,

Is anyone (particularly gtk folks) are using valgrind especially 
suppressions.txt file 
(https://github.com/WebKit/WebKit/blob/main/Tools/Scripts/valgrind/suppressions.txt).
 This file doesn't seems to be modified for a long time, so I am not sure if 
this is even being used.

I am looking into using inclusive naming, and this file contains the term 
'slave' (as "fun:gtk_im_multicontext_get_slave"). I can look into that specific 
line as well, but I was wondering if the file is used at all or not. 
 
Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Commit-Queue failing intermittently

2021-04-24 Thread Aakash Jain via webkit-dev
There were intermittent issues with Commit Queue even after adding the retries 
and few other fixes. The issue was related to git-svn specifically with our 
GitHub repo and caused intermittent failures while trying to push the commits 
to WebKit svn repository. We have now moved Commit Queue bots back to 
git.webkit.org in https://bugs.webkit.org/show_bug.cgi?id=224762

Commit-Queue seems to be working reliably since then (without a single such 
issue in last 5 days).

Thanks
Aakash

> On Feb 23, 2021, at 7:06 PM, Aakash Jain  wrote:
> 
> Most of the issues with commit-queue were fixed. We also added automatic 
> retries in commit-queue to better handle intermittent issues (in 
> https://webkit.org/b/222038).
> 
> Please let me know if anyone notices any issue.
> 
> Thanks
> Aakash
> 
>> On Feb 4, 2021, at 4:39 PM, Aakash Jain  wrote:
>> 
>> We are noticing intermittent issues with Commit-Queue failing to commit, 
>> since we moved it to github. We are looking into it. Meanwhile, if 
>> Commit-Queue fails on your patch, you can set cq+ again to retry.
>> 
>> Thanks
>> Aakash
>> 
>>> On Jan 19, 2021, at 4:56 PM, Aakash Jain  wrote:
>>> 
>>> All the issues have been fixed.
>>> 
>>> Please let me know if anyone notices any issue.
>>> 
>>> Thanks
>>> Aakash
>>> 
>>>> On Jan 19, 2021, at 12:23 PM, Aakash Jain  wrote:
>>>> 
>>>> We are noticing some issues on Style-Queue and Commit-Queue. We are 
>>>> looking into that.
>>>> 
>>>> Thanks
>>>> Aakash
>>>> 
>>>>> On Jan 19, 2021, at 9:58 AM, Aakash Jain  wrote:
>>>>> 
>>>>> Hi Everyone,
>>>>> 
>>>>> We need to do some maintenance on the EWS bots (in order to switch the 
>>>>> repository from git.webkit.org to https://github.com/webkit/webkit, 
>>>>> https://bugs.webkit.org/show_bug.cgi?id=220479). EWS buildbot will be 
>>>>> offline for about an hour, so it wouldn't process any new builds. 
>>>>> However, EWS status-bubbles will still keep loading on Bugzilla.
>>>>> 
>>>>> Thanks
>>>>> Aakash
>>>> 
>> 
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] New fast commit queue mode: fast-cq

2021-04-08 Thread Aakash Jain via webkit-dev
Hi Everyone,

I am happy to inform you that I have added a fast-commit-queue mode to the 
Commit-Queue. In this mode Commit-Queue would land patches quickly (typically 
taking ~1 minute, e.g.: 
https://ews-build.webkit.org/#/builders/28/builds/10876). This mode skips 
building and testing. It basically does sanity check (like validating 
ChangeLog, Reviewer, Committer etc.) and commits the patch. This is intended to 
be used for scenarios where the patch is a build fix, urgent fix, quick 
typo/follow-up fix etc.

To use this mode, you can use the --fast-cq parameter while uploading the patch 
using webkit-patch command. e.g.: 'webkit-patch upload --fast-cq'. For patches 
which are already uploaded to Bugzilla, you can rename the patch (by clicking 
on 'Details' button for the patch), and prefix patch name with [fast-cq], e.g.: 
'[fast-cq] Patch for landing'.

Pro-tip: you can also rename the patch to include [fast-cq] prefix while 
marking the patch cq+, from patch 'Details' page, and this mode would work.

Please let me know if you have any questions/feedback.

Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] New EWS queue: Stress Test EWS

2021-04-06 Thread Aakash Jain via webkit-dev
Hi Everyone,

I am happy to announce that I have added a new EWS queue: Stress Test EWS. From 
now on, you would see a new status bubbles (on Bugzilla bug) named: 
'mac-wk2-stress'. For the patches that add or modify layout tests, this EWS 
will run those added/modified layout-tests repeatedly large number of times 
(currently 100 iterations). It will also run those added/modified layout-tests 
repeatedly in guard-malloc mode. Example: 
https://ews-build.webkit.org/#/builders/62/builds/249. The aim is to find any 
issue with the layout-tests, especially flakiness, or failure when run in a 
certain manner.

Flaky layout-tests are currently a huge problem, as they sometimes cause false 
positives on EWS (and other parts of infrastructure), and slow down our 
infrastructure (as for every test failure, EWS needs to run the test-suite few 
times to ensure the failures are not flaky or pre-existing). Until now, we 
didn't have any good automated way to indicate the test flakiness to the 
engineers. With this EWS, we are adding pre-commit testing for layout-test 
flakiness. We understand that layout tests might be flaky in a variety of ways, 
sometimes in a very specific or unexpected manner. I am looking at covering 
more and more scenarios in this EWS by making incremental improvements to it.

Few ideas to further improve this EWS are:
- Test on more OS/configurations (this EWS is currently using macOS release wk2 
mode)
- Run the test being added/modified repeatedly along-with other layout-tests in 
same directory
- Improve the logic to identify layout-tests, current logic is naive and only 
recognizes html layout-tests

More ideas are welcome.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] EWS and build.webkit.org upgraded to Buildbot v2.10.5

2021-04-05 Thread Aakash Jain via webkit-dev
Hi Everyone,

Buildbot v2.10.5 was released today (few hours back). It contains few fixes 
which I made to Buildbot, e.g.:

- 'Show owners and worker columns on workers page' 
https://github.com/buildbot/buildbot/pull/5942
- 'Show owners and worker columns on changes page' 
https://github.com/buildbot/buildbot/pull/5945


It also fixes few issues which I reported to Buildbot which were impacting us, 
e.g.: 

- 'Regression: Owners column empty for pending build-requests' 
https://github.com/buildbot/buildbot/issues/5940 
- 'Message queue doesn't provide correct message for step finished' 
https://github.com/buildbot/buildbot/issues/5906, https://webkit.org/b/222973 
<https://webkit.org/b/222973>

List of bug-fixes in v2.10.5: 
https://docs.buildbot.net/latest/relnotes/2.x.html#buildbot-2-10-5-2021-04-05


I have upgraded both, EWS (ews-build.webkit.org) and build.webkit.org to this 
Buildbot (v2.10.5). Please let me know if anyone notices any issue.

Thanks
Aakash

> On Mar 5, 2021, at 7:32 PM, Aakash Jain  wrote:
> 
> Hi Everyone
> 
> I have also upgraded EWS (ews-build.webkit.org) to latest Buildbot (v2.10.1).
> 
> Please let me know if anyone notices any issue.
> 
> Thanks
> Aakash
> 
>> On Feb 8, 2021, at 8:04 PM, Aakash Jain  wrote:
>> 
>> Hello Everyone,
>> 
>> We have upgraded build.webkit.org <http://build.webkit.org/> to latest 
>> Buildbot (v2.10.1 released in 2021) (https://webkit.org/b/175056 
>> <https://webkit.org/b/175056>). Earlier it was running an ancient version of 
>> Buildbot, v0.8.6p1 (released in 2012). This is a brand new server. Buildbot 
>> doesn't support migrating data from old version. Old instance is accessible 
>> through https://build-old.webkit.org <https://build-old.webkit.org/>
>> 
>> Known issue: Login functionality is not enabled for everyone yet.
>> 
>> Please let me know if you notice any issue.
>> 
>> Thanks
>> Aakash

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Reminder: WebKit SVN tree clousures (March 2021)

2021-03-22 Thread Aakash Jain via webkit-dev
Hello everyone,

The maintenance has been completed. WebKit SVN tree is open again. Commit-queue 
bots are also back online.

Thanks for your patience. Please let us know if you notice any issue.

Thanks
Aakash

> On Mar 17, 2021, at 12:50 PM, Ling Ho via webkit-dev 
>  wrote:
> 
> Hello,
> 
> The 2nd maintenance mentioned in my previous email will take place this 
> coming weekend as planned.
> 
> The WebKit SVN tree will be closed during:
> Begin: Friday, March 19th, 2021. 5pm (PDT)
> End: Monday, March 22nd, 2021. No later than 6pm (PDT)
> 
> ...
> Ling Ho
> 
> On 2/24/21 5:26 PM, Ling Ho wrote:
>> Hello WebKit,
>> 
>> Due to planned infrastructure maintenance, our critical continuous 
>> integration services (EWS and post-commit bots) will be unavailable during 
>> the following weekends:
>> 
>> Dates:
>> Begin: Friday, March 5th, 2021. 5pm (PST)
>> End: Monday, March 8th, 2021. No later than 6pm (PST)
>> 
>> Begin: Friday, March 19th, 2021. 5pm (PDT)
>> End: Monday, March 22nd, 2021. No later than 6pm (PDT)
>> 
>> To avoid accumulating difficult to diagnose regressions during this lengthy 
>> outage, WebKit SVN tree will be closed.
>> 
>> Please email ad...@webkit.org  if you have any 
>> questions or concerns.
>> 
>> Thanks,
>> 
>> ...
>> Ling Ho
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] github.com repository force push

2021-03-19 Thread Aakash Jain via webkit-dev
Hi everyone,

We had little bit of issue with syncing between WebKit's github and svn 
repositories. To fix the issue, we had to force push and modify few git commits.

In case you notice any conflict in your github.com  
checkout of WebKit repo (which might be because of having one of those commits 
which we modified), please do the following: "git reset --hard HEAD~100", "git 
pull".

Please let me know if you face any issues.

Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] EWS Upgraded to latest Buildbot

2021-03-05 Thread Aakash Jain via webkit-dev
Hi Everyone

I have also upgraded EWS (ews-build.webkit.org) to latest Buildbot (v2.10.1).

Please let me know if anyone notices any issue.

Thanks
Aakash

> On Feb 8, 2021, at 8:04 PM, Aakash Jain  wrote:
> Hello Everyone,
> 
> We have upgraded build.webkit.org to latest Buildbot (v2.10.1 released in 
> 2021) (https://webkit.org/b/175056). Earlier it was running an ancient 
> version of Buildbot, v0.8.6p1 (released in 2012). This is a brand new server. 
> Buildbot doesn't support migrating data from old version. Old instance is 
> accessible through https://build-old.webkit.org
> 
> Known issue: Login functionality is not enabled for everyone yet.
> 
> Please let me know if you notice any issue.
> 
> Thanks
> Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Commit-Queue failing intermittently

2021-02-23 Thread Aakash Jain via webkit-dev
Most of the issues with commit-queue were fixed. We also added automatic 
retries in commit-queue to better handle intermittent issues (in 
https://webkit.org/b/222038).

Please let me know if anyone notices any issue.

Thanks
Aakash

> On Feb 4, 2021, at 4:39 PM, Aakash Jain  wrote:
> 
> We are noticing intermittent issues with Commit-Queue failing to commit, 
> since we moved it to github. We are looking into it. Meanwhile, if 
> Commit-Queue fails on your patch, you can set cq+ again to retry.
> 
> Thanks
> Aakash
> 
>> On Jan 19, 2021, at 4:56 PM, Aakash Jain  wrote:
>> 
>> All the issues have been fixed.
>> 
>> Please let me know if anyone notices any issue.
>> 
>> Thanks
>> Aakash
>> 
>>> On Jan 19, 2021, at 12:23 PM, Aakash Jain  wrote:
>>> 
>>> We are noticing some issues on Style-Queue and Commit-Queue. We are 
>>> looking into that.
>>> 
>>> Thanks
>>> Aakash
>>> 
>>>> On Jan 19, 2021, at 9:58 AM, Aakash Jain  wrote:
>>>> 
>>>> Hi Everyone,
>>>> 
>>>> We need to do some maintenance on the EWS bots (in order to switch the 
>>>> repository from git.webkit.org to https://github.com/webkit/webkit, 
>>>> https://bugs.webkit.org/show_bug.cgi?id=220479). EWS buildbot will be 
>>>> offline for about an hour, so it wouldn't process any new builds. However, 
>>>> EWS status-bubbles will still keep loading on Bugzilla.
>>>> 
>>>> Thanks
>>>> Aakash
>>> 
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] EWS down for emergency maintenance

2021-02-20 Thread Aakash Jain via webkit-dev
EWS (including commit-queue) is back online.  Please let me know if anyone 
notices any issue.

Thanks
Aakash

> On Feb 20, 2021, at 8:20 AM, Aakash Jain  wrote:
> 
> Hi Everyone,
> 
> EWS (including commit-queue) is currently down for emergency maintenance. 
> Status bubbles on existing Bugzilla bugs should still keep loading.
> 
> I'll send an update when the maintenance is complete.
> 
> Thanks
> Aakash

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] EWS down for emergency maintenance

2021-02-20 Thread Aakash Jain via webkit-dev
Hi Everyone,

EWS (including commit-queue) is currently down for emergency maintenance. 
Status bubbles on existing Bugzilla bugs should still keep loading.

I'll send an update when the maintenance is complete.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] build.webkit.org Upgraded to latest Buildbot

2021-02-08 Thread Aakash Jain via webkit-dev
Hello Everyone,

We have upgraded build.webkit.org to latest Buildbot (v2.10.1 released in 2021) 
(https://webkit.org/b/175056). Earlier it was running an ancient version of 
Buildbot, v0.8.6p1 (released in 2012). This is a brand new server. Buildbot 
doesn't support migrating data from old version. Old instance is accessible 
through https://build-old.webkit.org 

Known issue: Login functionality is not enabled for everyone yet.

Please let me know if you notice any issue.

Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Commit-Queue failing intermittently

2021-02-04 Thread Aakash Jain via webkit-dev
We are noticing intermittent issues with Commit-Queue failing to commit, since 
we moved it to github. We are looking into it. Meanwhile, if Commit-Queue fails 
on your patch, you can set cq+ again to retry.

Thanks
Aakash

> On Jan 19, 2021, at 4:56 PM, Aakash Jain  wrote:
> 
> All the issues have been fixed.
> 
> Please let me know if anyone notices any issue.
> 
> Thanks
> Aakash
> 
>> On Jan 19, 2021, at 12:23 PM, Aakash Jain  wrote:
>> 
>> We are noticing some issues on Style-Queue and Commit-Queue. We are looking 
>> into that.
>> 
>> Thanks
>> Aakash
>> 
>>> On Jan 19, 2021, at 9:58 AM, Aakash Jain  wrote:
>>> 
>>> Hi Everyone,
>>> 
>>> We need to do some maintenance on the EWS bots (in order to switch the 
>>> repository from git.webkit.org to https://github.com/webkit/webkit, 
>>> https://bugs.webkit.org/show_bug.cgi?id=220479). EWS buildbot will be 
>>> offline for about an hour, so it wouldn't process any new builds. However, 
>>> EWS status-bubbles will still keep loading on Bugzilla.
>>> 
>>> Thanks
>>> Aakash
>> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] EWS offline for maintenance for about an hour

2021-01-19 Thread Aakash Jain via webkit-dev
All the issues have been fixed.

Please let me know if anyone notices any issue.

Thanks
Aakash

> On Jan 19, 2021, at 12:23 PM, Aakash Jain  wrote:
> 
> We are noticing some issues on Style-Queue and Commit-Queue. We are looking 
> into that.
> 
> Thanks
> Aakash
> 
>> On Jan 19, 2021, at 9:58 AM, Aakash Jain  wrote:
>> 
>> Hi Everyone,
>> 
>> We need to do some maintenance on the EWS bots (in order to switch the 
>> repository from git.webkit.org to https://github.com/webkit/webkit, 
>> https://bugs.webkit.org/show_bug.cgi?id=220479). EWS buildbot will be 
>> offline for about an hour, so it wouldn't process any new builds. However, 
>> EWS status-bubbles will still keep loading on Bugzilla.
>> 
>> Thanks
>> Aakash
> 
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] EWS offline for maintenance for about an hour

2021-01-19 Thread Aakash Jain via webkit-dev
We are noticing some issues on Style-Queue and Commit-Queue. We are looking 
into that.

Thanks
Aakash

> On Jan 19, 2021, at 9:58 AM, Aakash Jain  wrote:
> 
> Hi Everyone,
> 
> We need to do some maintenance on the EWS bots (in order to switch the 
> repository from git.webkit.org to https://github.com/webkit/webkit, 
> https://bugs.webkit.org/show_bug.cgi?id=220479). EWS buildbot will be offline 
> for about an hour, so it wouldn't process any new builds. However, EWS 
> status-bubbles will still keep loading on Bugzilla.
> 
> Thanks
> Aakash

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] EWS offline for maintenance for about an hour

2021-01-19 Thread Aakash Jain via webkit-dev
Hi Everyone,

We need to do some maintenance on the EWS bots (in order to switch the 
repository from git.webkit.org to https://github.com/webkit/webkit, 
https://bugs.webkit.org/show_bug.cgi?id=220479). EWS buildbot will be offline 
for about an hour, so it wouldn't process any new builds. However, EWS 
status-bubbles will still keep loading on Bugzilla.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Problems with code review tool

2020-09-23 Thread Aakash Jain
Interesting. I haven't noticed such problem myself and wasn't aware of this 
issue. I was aware that the EWS status-bubble loading time has been increasing 
steadily as we have been increasing number of EWS queues. Tracked in 
https://bugs.webkit.org/show_bug.cgi?id=214821. I will prioritize fixing it.

Also bugs.webkit.org loads ews status-bubbles from ews.webkit.org in an iframe. 
I never noticed any issue on bugs.webkit.org itself because of slow loading of 
status-bubble. It would be interesting to know if this is a browser specific 
issue.

Thanks
Aakash

> On Sep 23, 2020, at 5:39 PM, Ken Russell  wrote:
> 
> Thanks Saam for your reply.
> 
> Unfortunately this happens nearly every time I use the code review tool.
> 
> Aakash, since this seems to be related to the EWS, is it something you might 
> be able to debug? I also frequently see bugs.webkit.org 
>  tabs display the spinner forever, and it seems to 
> be caused by loading resources from ews.webkit.org . 
> It's likely the same problem.
> 
> Thanks,
> 
> -Ken
> 
> 
> 
> On Tue, Sep 22, 2020 at 7:25 AM Saam Barati  > wrote:
> Hi Ken,
> 
> I think this is a known problem. Unfortunately, this happens to me a few 
> times a year as well.
> 
> - Saam
> 
>> On Sep 21, 2020, at 2:26 PM, Ken Russell > > wrote:
>> 
>> Hi,
>> 
>> Often when I publish code reviews on bugs.webkit.org 
>> , my browser tab gets stuck waiting for 
>> ews.webkit.org . Reloading the bug shows my 
>> comments, but because the submission doesn't complete, the review comments 
>> are still treated as pending by the tool, so publishing again causes 
>> duplicates.
>> 
>> Has anyone else run into this?
>> 
>> (My browser is Chrome on macOS.)
>> 
>> Thanks,
>> 
>> -Ken
>> 
>> 
>> -- 
>> I support flexible work schedules, and I’m sending this email now because it 
>> is within the hours I’m working today.  Please do not feel obliged to reply 
>> straight away - I understand that you will reply during the hours you work, 
>> which may not match mine. (credit: jparent@)
>> 
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org 
>> https://lists.webkit.org/mailman/listinfo/webkit-dev 
>> 
> 
> 
> 
> -- 
> I support flexible work schedules, and I’m sending this email now because it 
> is within the hours I’m working today.  Please do not feel obliged to reply 
> straight away - I understand that you will reply during the hours you work, 
> which may not match mine. (credit: jparent@)
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] EWS now cq- the patches with build or layout test failure

2020-09-09 Thread Aakash Jain
Hi All,

From now on, whenever EWS detects any build failure or layout test failure 
caused by a patch, it will mark the patch cq-.

If the patch was in commit queue at the time it was marked cq-, commit queue 
would not land it. You can always mark the patch cq+ again (if you think that 
EWS was incorrect), and commit queue would land the patch.

Reference: https://bugs.webkit.org/show_bug.cgi?id=214194

Please let me know if you notice any issue or have any feedback.

Thanks
Aakash

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] EWS now sends email notifications for failures

2020-08-25 Thread Aakash Jain
Hi Everyone,

I am happy to announce another EWS feature: email notifications.

>From now on, whenever EWS detects any build or test failure caused by a patch, 
>it will send email notification to the patch author. In case of build failure, 
>email will contain the relevant error lines, so as to help in quickly 
>identifying the issue. In case of test failure, email will contain the list of 
>failing test, along with the link to test history.

In case someone prefers not to receive these emails, they can unsubscribe 
either by emailing me or adding their email id to EMAIL_IDS_TO_UNSUBSCRIBE in 
Tools/BuildSlaveSupport/ews-build/emails.json

These email notifications are added as per the feedback received in a survey we 
did earlier (in 
https://lists.webkit.org/pipermail/webkit-dev/2019-December/030980.html). Some 
more details in: https://bugs.webkit.org/show_bug.cgi?id=215220#c0

Please let me know if you notice any issue or have any feedback.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] EWS now parses error logs in case of build failure

2020-08-11 Thread Aakash Jain
Hi Everyone,

I have added the ability to parse error logs in EWS for Windows and WinCairo as 
well (in https://trac.webkit.org/r265391).

It seems to be working fine. For example, in 
https://ews-build.webkit.org/#/builders/10/builds/33343, in step #8 WebKit 
failed to compile. Now, the 'errors' log is also present, containing only 1 
line which is the relevant error. Hopefully this would make it easier to spot 
the relevant errors from large logs.

Please let me know if you notice any issue or have any feedback..

Thanks
Aakash

> On Oct 29, 2019, at 7:05 PM, Aakash Jain  wrote:
> 
> Hi Everyone,
> 
> I am happy to announce another EWS feature.
> 
> From now on, in case of build failure, EWS will parse the errors and display 
> them in a separate 'errors' log. You wouldn't have to search through 
> thousands of lines of logs to find the error message.
> 
> For example, in https://ews-build.webkit.org/#/builders/16/builds/6054, in 
> step #7 WebKit failed to compile. Complete logs (stdio) are 38,000+ lines, 
> and the error is not at the end of the logs. Normally, it requires some 
> searching through the logs to find the relevant errors. But now, there is 
> another 'errors' log, which contains just the relevant 11 lines (containing 
> error and few related lines to provide additional context).
> 
> Hopefully this would save some time and efforts previously spent on searching 
> through the large logs.
> 
> Note that this information is not displayed in status-bubble tool-tip, since 
> this might be lot of text to display in the tooltip. My further plan is to 
> make this information more readily available, by adding it to a custom 
> designed page which will open on clicking the status bubble 
> https://webkit.org/b/197522
> 
> Please let me know if you notice any issues or have any feedback.
> 
> Thanks
> Aakash
> 
> Reference: https://webkit.org/b/203418


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] New EWS queues for tvOS and watchOS

2020-07-14 Thread Aakash Jain
I just modified watchOS EWS queue to build both arm64_32 and armv7k 
architectures (in https://bugs.webkit.org/show_bug.cgi?id=214279). Thanks 
Yusuke for pointing it out.

Also, in the process, I figured out that the build for armv7k was in fact 
broken. Thanks Jonathan for fixing that (in https://trac.webkit.org/r264357).

As always, please let me know if you notice any issue or have any feedback.

Thanks
Aakash

> On Jul 13, 2020, at 12:36 AM, Keith Miller  wrote:
> 
> It looks like just arm64_32.
> 
> Cheers,
> Keith
> 
>> On Jul 11, 2020, at 8:41 PM, Yusuke Suzuki  wrote:
>> 
>> Nice,
>> 
>> Is watchOS build ARMv7k? Or is it ARM64_32?
>> 
>> -Yusuke
>> 
>>> On Jul 11, 2020, at 8:09 AM, Aakash Jain  wrote:
>>> 
>>> Hi Everyone,
>>> 
>>> I am happy to announce that we have added new EWS queues for tvOS and 
>>> watchOS. From now on, you would see four new status bubbles (on Bugzilla 
>>> bug) named: tv, tv-sim, watch, watch-sim (corresponding to device and 
>>> simulator builds for watchOS and tvOS). This should help in preventing any 
>>> accidental build breakage on these platforms.
>>> 
>>> Please let me know if you notice any issue.
>>> 
>>> Thanks
>>> Aakash
>>> ___
>>> webkit-dev mailing list
>>> webkit-dev@lists.webkit.org
>>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>> 
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] New EWS queues for tvOS and watchOS

2020-07-11 Thread Aakash Jain
Hi Everyone,

I am happy to announce that we have added new EWS queues for tvOS and watchOS. 
From now on, you would see four new status bubbles (on Bugzilla bug) named: tv, 
tv-sim, watch, watch-sim (corresponding to device and simulator builds for 
watchOS and tvOS). This should help in preventing any accidental build breakage 
on these platforms.

Please let me know if you notice any issue.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] ios-wk2 ews queue slow due to fast/hidpi/image-srcset-relative-svg-canvas-2x.html test

2020-03-31 Thread Aakash Jain
https://bugs.webkit.org/show_bug.cgi?id=207038 has been fixed, thanks to Said 
Abou-Hallawa.

ios-wk2 ews queue is fast again (https://ews-build.webkit.org/#/builders/24). 
Please let me know if you notice any issue.

Thanks
Aakash

> On Mar 26, 2020, at 7:01 PM, Aakash Jain  wrote:
> 
> Hi Everyone,
> 
> Many of you might have noticed that ios-wk2 EWS queue has been slow for a 
> while now. This is primarily due to the layout test 
> fast/hidpi/image-srcset-relative-svg-canvas-2x.html being extremely flaky on 
> EWS. Sometimes this also results in false positive, causing EWS to think that 
> the patch being tested broke this test (please ignore that false positive for 
> now).
> 
> We tried to skip this test, but that caused subsequent test to fail. This is 
> because some previous test is changing the state in a manner which causes 
> this (or similar) test to fail. We are looking into identifying the bad test 
> and fix the root-cause, but it seems like it might take a while to get that 
> fixed. I'll let you know once it's fixed.
> 
> Tracked in https://bugs.webkit.org/show_bug.cgi?id=206993 and 
> https://bugs.webkit.org/show_bug.cgi?id=207038
> 
> Thanks
> Aakash

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] ios-wk2 ews queue slow due to fast/hidpi/image-srcset-relative-svg-canvas-2x.html test

2020-03-26 Thread Aakash Jain
Hi Everyone,

Many of you might have noticed that ios-wk2 EWS queue has been slow for a while 
now. This is primarily due to the layout test 
fast/hidpi/image-srcset-relative-svg-canvas-2x.html being extremely flaky on 
EWS. Sometimes this also results in false positive, causing EWS to think that 
the patch being tested broke this test (please ignore that false positive for 
now).

We tried to skip this test, but that caused subsequent test to fail. This is 
because some previous test is changing the state in a manner which causes this 
(or similar) test to fail. We are looking into identifying the bad test and fix 
the root-cause, but it seems like it might take a while to get that fixed. I'll 
let you know once it's fixed.

Tracked in https://bugs.webkit.org/show_bug.cgi?id=206993 and 
https://bugs.webkit.org/show_bug.cgi?id=207038

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Transitioning commit-queue to new EWS and deprecating old EWS

2020-03-23 Thread Aakash Jain
Hi Everyone,

commit-queue should be much faster now (after https://webkit.org/b/208938). For 
e.g. in https://ews-build.webkit.org/#/builders/28/builds/117 commit-queue took 
~2 minutes to commit. This is because commit-queue now skips layout tests if 
the mac-wk2 ews is already green. Also builds are usually fast since EWS does 
incremental builds. This feature was there in old commit-queue but was mostly 
broken.

I also want to get some feedback on 'https://webkit.org/b/208941 regarding how 
should commit-queue handle build/test failures on other EWSes. Following might 
be few options, please feel free to add anything.

#1: Whenever a builder or tester fail, it will set commit-queue flag to cq-. 
This would also result in bugzilla sending a notification email to the patch 
author and cced people. Anyone can still cq+ the patch and commit-queue will 
commit it. This is similar to old EWS behavior.

#2: commit-queue should check the patch status on few specific builders (e.g.: 
ios, mac, mac-debug, gtk, wpe etc), and do not commit if any of those EWS is 
red. This might make it little difficult for people to override EWS false 
positive (that's why this check should be implemented only for builders).

#3: commit-queue should wait for all EWSes to be green (this might not be very 
practical given that we have various flaky tests which sometimes create false 
positive, or one particular queue might get backlogged sometimes due to issues 
on bots).

Thanks
Aakash

> On Mar 18, 2020, at 3:12 PM, Aakash Jain  wrote:
> 
> Hello Everyone,
> 
> I have transitioned commit-queue from old EWS to new EWS. This was the last 
> queue on old EWS. I will soon start deleting old EWS code from the repository.
> 
> While transitioning the commit-queue, I have also made several improvements:
> 
> - commit-queue now supports security bugs
> 
> - commit-queue now leaves only one comment on Bugzilla (instead of two 
> previously) after successfully landing the patch
> 
> - Less bugzilla comments on failures and more readable content (no more 
> detailed logs in comments like 
> https://bugs.webkit.org/show_bug.cgi?id=207727#c10).
> 
> Also commit-queue now runs mac-wk2 tests instead of mac-wk1 tests. Please let 
> me know if you notice any issue.
> 
> Thanks
> Aakash

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Transitioning commit-queue to new EWS and deprecating old EWS

2020-03-18 Thread Aakash Jain
Hello Everyone,

I have transitioned commit-queue from old EWS to new EWS. This was the last 
queue on old EWS. I will soon start deleting old EWS code from the repository.

While transitioning the commit-queue, I have also made several improvements:

- commit-queue now supports security bugs

- commit-queue now leaves only one comment on Bugzilla (instead of two 
previously) after successfully landing the patch

- Less bugzilla comments on failures and more readable content (no more 
detailed logs in comments like 
https://bugs.webkit.org/show_bug.cgi?id=207727#c10 
).

Also commit-queue now runs mac-wk2 tests instead of mac-wk1 tests. Please let 
me know if you notice any issue.

Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Updating iOS EWS

2020-02-12 Thread Aakash Jain
The bug (https://webkit.org/b/207115) was fixed yesterday, and api-ios ews is back to normal. Please let me know if you notice any issue.ThanksAakashOn Feb 3, 2020, at 1:56 PM, Aakash Jain  wrote:We are noticing various API test failures on iOS after the OS update. Some of the API tests on iOS are being very flaky, causing api-ios EWS queue to slow down and occasionally blaming the patch being tested for those test failure. Please verify the api-ios EWS result accordingly.We are actively looking into the issue. Tracked in https://bugs.webkit.org/show_bug.cgi?id=207115ThanksAakashOn Jan 31, 2020, at 2:28 PM, Matt Lewis  wrote:Hello,We are updating iOS ews to the newest release of Catalina with the coinciding sdk for iOS. EWS for these queues will be down temporarily. Please plan accordingly, we don't expect it to take more than an hour or so.___webkit-dev mailing listwebkit-dev@lists.webkit.orghttps://lists.webkit.org/mailman/listinfo/webkit-dev___webkit-dev mailing listwebkit-dev@lists.webkit.orghttps://lists.webkit.org/mailman/listinfo/webkit-dev___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Updating iOS EWS

2020-02-03 Thread Aakash Jain
We are noticing various API test failures on iOS after the OS update. Some of 
the API tests on iOS are being very flaky, causing api-ios EWS queue to slow 
down and occasionally blaming the patch being tested for those test failure. 
Please verify the api-ios EWS result accordingly.

We are actively looking into the issue. Tracked in 
https://bugs.webkit.org/show_bug.cgi?id=207115

Thanks
Aakash

> On Jan 31, 2020, at 2:28 PM, Matt Lewis  wrote:
> 
> Hello,
> 
> We are updating iOS ews to the newest release of Catalina with the coinciding 
> sdk for iOS. EWS for these queues will be down temporarily. Please plan 
> accordingly, we don't expect it to take more than an hour or so.
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] macOS EWS queues down for maintenance

2020-01-10 Thread Aakash Jain
The upgrade is complete and all the queues are back online (as of this morning).

We are noticing some layout-test failures on mac-debug-wk1 queue and currently 
investigating it, tracked in https://bugs.webkit.org/show_bug.cgi?id=206073

If you notice any issue, please let us know.

Thanks
Aakash

> On Jan 9, 2020, at 6:23 PM, Aakash Jain  wrote:
> 
> Quick Update: we are still working on updating the bots and resolving any 
> issues. We also upgraded hardware for few of the bots (from MacPro5,1 to 
> MacPro 6,1). We expect it to be finished by tomorrow.
> 
> Thanks
> Aakash
> 
>> On Jan 8, 2020, at 10:41 PM, Aakash Jain > <mailto:aakash_j...@apple.com>> wrote:
>> 
>> Hi Everyone,
>> 
>> We are in process of upgrading our macOS bots from High Sierra to Mojave 
>> https://bugs.webkit.org/show_bug.cgi?id=205948 
>> <https://bugs.webkit.org/show_bug.cgi?id=205948>
>> 
>> We are noticing some unexpected configuration issues. For now, we have 
>> paused most of macOS EWS queues while we do the upgrade and fix any 
>> configuration issues on the bots.
>> 
>> We will try to keep the downtime to minimum. Sorry for the inconvenience and 
>> thank you for your patience.
>> 
>> Thanks
>> Aakash
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] macOS EWS queues down for maintenance

2020-01-09 Thread Aakash Jain
Quick Update: we are still working on updating the bots and resolving any 
issues. We also upgraded hardware for few of the bots (from MacPro5,1 to MacPro 
6,1). We expect it to be finished by tomorrow.

Thanks
Aakash

> On Jan 8, 2020, at 10:41 PM, Aakash Jain  wrote:
> 
> Hi Everyone,
> 
> We are in process of upgrading our macOS bots from High Sierra to Mojave 
> https://bugs.webkit.org/show_bug.cgi?id=205948 
> <https://bugs.webkit.org/show_bug.cgi?id=205948>
> 
> We are noticing some unexpected configuration issues. For now, we have paused 
> most of macOS EWS queues while we do the upgrade and fix any configuration 
> issues on the bots.
> 
> We will try to keep the downtime to minimum. Sorry for the inconvenience and 
> thank you for your patience.
> 
> Thanks
> Aakash

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] macOS EWS queues down for maintenance

2020-01-08 Thread Aakash Jain
Hi Everyone,

We are in process of upgrading our macOS bots from High Sierra to Mojave 
https://bugs.webkit.org/show_bug.cgi?id=205948 


We are noticing some unexpected configuration issues. For now, we have paused 
most of macOS EWS queues while we do the upgrade and fix any configuration 
issues on the bots.

We will try to keep the downtime to minimum. Sorry for the inconvenience and 
thank you for your patience.

Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Handling flaky layout-test failures in EWS

2019-12-05 Thread Aakash Jain


> On Dec 3, 2019, at 1:54 PM, Ryosuke Niwa  wrote:
> 
> 
> On Tue, Dec 3, 2019 at 9:29 AM Alexey Proskuryakov  <mailto:a...@webkit.org>> wrote:
> 
> Yes, I think that this makes more sense than retrying.
> 
> What is the current behavior when a patch introduces substantial flakiness? 
> E.g. this scenario:
> 
> - First test run produces 5 failures.
> - Second test run produces 5 different failures.
> - Clean re-run produces no failures.
> 
> This looks like decent evidence to make the EWS bubble red. I don't know what 
> exactly the threshold should be, but that should certainly be below 30.
> 
> This makes sense to me.
> 
> Another scenario where flaky failures might be a real regression is when 
> tests which never failed before starts failing flakily.
> 
> Can we fetch data from the flakiness dashboard and see if we’ve ever observed 
> the flakiness on a given test? That would let us get out of the guessing game.

Yes, that's a good idea, and something we have been exploring lately. Now we 
have new results database (https://results.webkit.org 
<https://results.webkit.org/>) as well which have REST API support which EWS 
can use. Using data from results database, we can have significant improvements 
in efficiency and flakiness detection in EWS. We started with some ideas in 
https://bugs.webkit.org/show_bug.cgi?id=204368 
<https://bugs.webkit.org/show_bug.cgi?id=204368>.

> 
> I understand some flakiness might be specific to EWS but I’d imagine that’d 
> be more of minority. If it is common, we could also make EWS bots 
> periodically run full tests without patches and report results to flakiness 
> dashboard so that we have a corpse of flaky teat failures on EWS bots as well.
> 
> - R. Niwa
> 
>> 3 дек. 2019 г., в 8:40 AM, Aakash Jain > <mailto:aakash_j...@apple.com>> написал(а):
> 
>> 
>> Hi Everyone,
>> 
>> We have various layout-tests which are flaky (which sometimes pass and 
>> sometimes fail/crash/timeout). EWS needs to work despite these flaky tests, 
>> and need to be able to tell whether the patch being tested introduced any 
>> test failure or not.
>> 
>> In EWS, we have logic (same logic in both old and new EWS) on how to deal 
>> with flaky tests. The logic is roughly following:
>> 
>> - We apply the patch and build.
>> 
>> - Run layout-tests. During the test-run, we retry the failed tests. If those 
>> tests pass in retry, the run is considered to have no failing test (this 
>> retry part is same for EWS / build.webkit.org <http://build.webkit.org/> / 
>> manual-run and part of run-webkit-test script itself).
>> 
>> - If a test-run has some failures, EWS retry the test-run one more time (to 
>> check if those test failures were flaky). 
>> 
>> - Then we do one more test-run on clean-tree (without the patch), and 
>> compare the results of first run, second run, and clean tree run.
>> 
>> - After that we analyze the results from all three test-runs (which we call 
>> first_run, second_run and clean_tree_run). 
>> 
>> 
>> If there are different test failures in first and second runs (i.e.: flaky 
>> test failures), and the patch being tested did not introduce any new test 
>> failure, we retry the entire build (probably hoping that next time we will 
>> not see the flakiness). This retry can result in almost infinite loop, if 
>> someone commits a very flaky test (which is not rare), and would cause EWS 
>> to be almost stuck until the flakiness is fixed.
>> 
>> From 
>> https://trac.webkit.org/browser/webkit/trunk/Tools/Scripts/webkitpy/tool/bot/patchanalysistask.py#L352
>>  
>> <https://trac.webkit.org/browser/webkit/trunk/Tools/Scripts/webkitpy/tool/bot/patchanalysistask.py#L352>
>>   ('Defer' means retrying the build).
>> '''
>> 350# At this point we know that at least one test flaked, but no 
>> consistent failures
>> 351# were introduced. This is a bit of a grey-zone.
>> 352return False  # Defer patch
>> '''
>> 
>> 
>> Retrying the entire build, just because of few flaky tests seems excessive 
>> and inefficient. This doesn't help identify flaky tests, and only delays the 
>> EWS result. Instead, we should mark the build as SUCCESS (since the patch 
>> did not introduce any new consistent test failure).
>> 
>> In other EWS test-suites like API tests and JSC tests, we do not retry the 
>> entire build on flaky test failures, we ignore the flaky tests, and only 
>> focus on tests which failed consistently. We should do the similar thing for 
>> layout-tests as well.
>> 
>

Re: [webkit-dev] Handling flaky layout-test failures in EWS

2019-12-05 Thread Aakash Jain
> What is the current behavior when a patch introduces substantial flakiness?

For the patch which introduces substantial flakiness (and no consistent failure 
in EWS), prior to r253049 (which I landed two days back), the patch would be 
infinitely retried (unless atleast one of the flaky test failed consistently, 
or the patch is obsolete).

After r253049, the patch would be marked as passing. EWS is simply not 
detecting flakiness introduced by a patch (since it doesn't know if the 
flakiness is introduced by the patch or was pre-existing). However, if any of 
those 10 tests failed consistently in two runs, the patch would be marked as 
failing.

We can work towards improving EWS to better detect flakiness by using data from 
new flakiness dashboard (which now has the API as well).

> This looks like decent evidence to make the EWS bubble red.
We can do that. Have we seen any patch in the past which would fail 5+ tests in 
first run, and fail completely different 5+ tests in second run? Also, what 
should be the threshold, should it be 10 (5+5) different test failures (or 
lower) in two runs?

Thanks
Aakash

> On Dec 3, 2019, at 12:28 PM, Alexey Proskuryakov  wrote:
> 
> 
> Yes, I think that this makes more sense than retrying.
> 
> What is the current behavior when a patch introduces substantial flakiness? 
> E.g. this scenario:
> 
> - First test run produces 5 failures.
> - Second test run produces 5 different failures.
> - Clean re-run produces no failures.
> 
> This looks like decent evidence to make the EWS bubble red. I don't know what 
> exactly the threshold should be, but that should certainly be below 30.
> 
> - Alexey
> 
> 
>> 3 дек. 2019 г., в 8:40 AM, Aakash Jain > <mailto:aakash_j...@apple.com>> написал(а):
>> 
>> Hi Everyone,
>> 
>> We have various layout-tests which are flaky (which sometimes pass and 
>> sometimes fail/crash/timeout). EWS needs to work despite these flaky tests, 
>> and need to be able to tell whether the patch being tested introduced any 
>> test failure or not.
>> 
>> In EWS, we have logic (same logic in both old and new EWS) on how to deal 
>> with flaky tests. The logic is roughly following:
>> 
>> - We apply the patch and build.
>> 
>> - Run layout-tests. During the test-run, we retry the failed tests. If those 
>> tests pass in retry, the run is considered to have no failing test (this 
>> retry part is same for EWS / build.webkit.org <http://build.webkit.org/> / 
>> manual-run and part of run-webkit-test script itself).
>> 
>> - If a test-run has some failures, EWS retry the test-run one more time (to 
>> check if those test failures were flaky). 
>> 
>> - Then we do one more test-run on clean-tree (without the patch), and 
>> compare the results of first run, second run, and clean tree run.
>> 
>> - After that we analyze the results from all three test-runs (which we call 
>> first_run, second_run and clean_tree_run). 
>> 
>> 
>> If there are different test failures in first and second runs (i.e.: flaky 
>> test failures), and the patch being tested did not introduce any new test 
>> failure, we retry the entire build (probably hoping that next time we will 
>> not see the flakiness). This retry can result in almost infinite loop, if 
>> someone commits a very flaky test (which is not rare), and would cause EWS 
>> to be almost stuck until the flakiness is fixed.
>> 
>> From 
>> https://trac.webkit.org/browser/webkit/trunk/Tools/Scripts/webkitpy/tool/bot/patchanalysistask.py#L352
>>  
>> <https://trac.webkit.org/browser/webkit/trunk/Tools/Scripts/webkitpy/tool/bot/patchanalysistask.py#L352>
>>   ('Defer' means retrying the build).
>> '''
>> 350# At this point we know that at least one test flaked, but no 
>> consistent failures
>> 351# were introduced. This is a bit of a grey-zone.
>> 352return False  # Defer patch
>> '''
>> 
>> 
>> Retrying the entire build, just because of few flaky tests seems excessive 
>> and inefficient. This doesn't help identify flaky tests, and only delays the 
>> EWS result. Instead, we should mark the build as SUCCESS (since the patch 
>> did not introduce any new consistent test failure).
>> 
>> In other EWS test-suites like API tests and JSC tests, we do not retry the 
>> entire build on flaky test failures, we ignore the flaky tests, and only 
>> focus on tests which failed consistently. We should do the similar thing for 
>> layout-tests as well.
>> 
>> I am going to make that change for layout tests in 
>> https://bugs.webkit.org/show_bug.cgi?id=204769 
>> <https://bugs.webkit.org/show_bug.cgi?id=204769>. Please let me know if 
>> anyone has a different opinion.
>> 
>> Thanks
>> Aakash
> 
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Handling flaky layout-test failures in EWS

2019-12-03 Thread Aakash Jain
Hi Everyone,

We have various layout-tests which are flaky (which sometimes pass and 
sometimes fail/crash/timeout). EWS needs to work despite these flaky tests, and 
need to be able to tell whether the patch being tested introduced any test 
failure or not.

In EWS, we have logic (same logic in both old and new EWS) on how to deal with 
flaky tests. The logic is roughly following:

- We apply the patch and build.

- Run layout-tests. During the test-run, we retry the failed tests. If those 
tests pass in retry, the run is considered to have no failing test (this retry 
part is same for EWS / build.webkit.org / manual-run and part of 
run-webkit-test script itself).

- If a test-run has some failures, EWS retry the test-run one more time (to 
check if those test failures were flaky). 

- Then we do one more test-run on clean-tree (without the patch), and compare 
the results of first run, second run, and clean tree run.

- After that we analyze the results from all three test-runs (which we call 
first_run, second_run and clean_tree_run). 


If there are different test failures in first and second runs (i.e.: flaky test 
failures), and the patch being tested did not introduce any new test failure, 
we retry the entire build (probably hoping that next time we will not see the 
flakiness). This retry can result in almost infinite loop, if someone commits a 
very flaky test (which is not rare), and would cause EWS to be almost stuck 
until the flakiness is fixed.

>From 
>https://trac.webkit.org/browser/webkit/trunk/Tools/Scripts/webkitpy/tool/bot/patchanalysistask.py#L352
>  ('Defer' means retrying the build).
'''
350# At this point we know that at least one test flaked, but no consistent 
failures
351# were introduced. This is a bit of a grey-zone.
352return False  # Defer patch
'''


Retrying the entire build, just because of few flaky tests seems excessive and 
inefficient. This doesn't help identify flaky tests, and only delays the EWS 
result. Instead, we should mark the build as SUCCESS (since the patch did not 
introduce any new consistent test failure).

In other EWS test-suites like API tests and JSC tests, we do not retry the 
entire build on flaky test failures, we ignore the flaky tests, and only focus 
on tests which failed consistently. We should do the similar thing for 
layout-tests as well.

I am going to make that change for layout tests in 
https://bugs.webkit.org/show_bug.cgi?id=204769. Please let me know if anyone 
has a different opinion.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

2019-12-02 Thread Aakash Jain
There were multiple ideas discussed in this thread. I would like to gather more 
data about what do most people prefer. I have sent out a short survey in 
https://lists.webkit.org/pipermail/webkit-dev/2019-December/030980.html 
<https://lists.webkit.org/pipermail/webkit-dev/2019-December/030980.html>

Thanks
Aakash

> On Nov 5, 2019, at 12:04 PM, Alexey Proskuryakov  wrote:
> 
> 
> 
>> 4 нояб. 2019 г., в 1:37 PM, Ryosuke Niwa > <mailto:rn...@webkit.org>> написал(а):
>> 
>> 
>> On Mon, Nov 4, 2019 at 9:40 AM Alexey Proskuryakov > <mailto:a...@webkit.org>> wrote:
>> 
>> Can you elaborate on that, how exactly is e-mailing on first failure useful 
>> to reviewers?
>> 
>> Getting rid of Bugzilla comments was one of the goals of EWS rewrite, based 
>> on engineering feedback about noise in bugs and in e-mail, and I 
>> wholeheartedly agree with this feedback. So I think that comments are 
>> generally undesirable.
>> 
>> Since I don't understand what your precise scenario is, I may be make straw 
>> man arguments below, but here are some things that I think make the proposed 
>> behavior unhelpful (add a comment on first failure, or when all EWSes pass).
>> 
>> 1. EWS comments in Bugzilla are so annoying that some people take the 
>> radical step of manually hiding them. EWS history is archived anyway, there 
>> is no need to look into comments for it.
>> 
>> 2. There are often many people CC'ed on the bug to whom EWS data is 
>> irrelevant or even mysterious (e.g. reporters, web developers or 
>> non-reviewers). The noise is a slight annoyance, discouraging further 
>> participation in the project.
>> 
>> 3. I believe that for most reviewers, the mode of operation is one of the 
>> two: (1) do it when pinged directly, or (2) go over the review queue when 
>> one has the time. Getting EWS comments helps neither.
>> 
>> 4. Commenting when all EWSes pass is not very practical - it's too often 
>> that we have some stragglers that take days (or forever). I don't think that 
>> we can make it reliable even if we start actively policing EWS 
>> responsiveness.
>> 
>> 5. The reviewer likely wants to know the state of multiple EWSes if they are 
>> going to wait for EWS at all. What exactly are they going to do after 
>> getting an e-mail that one EWS failed?
>> 
>> I often use a EWS failure as a signal to wait reviewing a patch. Otherwise, 
>> a bug mail will stay in my inbox as one of items to get to.
>> 
>> I can see the usefulness in the somewhat unusual case of a super urgent 
>> patch. We may want multiple people to watch it, so that members of CC list 
>> would go and ask the patch author to update it with more urgency than e-mail 
>> allows for. I think that opt-in is a better mechanism for that, so that 
>> people who opted in would receive information about each EWS data point.
>> 
>> I think there is a value in knowing that a patch isn't ready instead of 
>> having to open the bug to realize that.
> 
> So just to clarify, 
> 
> - a major part of how you get to review bugs is by being CC'ed, and you 
> review them when you have the time to read bugmail;
> - and you don't open the bug in Bugzilla if there is already an EWS failure 
> by the time you read the e-mail where review is requested?
> 
> That's clearly a valid benefit. In my mind, it probably doesn't outweigh the 
> downsides. On the other hand, yours is a voice of someone who reviews way 
> more patches than Maciej and me combined these days, so maybe more e-mail is 
> an overall benefit to many of the reviewers.
> 
> - Alexey
> 
> 
> 
>> - R. Niwa
>>> 3 нояб. 2019 г., в 6:58 PM, Maciej Stachowiak >> <mailto:m...@apple.com>> написал(а):
>>> 
>>> 
>>> I think they are useful to actual and potential reviewers. Direct email to 
>>> the patch author is not something anyone can Cc themselves on, and is not 
>>> archived, so seems like a strictly worse form of communication.
>>> 
>>>> On Nov 2, 2019, at 9:34 AM, Alexey Proskuryakov >>> <mailto:a...@apple.com>> wrote:
>>>> 
>>>> 
>>>> My preference is still e-mailing the patch author directly (possibly, also 
>>>> having an option to opt in for anyone). Bugzilla comments will always be 
>>>> irrelevant for most people CC'ed on the bug, and they are almost always 
>>>> undesirable to keep within the discussion flow.
>>>> 
>>>> - Alexey
>>>> 
>>>>> 1 нояб. 2019 г., в 18:28, Aakash Ja

[webkit-dev] Survey regarding EWS

2019-12-02 Thread Aakash Jain
Hi Everyone,

I want to get everyone's opinion on EWS, especially regarding if (and how) EWS 
should send notifications when a patch fails on any EWS queue. Here is a link 
to a short survey: https://www.surveymonkey.com/r/XQ77LQD 


I would appreciate if you can fill it out.

Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] iOS EWS behind by 3 days??

2019-11-10 Thread Aakash Jain
This queue is back in good shape.

Alexey figured out that the root-cause was a configuration issue, which started 
after our recent upgrade of iOS EWS bots to Catalina.

Thanks
Aakash

> On Nov 5, 2019, at 11:57 AM, Alexey Proskuryakov  wrote:
> 
> 
> 
>> 4 нояб. 2019 г., в 11:30 PM, Ryosuke Niwa > > написал(а):
>> 
>> Hi all,
>> 
>> Does anyone know what's happening with iOS EWS?
> 
> Not yet, but it's part of a cluster of problems related to interactions 
> between simulator and specific macOS host versions.
> 
> https://bugs.webkit.org/show_bug.cgi?id=203792 
>  tracks this particular issue.
> 
> - Alexey
> 
> 
>> They're ~3 days behind now:
>> https://ews-build.webkit.org/#/builders/24 
>> 
>> 
>> - R. Niwa
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org 
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
> 
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Wincairo build broken

2019-11-02 Thread Aakash Jain
Hello,

Wincairo build seems to be broken.

I filed: https://bugs.webkit.org/show_bug.cgi?id=203791 
 [wincairo] 'deref': is not a 
member of 'WebCore::ServiceWorkerContainer::jobResolvedWithRegistration::___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

2019-11-01 Thread Aakash Jain
Sounds good. I prefer the single comment when the first failure occur. That way 
notification would be sent as soon as the first failure happens.

I'll implement that (assuming it's acceptable to everyone).

Thanks
Aakash

> On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak  wrote:
> 
> 
> How about only a single comment when the first failure occurs? (Or else when 
> all bots pass, if there is never a failure.)
> 
> This should help the author, the reviewer, and anyone else cc’d, without 
> being too spammy.
> 
>> On Nov 1, 2019, at 5:20 PM, Aakash Jain  wrote:
>> 
>> Hi Ryosuke,
>> 
>> Many people didn't like the noise by the EWS comments, and we removed the 
>> comments as per previous discussion in: 
>> https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html.
>> 
>> I agree with your point that having some kind of notification might be 
>> useful.
>> 
>> I proposed some ideas in 
>> https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html, 
>> but didn't get much feedback. If we can all agree on a solution, I can look 
>> into implementing it.
>> 
>> Thanks
>> Aakash
>> 
>>> On Oct 30, 2019, at 1:03 AM, Ryosuke Niwa  wrote:
>>> 
>>> These enhancements are great. There is one feature of the old EWS that I 
>>> really miss, which is that I used to get emails when some EWS failed. With 
>>> new EWS, I have to keep checking back the bugzilla to see if any of them 
>>> have failed periodically.
>>> 
>>> Can we add a feature to opt into such an email notification? Maybe a flag 
>>> on a patch or JSON configuration file somewhere.
>>> 
>>> - R. Niwa
>>> 
>>> On Tue, Oct 29, 2019 at 4:05 PM Aakash Jain  wrote:
>>> Hi Everyone,
>>> 
>>> I am happy to announce another EWS feature.
>>> 
>>> From now on, in case of build failure, EWS will parse the errors and 
>>> display them in a separate 'errors' log. You wouldn't have to search 
>>> through thousands of lines of logs to find the error message.
>>> 
>>> For example, in https://ews-build.webkit.org/#/builders/16/builds/6054, in 
>>> step #7 WebKit failed to compile. Complete logs (stdio) are 38,000+ lines, 
>>> and the error is not at the end of the logs. Normally, it requires some 
>>> searching through the logs to find the relevant errors. But now, there is 
>>> another 'errors' log, which contains just the relevant 11 lines (containing 
>>> error and few related lines to provide additional context).
>>> 
>>> Hopefully this would save some time and efforts previously spent on 
>>> searching through the large logs.
>>> 
>>> Note that this information is not displayed in status-bubble tool-tip, 
>>> since this might be lot of text to display in the tooltip. My further plan 
>>> is to make this information more readily available, by adding it to a 
>>> custom designed page which will open on clicking the status bubble 
>>> https://webkit.org/b/197522
>>> 
>>> Please let me know if you notice any issues or have any feedback.
>>> 
>>> Thanks
>>> Aakash
>>> 
>>> Reference: https://webkit.org/b/203418
>>> ___
>>> webkit-dev mailing list
>>> webkit-dev@lists.webkit.org
>>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>> -- 
>>> - R. Niwa
>> 
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

2019-11-01 Thread Aakash Jain
Hi Ryosuke,

Many people didn't like the noise by the EWS comments, and we removed the 
comments as per previous discussion in: 
https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html.

I agree with your point that having some kind of notification might be useful.

I proposed some ideas in 
https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html, but 
didn't get much feedback. If we can all agree on a solution, I can look into 
implementing it.

Thanks
Aakash

> On Oct 30, 2019, at 1:03 AM, Ryosuke Niwa  wrote:
> 
> These enhancements are great. There is one feature of the old EWS that I 
> really miss, which is that I used to get emails when some EWS failed. With 
> new EWS, I have to keep checking back the bugzilla to see if any of them have 
> failed periodically.
> 
> Can we add a feature to opt into such an email notification? Maybe a flag on 
> a patch or JSON configuration file somewhere.
> 
> - R. Niwa
> 
> On Tue, Oct 29, 2019 at 4:05 PM Aakash Jain  wrote:
> Hi Everyone,
> 
> I am happy to announce another EWS feature.
> 
> From now on, in case of build failure, EWS will parse the errors and display 
> them in a separate 'errors' log. You wouldn't have to search through 
> thousands of lines of logs to find the error message.
> 
> For example, in https://ews-build.webkit.org/#/builders/16/builds/6054, in 
> step #7 WebKit failed to compile. Complete logs (stdio) are 38,000+ lines, 
> and the error is not at the end of the logs. Normally, it requires some 
> searching through the logs to find the relevant errors. But now, there is 
> another 'errors' log, which contains just the relevant 11 lines (containing 
> error and few related lines to provide additional context).
> 
> Hopefully this would save some time and efforts previously spent on searching 
> through the large logs.
> 
> Note that this information is not displayed in status-bubble tool-tip, since 
> this might be lot of text to display in the tooltip. My further plan is to 
> make this information more readily available, by adding it to a custom 
> designed page which will open on clicking the status bubble 
> https://webkit.org/b/197522
> 
> Please let me know if you notice any issues or have any feedback.
> 
> Thanks
> Aakash
> 
> Reference: https://webkit.org/b/203418
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
> -- 
> - R. Niwa

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] EWS now parses error logs in case of build failure

2019-10-29 Thread Aakash Jain
Hi Everyone,

I am happy to announce another EWS feature.

>From now on, in case of build failure, EWS will parse the errors and display 
>them in a separate 'errors' log. You wouldn't have to search through thousands 
>of lines of logs to find the error message.

For example, in https://ews-build.webkit.org/#/builders/16/builds/6054, in step 
#7 WebKit failed to compile. Complete logs (stdio) are 38,000+ lines, and the 
error is not at the end of the logs. Normally, it requires some searching 
through the logs to find the relevant errors. But now, there is another 
'errors' log, which contains just the relevant 11 lines (containing error and 
few related lines to provide additional context).

Hopefully this would save some time and efforts previously spent on searching 
through the large logs.

Note that this information is not displayed in status-bubble tool-tip, since 
this might be lot of text to display in the tooltip. My further plan is to make 
this information more readily available, by adding it to a custom designed page 
which will open on clicking the status bubble https://webkit.org/b/197522

Please let me know if you notice any issues or have any feedback.

Thanks
Aakash

Reference: https://webkit.org/b/203418
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Planned EWS improvements

2019-10-22 Thread Aakash Jain
Hi Everyone,

I am very happy to announce that I have added the ability to retry the patches 
in EWS :)

From now on, for any patch, whenever any of the EWS status-bubble is red, you 
will see another button named 'Retry failed builds' next to the status-bubbles. 
Clicking on 'Retry failed builds' button would cause all the failed EWS builds 
for that patch to be retried. Please click on this button only when needed, as 
unnecessary retries might put load on the system and others patches might have 
to wait longer.

As always, please feel free to let me know if you notice any issue or have any 
feedback.

Known UI issues: https://bugs.webkit.org/show_bug.cgi?id=203249 
<https://bugs.webkit.org/show_bug.cgi?id=203249>

Thanks
Aakash


> On Sep 26, 2019, at 7:56 PM, Saam Barati  wrote:
> 
> Hi Aakash,
> 
> Thanks for doing this work.
> 
>> On Sep 26, 2019, at 11:27 AM, Aakash Jain > <mailto:aakash_j...@apple.com>> wrote:
>> 
>> ...
>> 
>> 3) Add a way to retry a patch in EWS. This would allow engineers to confirm 
>> that the failures indicated by EWS aren't flaky/incorrect. Maybe a good 
>> place to add the 'retry' button would be the status-bubble's tool-tip 
>> (visible only if the bubble is red) https://webkit.org/b/196599 
>> <https://webkit.org/b/196599>This would be amazing. (My 2c: I'd vote for the 
>> button being visible without tooltip.)
> 
> - Saam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Some EWS bots facing network issues while trying to reach S3

2019-10-22 Thread Aakash Jain
Hi All,

Some of the EWS bots are facing network issues while trying to reach S3 (since 
this morning). We are working to fix the issue as quickly as possible. Tracking 
in: https://bugs.webkit.org/show_bug.cgi?id=203246 


If you have any questions/concerns, please don't hesitate to reach out.

Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Wincairo build seems to be broken

2019-10-13 Thread Aakash Jain
Thanks Stephan for fixing the build failures (https://webkit.org/b/202879 and 
https://webkit.org/b/202893).

Both EWS and build.webkit.org are green again for WinCairo. e.g.: 
https://ews-build.webkit.org/#/builders/12/builds/7577 and 
https://build.webkit.org/builders/WinCairo%2064-bit%20WKL%20Debug%20%28Build%29/builds/11055

Thanks
Aakash

> On Oct 12, 2019, at 2:43 AM, Aakash Jain  wrote:
> 
> Tracking in https://bugs.webkit.org/show_bug.cgi?id=202879 
> <https://bugs.webkit.org/show_bug.cgi?id=202879>. Probably broken in 
> https://bugs.webkit.org/show_bug.cgi?id=202780 
> <https://bugs.webkit.org/show_bug.cgi?id=202780>
> 
> Unfortunately, as WinCairo EWS bots aren't in a good shape, it's probably 
> easy to accidentally break WinCairo (WinCairo EWS didn't run properly in 
> above case as well).
> 
> -Aakash
> 
>> On Oct 12, 2019, at 2:33 AM, Aakash Jain > <mailto:aakash_j...@apple.com>> wrote:
>> 
>> Hello All,
>> 
>> Wincairo build seems to be broken. EWS and build.webkit.org 
>> <http://build.webkit.org/> seems to be failing with below build error, e.g.: 
>> https://ews-build.webkit.org/#/builders/12/builds/7400/steps/9/logs/stdio 
>> <https://ews-build.webkit.org/#/builders/12/builds/7400/steps/9/logs/stdio>, 
>> https://build.webkit.org/builders/WinCairo%2064-bit%20WKL%20Debug%20%28Build%29/builds/11035/steps/compile-webkit/logs/stdio
>>  
>> <https://build.webkit.org/builders/WinCairo%2064-bit%20WKL%20Debug%20%28Build%29/builds/11035/steps/compile-webkit/logs/stdio>
>> 
>> Can someone have a look?
>> 
>> 
>> [1854/5075] Linking CXX shared library bin64\libGLESv2.dll
>> FAILED: bin64/libGLESv2.dll lib64/libGLESv2.lib 
>> cmd.exe /C "cd . && C:\tools\cmake\bin\cmake.exe -E vs_link_dll 
>> --intdir=Source\ThirdParty\ANGLE\CMakeFiles\GLESv2.dir --rc="C:\Program 
>> Files (x86)\Windows Kits\10\bin\10.0.17763.0\x64\rc.exe" --mt="C:\Program 
>> Files (x86)\Windows Kits\10\bin\10.0.17763.0\x64\mt.exe"
>> ... 
>> ...
>> ...
>>  winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib 
>> advapi32.lib /MANIFEST /MANIFESTFILE:bin64\libGLESv2.dll.manifest" failed 
>> (exit code 1120) with the following output:
>>Creating library lib64\libGLESv2.lib and object lib64\libGLESv2.exp
>> 
>> LINK : warning LNK4217: symbol 'ANGLEGetDisplayPlatform' defined in 
>> 'ANGLE.lib(Platform.cpp.obj)' is imported by 
>> 'proc_table_egl_autogen.cpp.obj' in function ...
>> 
>> LINK : warning LNK4217: symbol 'ANGLEResetDisplayPlatform' defined in 
>> 'ANGLE.lib(Platform.cpp.obj)' is imported by 
>> 'proc_table_egl_autogen.cpp.obj' in function ...
>> 
>> ANGLE.lib(Renderer11.cpp.obj) : error LNK2019: unresolved external symbol 
>> CreateDXGIFactory1 referenced in function ...
>> 
>> bin64\libGLESv2.dll : fatal error LNK1120: 1 unresolved externals
>> 
>> [1858/5075] Building CXX object 
>> Source\WebCore\PAL\pal\CMakeFiles\PAL.dir\FileSizeFormatter.cpp.obj
>> ninja: build stopped: subcommand failed.
>> 
>> 
>> Thanks
>> Aakash
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Wincairo build seems to be broken

2019-10-12 Thread Aakash Jain
Tracking in https://bugs.webkit.org/show_bug.cgi?id=202879 
<https://bugs.webkit.org/show_bug.cgi?id=202879>. Probably broken in 
https://bugs.webkit.org/show_bug.cgi?id=202780 
<https://bugs.webkit.org/show_bug.cgi?id=202780>

Unfortunately, as WinCairo EWS bots aren't in a good shape, it's probably easy 
to accidentally break WinCairo (WinCairo EWS didn't run properly in above case 
as well).

-Aakash

> On Oct 12, 2019, at 2:33 AM, Aakash Jain  wrote:
> 
> Hello All,
> 
> Wincairo build seems to be broken. EWS and build.webkit.org 
> <http://build.webkit.org/> seems to be failing with below build error, e.g.: 
> https://ews-build.webkit.org/#/builders/12/builds/7400/steps/9/logs/stdio 
> <https://ews-build.webkit.org/#/builders/12/builds/7400/steps/9/logs/stdio>, 
> https://build.webkit.org/builders/WinCairo%2064-bit%20WKL%20Debug%20%28Build%29/builds/11035/steps/compile-webkit/logs/stdio
>  
> <https://build.webkit.org/builders/WinCairo%2064-bit%20WKL%20Debug%20%28Build%29/builds/11035/steps/compile-webkit/logs/stdio>
> 
> Can someone have a look?
> 
> 
> [1854/5075] Linking CXX shared library bin64\libGLESv2.dll
> FAILED: bin64/libGLESv2.dll lib64/libGLESv2.lib 
> cmd.exe /C "cd . && C:\tools\cmake\bin\cmake.exe -E vs_link_dll 
> --intdir=Source\ThirdParty\ANGLE\CMakeFiles\GLESv2.dir --rc="C:\Program Files 
> (x86)\Windows Kits\10\bin\10.0.17763.0\x64\rc.exe" --mt="C:\Program Files 
> (x86)\Windows Kits\10\bin\10.0.17763.0\x64\mt.exe"
> ... 
> ...
> ...
>  winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib 
> advapi32.lib /MANIFEST /MANIFESTFILE:bin64\libGLESv2.dll.manifest" failed 
> (exit code 1120) with the following output:
>Creating library lib64\libGLESv2.lib and object lib64\libGLESv2.exp
> 
> LINK : warning LNK4217: symbol 'ANGLEGetDisplayPlatform' defined in 
> 'ANGLE.lib(Platform.cpp.obj)' is imported by 'proc_table_egl_autogen.cpp.obj' 
> in function ...
> 
> LINK : warning LNK4217: symbol 'ANGLEResetDisplayPlatform' defined in 
> 'ANGLE.lib(Platform.cpp.obj)' is imported by 'proc_table_egl_autogen.cpp.obj' 
> in function ...
> 
> ANGLE.lib(Renderer11.cpp.obj) : error LNK2019: unresolved external symbol 
> CreateDXGIFactory1 referenced in function ...
> 
> bin64\libGLESv2.dll : fatal error LNK1120: 1 unresolved externals
> 
> [1858/5075] Building CXX object 
> Source\WebCore\PAL\pal\CMakeFiles\PAL.dir\FileSizeFormatter.cpp.obj
> ninja: build stopped: subcommand failed.
> 
> 
> Thanks
> Aakash

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Wincairo build seems to be broken

2019-10-12 Thread Aakash Jain
Hello All,

Wincairo build seems to be broken. EWS and build.webkit.org 
 seems to be failing with below build error, e.g.: 
https://ews-build.webkit.org/#/builders/12/builds/7400/steps/9/logs/stdio, 
https://build.webkit.org/builders/WinCairo%2064-bit%20WKL%20Debug%20%28Build%29/builds/11035/steps/compile-webkit/logs/stdio

Can someone have a look?


[1854/5075] Linking CXX shared library bin64\libGLESv2.dll
FAILED: bin64/libGLESv2.dll lib64/libGLESv2.lib 
cmd.exe /C "cd . && C:\tools\cmake\bin\cmake.exe -E vs_link_dll 
--intdir=Source\ThirdParty\ANGLE\CMakeFiles\GLESv2.dir --rc="C:\Program Files 
(x86)\Windows Kits\10\bin\10.0.17763.0\x64\rc.exe" --mt="C:\Program Files 
(x86)\Windows Kits\10\bin\10.0.17763.0\x64\mt.exe"
... 
...
...
 winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib 
advapi32.lib /MANIFEST /MANIFESTFILE:bin64\libGLESv2.dll.manifest" failed (exit 
code 1120) with the following output:
   Creating library lib64\libGLESv2.lib and object lib64\libGLESv2.exp

LINK : warning LNK4217: symbol 'ANGLEGetDisplayPlatform' defined in 
'ANGLE.lib(Platform.cpp.obj)' is imported by 'proc_table_egl_autogen.cpp.obj' 
in function ...

LINK : warning LNK4217: symbol 'ANGLEResetDisplayPlatform' defined in 
'ANGLE.lib(Platform.cpp.obj)' is imported by 'proc_table_egl_autogen.cpp.obj' 
in function ...

ANGLE.lib(Renderer11.cpp.obj) : error LNK2019: unresolved external symbol 
CreateDXGIFactory1 referenced in function ...

bin64\libGLESv2.dll : fatal error LNK1120: 1 unresolved externals

[1858/5075] Building CXX object 
Source\WebCore\PAL\pal\CMakeFiles\PAL.dir\FileSizeFormatter.cpp.obj
ninja: build stopped: subcommand failed.


Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] JSC EWS being too optimistic

2019-10-09 Thread Aakash Jain
Hi Caio,

Thanks for reporting this issue.

Jonathan fixed the bug you reported 
(https://bugs.webkit.org/show_bug.cgi?id=202419 
). It was a regression from a 
recent change.

I verified the fix using the same patch you tested with earlier. jsc EWS is now 
(correctly) failing on that patch 
(https://bugs.webkit.org/show_bug.cgi?id=202140 
). Please let me know if you 
notice any issue.

Thanks
Aakash

> On Oct 3, 2019, at 10:14 AM, Caio Lima  wrote:
> 
> Hi all.
> 
> I've noticed that JSC bots (jsc, jsc-armv7, jsc-mips, jsc-386) are
> marking every patch as success, even if there is a build failure (see
> https://bug-202140-attachments.webkit.org/attachment.cgi?id=379917 for
> https://bugs.webkit.org/show_bug.cgi?id=202140). Is it already in
> someone's radar?If not, could anybody take a look on this? I tried to
> investigate, but I'm lacking knowledge of EWS code base. This already
> caused some noise into JSC patches that broke ARMv7 and MIPS ports. I
> opened this bug to report it
> https://bugs.webkit.org/show_bug.cgi?id=202419.
> 
> Thanks in advance,
> Caio Lima.
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Upgrading Buildbot on EWS

2019-10-06 Thread Aakash Jain
Hi All,

The upgrade went smoothly. EWS with Buildbot v1.8.2 seems to be running fine. 
Please let me know if anyone notices any issue.

https://ews-build.webkit.org/#/about <https://ews-build.webkit.org/#/about>

Thanks
Aakash


> On Oct 5, 2019, at 10:22 AM, Aakash Jain  wrote:
> 
> Hi All,
> 
> I am planning to upgrade Buildbot on EWS (ews-build.webkit.org) tomorrow 
> (Sunday, Oct 6th) around 6am PST from v1.7 to v1.8.2. It would give us few 
> security fixes. Note that we aren't upgrading to Buildbot 2.0+ yet, since it 
> need Python 3 compatibility. Python 3 compatibility work would be done 
> separately.
> 
> I have already tested this version (v1.8.2) on the UAT instance 
> (ews-build.webkit-uat.org). The risk is minimal and the downtime should be 
> only few minutes. Please let me know if there is any concern.
> 
> Thanks
> Aakash

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Upgrading Buildbot on EWS

2019-10-05 Thread Aakash Jain
Hi All,

I am planning to upgrade Buildbot on EWS (ews-build.webkit.org) tomorrow 
(Sunday, Oct 6th) around 6am PST from v1.7 to v1.8.2. It would give us few 
security fixes. Note that we aren't upgrading to Buildbot 2.0+ yet, since it 
need Python 3 compatibility. Python 3 compatibility work would be done 
separately.

I have already tested this version (v1.8.2) on the UAT instance 
(ews-build.webkit-uat.org). The risk is minimal and the downtime should be only 
few minutes. Please let me know if there is any concern.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Planned EWS improvements

2019-09-30 Thread Aakash Jain


> On Sep 30, 2019, at 9:32 AM, Adrian Perez de Castro  wrote:
> 
> Hello Aakash (and everybody else),
> 
> On Thu, 26 Sep 2019 14:27:25 -0400, Aakash Jain  wrote:
> 
>> I have been making number of improvements to EWS. I also have various
>> planned improvements to EWS […]
> 
> Thanks once again for your continued work on improving the EWS, it is
> much appreciated =)

Thank You for the feedback. I am hoping that Improved EWS would increase 
overall team's productivity as people have to deal less with build/test 
breakages and rollouts.

> 
>> Here is the list of improvements (in no particular order):
>> 
>> […]
>> 
>> 3) Add a way to retry a patch in EWS. This would allow engineers to confirm
>> that the failures indicated by EWS aren't flaky/incorrect. Maybe a good
>> place to add the 'retry' button would be the status-bubble's tool-tip
>> (visible only if the bubble is red) https://webkit.org/b/196599
>> <https://webkit.org/b/196599>
> 
> Being able to retry a patch without having to re-upload it, and more
> importantly without having to ping another person, would be neat. I
> don't have a strong opinion about which tasks to prioritize first, tho.
> 
>> […]
>> 
>> 6) Add more test-suites to EWS (e.g.: LLDB tests, resultsdbpy tests)
>> https://webkit.org/b/189206 <https://webkit.org/b/189206>,
>> https://webkit.org/b/201928 <https://webkit.org/b/201928>
> 
> In this line of adding more things that the EWS could test, among the
> developers of the GTK/WPE ports we think it would be manageable for us
> to have bots run API tests. There is some work we need to do, and the
> intention is to start with GTK first, see how it goes, and then add WPE
> afterwards. Carlos opened a bug for this, I took the liberty of CC'ing
> you assuming that you might want to at least keep tabs on it (feel free
> to unsubscribe):
> 
>  https://bugs.webkit.org/show_bug.cgi?id=202361

Sure, I will work with you and Carlos on this.

> 
> Best regards,
> —Adrián

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] ios-wk2 EWS in a bad state because of flaky tests

2019-09-29 Thread Aakash Jain
This queue is back in good state. Please let me know if you notice any issue.

Thanks
Aakash

> On Sep 27, 2019, at 9:21 PM, Aakash Jain  wrote:
> 
> Hi All,
> 
> Last week we updated the iOS bots from iOS 12 to iOS 13. Since then we have 
> been noticing a lot of flaky tests. Because of that ios-wk2 queue has been in 
> a bad state. Most of the time, it is unable to determine if the test failures 
> were introduced by the patch or were pre-existing, and so it keep re-testing 
> the patch. Because of this various patches keep waiting for long time.
> 
> Me and few bot watchers are looking into these flaky failures, but it might 
> take a while to get the queue back to a good state. Meanwhile, if anyone of 
> you has any recent bugs assigned to you related to flaky tests, it would be 
> nice to prioritize that.
> 
> Thanks
> Aakash
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] ios-wk2 EWS in a bad state because of flaky tests

2019-09-27 Thread Aakash Jain
Hi All,

Last week we updated the iOS bots from iOS 12 to iOS 13. Since then we have 
been noticing a lot of flaky tests. Because of that ios-wk2 queue has been in a 
bad state. Most of the time, it is unable to determine if the test failures 
were introduced by the patch or were pre-existing, and so it keep re-testing 
the patch. Because of this various patches keep waiting for long time.

Me and few bot watchers are looking into these flaky failures, but it might 
take a while to get the queue back to a good state. Meanwhile, if anyone of you 
has any recent bugs assigned to you related to flaky tests, it would be nice to 
prioritize that.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Planned EWS improvements

2019-09-27 Thread Aakash Jain
Thank You everyone (who responded) for the feedback.

I will prioritize the features some of you mentioned, especially adding a way 
to retry the patch.

Thanks
Aakash

> On Sep 27, 2019, at 9:27 AM, Konstantin Tokarev  wrote:
> 
> 
> 
> 27.09.2019, 08:15, "Simon Fraser" :
>>> On Sep 27, 2019, at 3:27 AM, Aakash Jain  wrote:
>>> 
>>> Hi Everyone,
>>> 
>>> I have been making number of improvements to EWS. I also have various 
>>> planned improvements to EWS. I wanted to reach out to you guys to see if 
>>> anyone
>> 
>> Not everyone on this list is a guy.
> 
> FWIW, plural form of "guy" is often used as gender-neutral vocative, see e.g. 
> [1,2]
> 
> [1] https://www.merriam-webster.com/dictionary/guy
> [2] https://dictionary.cambridge.org/us/dictionary/english/guy
> 
> 
> -- 
> Regards,
> Konstantin

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] WinCairo EWS

2019-09-27 Thread Aakash Jain
Do you think adding a build step before svn-apply which deletes “ 
C:\Users\ContainerAdministrator\AppData\Local\Temp” would help?

Thanks
Aakash

> On Sep 27, 2019, at 1:34 PM, ross.kirsl...@sony.com wrote:
> 
> 
> It provides every bit as much useful information as any other EWS bot when 
> it’s not running into that specific issue (`patch` failure). :)
>  
> Intermittent or not, it does seem like this is happening with shocking 
> frequency lately. I have no idea what causes it, but I hope we can quickly 
> find a better solution than manually rebooting the bot on a semi-daily basis.
>  
> Ross
>  
> From: webkit-dev  on behalf of Alex 
> Christensen 
> Date: Friday, September 27, 2019 at 7:09 AM
> To: "webkit-dev@lists.webkit.org" 
> Subject: [webkit-dev] WinCairo EWS
>  
> The WinCairo EWS bot has been in pretty sad shape for a while, providing no 
> useful information and lots of false negatives.  Could someone please fix it 
> or remove it?  See https://ews-build.webkit.org/#/builders/12/builds/5501 for 
> an example.
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Planned EWS improvements

2019-09-26 Thread Aakash Jain
Hi Everyone,

I have been making number of improvements to EWS. I also have various planned 
improvements to EWS. I wanted to reach out to you guys to see if anyone wants 
me to prioritize any particular improvement(s). If there is any improvement 
which you want to see and is not listed below, please feel free to let me know. 
Also most of the queues have been transitioned from old to new EWS and I am 
working on the remaining ones (jsc, windows and commit-queue).

Here is the list of improvements (in no particular order):

1) Develop a webpage showing summary of EWS builds for a patch. This page would 
provide the summary of important build-steps, high-level details about the 
failure (e.g.: name of the tests which failed, or possibly relevant build 
failure logs), and include link(s) to the Buildbot page(s). This page will open 
on clicking the status-bubbles (and would be replacement of old EWS status page 
like https://webkit-queues.webkit.org/patch/379563/win-ews 
). Currently clicking 
the status-bubble opens the Buildbot build page, which contains a lot of 
infrastructure details, and probably is information-overload for many 
engineers, so this summary page should help with that. 
https://webkit.org/b/197522 

2) Redesign status-bubble tooltip to include more detailed information about 
failures (e.g.: each test failure name along-with url to flakiness dashboard, 
and url to complete results.html file, as suggested by David Kilzer in 
https://lists.webkit.org/pipermail/webkit-dev/2019-September/030799.html 
). We 
should also add the tooltip support for iPad/iPhone https://webkit.org/b/201940 


3) Add a way to retry a patch in EWS. This would allow engineers to confirm 
that the failures indicated by EWS aren't flaky/incorrect. Maybe a good place 
to add the 'retry' button would be the status-bubble's tool-tip (visible only 
if the bubble is red) https://webkit.org/b/196599 

4) Parse the relevant build failure message from build logs (and display in 
summary page) https://webkit.org/b/201941 

5) Style failure should be displayed in-line on the review page along-with the 
code, just like the reviewer's comments https://webkit.org/b/202252 


6) Add more test-suites to EWS (e.g.: LLDB tests, resultsdbpy tests) 
https://webkit.org/b/189206 , 
https://webkit.org/b/201928 

7) Add commit-queue support for security bugs https://webkit.org/b/201939 


8) API tests should upload crashlogs https://webkit.org/b/201929 


Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Requesting feedback about EWS comments on Bugzilla bugs

2019-09-25 Thread Aakash Jain
Yeah, we can re-design status-bubble tooltip (https://webkit.org/b/201940) and 
include such information there.

Another idea to help with making style failures more noticeable is to have the 
Style failure displayed in-line on the review page along-with the reviewer's 
comments (https://webkit.org/b/202252).

-Aakash

> On Sep 19, 2019, at 1:41 PM, David Kilzer  wrote:
> 
> I actually “missed” output from the stylebot insofar as I was expecting the 
> errors to be posted, and then I was going to reply to that automated message.
> 
> I think it’s too easy to ignore style errors if you have to click to load 
> another website, though, so I filed this bug with some thoughts:
> 
> Bug 201993: EWS style bot should show error output via semi-modal dialog 
> rather than requiring to click through
> <https://bugs.webkit.org/show_bug.cgi?id=201993 
> <https://bugs.webkit.org/show_bug.cgi?id=201993>>
> 
> Maybe we could do something similar for layout test failures (a semi-modal 
> dialog that lists the errors, and links directly to the results rather than 
> having to click (twice) through the buildbot landing page?
> 
> Dave
> 
> 
> On Sep 19, 2019, at 9:45 AM, Aakash Jain  <mailto:aakash_j...@apple.com>> wrote:
> 
>> On Jun 16, 2019, at 2:14 PM, Darin Adler > <mailto:da...@apple.com>> wrote:
>> 
>>> On Jun 15, 2019, at 9:13 PM, Aakash Jain >> <mailto:aakash_j...@apple.com>> wrote:
>>> 
>>>> 1) Do not upload archive (for layout-test-results) on bugzilla, instead 
>>>> upload it to another server, unzip it and post a link to the results.html.
>>>> Pros:
>>>> a) Engineers won't have to download the attachment, unzip it, look for 
>>>> failures, and then delete it from their disk. They can simply click the 
>>>> url to view the results. 
>>>> b) This approach will also reduce 2 comments per failure to 1 comment. 
>>>> Currently there are two comments per failure, one for failure details, 
>>>> second for bugzilla attachment.
>>> 
>>> Great improvement to do this.
>> 
>> We have implemented this in the new EWS. Layout test results are no longer 
>> added to EWS as attachments. Instead they are available to view in browser 
>> or download from the Buildbot build page.
>> 
>>> The most confusing thing about build bot comments is all the “creation of 
>>> attachments” extra text with things like “attachment number” and “patch".
>>> 
>>> However, it’s really nice that I can download a directory full of test 
>>> results easily. I’d like to see the EWS website still have that feature.
>>> 
>>>> 4) When a patch becomes 'obsolete', tag the corresponding EWS comments as 
>>>> 'obsolete', so that they will be hidden.
>>> 
>>> Incredibly valuable.
>>> 
>>>> 5) Do not comment on bugzilla bug at all
>>> 
>>> I think this makes sense. I don’t see a reason that test results need to be 
>>> comments. I think the “red bubble” in EWS already calls someone’s attention 
>>> to failures.
>>> 
>>> If we want to augment it, we should think of what we are aiming at. I do 
>>> find it useful to see which tests are failing, and when I click on the red 
>>> bubble I don’t see that information. I have to click once to see the “log 
>>> of activities” then click on “results”, then see a confusing giant file 
>>> with lots of other information. At the bottom of that file the one thing I 
>>> want to know.
>>> 
>>> A better hierarchy is to put that “what new tests are failing” summary 
>>> right t the top and let the logs be fallbacks, not the primary place to see 
>>> the features.
>>> 
>>>> instead send email to the author of the patch.
>>> 
>>> Why? I don’t think this should send any emails at all, unless the person 
>>> requested it.
>>> 
>>>> Pros: less noisy, also this will allow to include more detailed 
>>>> information about the failure in email.
>>> 
>>> I think the more detailed information should be on the webpage, not in an 
>>> email.
>>> 
>>>> Cons: reviewers would have to click status-bubbles to see the failures, 
>>>> failure information is not immediately present in the comments.
>>> 
>>> I think we should start with this approach, eliminating the comments 
>>> entirely.
>> 
>> Following this suggestion, we eliminated the comments entirely in the new 
>> EWS. However, some people mentioned that since there are no comments, they 
>> do not get email notifications on failure, e.g.: https://webkit.org/b/200399 
>> <https://webkit.org/b/200399>. Maybe we should have some kind of comments by 
>> EWS. Few ideas:
>> 
>> 1) Comment on first failure for a patch, e.g.: "Some failures were noticed 
>> by EWS, please check the status bubbles".
>> 
>> 2) Comment about success on all queues, e.g.: "Patch passed all EWS queues".
>> 
>> 
>> What do you guys think?
>> 
>>> 
>>> — Darin
>> 
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Requesting feedback about EWS comments on Bugzilla bugs

2019-09-19 Thread Aakash Jain


> On Jun 16, 2019, at 2:14 PM, Darin Adler  wrote:
> 
>> On Jun 15, 2019, at 9:13 PM, Aakash Jain > <mailto:aakash_j...@apple.com>> wrote:
>> 
>> 1) Do not upload archive (for layout-test-results) on bugzilla, instead 
>> upload it to another server, unzip it and post a link to the results.html.
>> Pros:
>> a) Engineers won't have to download the attachment, unzip it, look for 
>> failures, and then delete it from their disk. They can simply click the url 
>> to view the results. 
>> b) This approach will also reduce 2 comments per failure to 1 comment. 
>> Currently there are two comments per failure, one for failure details, 
>> second for bugzilla attachment.
> 
> Great improvement to do this.

We have implemented this in the new EWS. Layout test results are no longer 
added to EWS as attachments. Instead they are available to view in browser or 
download from the Buildbot build page.

> The most confusing thing about build bot comments is all the “creation of 
> attachments” extra text with things like “attachment number” and “patch".
> 
> However, it’s really nice that I can download a directory full of test 
> results easily. I’d like to see the EWS website still have that feature.
> 
>> 4) When a patch becomes 'obsolete', tag the corresponding EWS comments as 
>> 'obsolete', so that they will be hidden.
> 
> Incredibly valuable.
> 
>> 5) Do not comment on bugzilla bug at all
> 
> I think this makes sense. I don’t see a reason that test results need to be 
> comments. I think the “red bubble” in EWS already calls someone’s attention 
> to failures.
> 
> If we want to augment it, we should think of what we are aiming at. I do find 
> it useful to see which tests are failing, and when I click on the red bubble 
> I don’t see that information. I have to click once to see the “log of 
> activities” then click on “results”, then see a confusing giant file with 
> lots of other information. At the bottom of that file the one thing I want to 
> know.
> 
> A better hierarchy is to put that “what new tests are failing” summary right 
> t the top and let the logs be fallbacks, not the primary place to see the 
> features.
> 
>> instead send email to the author of the patch.
> 
> Why? I don’t think this should send any emails at all, unless the person 
> requested it.
> 
>> Pros: less noisy, also this will allow to include more detailed information 
>> about the failure in email.
> 
> I think the more detailed information should be on the webpage, not in an 
> email.
> 
>> Cons: reviewers would have to click status-bubbles to see the failures, 
>> failure information is not immediately present in the comments.
> 
> I think we should start with this approach, eliminating the comments entirely.

Following this suggestion, we eliminated the comments entirely in the new EWS. 
However, some people mentioned that since there are no comments, they do not 
get email notifications on failure, e.g.: https://webkit.org/b/200399 
<https://webkit.org/b/200399>. Maybe we should have some kind of comments by 
EWS. Few ideas:

1) Comment on first failure for a patch, e.g.: "Some failures were noticed by 
EWS, please check the status bubbles".

2) Comment about success on all queues, e.g.: "Patch passed all EWS queues".


What do you guys think?

> 
> — Darin

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Questions about JSC EWS queues

2019-09-19 Thread Aakash Jain



> On Sep 19, 2019, at 7:22 AM, Guillaume Emont  wrote:
> 
> Quoting Aakash Jain (2019-09-18 19:21:02)
>> Hi All,
>> 
>> I am working on moving JSC EWS queues from old EWS to new EWS. I am trying to
>> clearly understand various JSC EWS queues. I have few questions:
>> 
>> 1) What does 'jsc-only' port represent? From https://trac.webkit.org/browser/
>> webkit/trunk/Tools/Scripts/webkitpy/common/config/ews.json#L45 it seems like
>> Apple JSC EWS uses 'mac' port, while linux jsc EWSes use 'jsc-only' port. Is
>> jsc-only port specific to linux? Is there a corresponding jsc port for mac?
>> 
>> 2) Is there any difference between the three linux JSC EWSes (JSC i386, JSC
>> MIPS32el, JSC Armv7) apart from the architecture and few queues running/
>> skipping tests? 
> 
> I think the main difference is indeed the different architecture. There
> is also the fact that the JSC i386 EWS runs the tests natively, but the
> mips and armv7 EWSes cross-compile and run the tests on a set of remote
> devices.
> 
>> 
>> 3) Where is the architecture for various linux JSC EWS queues configured in 
>> the
>> old EWS code (e.g.: I don't see the architecture being added in https://
>> trac.webkit.org/changeset/237001/webkit)? Is the queue configuration for
>> various JSC EWS queues same, and the device connected to the queue actually
>> decides the architecture?
> 
> I guess it's not defined anywhere indeed. The scripts that start the
> queues for the arm and mips EWSes define a bunch of env variables to
> point to the cross-compiler and also add a few options in
> JSCTESTS_OPTIONS, such as a  --remote-config-file and --memory-limited.
> I can share these scripts with you if you need more details.

Thanks for the reply.

Yeah, I would need the details. It would be nice if you can share those scripts 
with me.

> 
> Hope this helps.
> 
> Cheers,
> 
> Guij
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Questions about JSC EWS queues

2019-09-18 Thread Aakash Jain
Hi All,

I am working on moving JSC EWS queues from old EWS to new EWS. I am trying to 
clearly understand various JSC EWS queues. I have few questions:

1) What does 'jsc-only' port represent? From 
https://trac.webkit.org/browser/webkit/trunk/Tools/Scripts/webkitpy/common/config/ews.json#L45
 

 it seems like Apple JSC EWS uses 'mac' port, while linux jsc EWSes use 
'jsc-only' port. Is jsc-only port specific to linux? Is there a corresponding 
jsc port for mac?

2) Is there any difference between the three linux JSC EWSes (JSC i386, JSC 
MIPS32el, JSC Armv7) apart from the architecture and few queues 
running/skipping tests? 

3) Where is the architecture for various linux JSC EWS queues configured in the 
old EWS code (e.g.: I don't see the architecture being added in 
https://trac.webkit.org/changeset/237001/webkit 
)? Is the queue configuration 
for various JSC EWS queues same, and the device connected to the queue actually 
decides the architecture?

Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Emojis for builder vs tester in EWS status-bubbles

2019-09-13 Thread Aakash Jain
Hi Everyone,

We had this long-standing issue with the EWS bubbles that many engineers were 
confused whether a queue was build-only, test-only or build-and-test.

For e.g.: https://webkit.org/b/152790
Simon Fraser 2016-01-06: "From the EWS bubbles you can't tell if a bot is a 
build-only bot, or a build-and-test bot. All this time I've been assuming that 
GTK was running tests, and that assumption was wrong."


To solve that, we recently added emojis in the status-bubbles. Builder queues 
will only have builder emoji, tester queues will only have tester emoji, and 
build-and-test queues will have both builder and tester emojis. Also, we have 
separated most of the build-and-test queues into separate builder queues and 
tester queues. These changes should help in clarifying which queue does what. 
The emojis we have used are:

Builder: ️ (https://emojipedia.org/hammer-and-wrench/)

Tester:  (https://emojipedia.org/microscope/)

The emoji for tester doesn't feel very appropriate, and I am curious to know if 
anyone has a better suggestion. I previously used 離 
(https://emojipedia.org/test-tube/), but this emoji was added to Unicode last 
year only, and older OSes don't have this emoji. So I switched to the current 
(microscope) emoji in https://webkit.org/b/201532.

Please let me know if anyone has suggestions for better emojis. If not, atleast 
this email would let people know the purpose of these emojis.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] EWS Watchlist

2019-09-10 Thread Aakash Jain
Hi Everyone,

For those who don't know we have a cool little feature named watchlist, which 
allows you to monitor any particular code in WebKit repository. You can specify 
the files/folders to monitor (in 
https://trac.webkit.org/browser/webkit/trunk/Tools/Scripts/webkitpy/common/config/watchlist
 
),
 and whenever any patch modifies those, EWS would cc you on that bug.

Watchlist was partially broken for a while (because of containing incorrect 
email addresses). I just fixed it in 
https://bugs.webkit.org/show_bug.cgi?id=201433 
. So, now you might see EWS 
cc'ing bunch of people, more frequently than previously.

Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Recent EWS improvements

2019-09-04 Thread Aakash Jain


> On Sep 4, 2019, at 11:49 am, Adrian Perez de Castro  wrote:
> 
> Hello Aakash, WebKittens,
> 
> On Thu, 29 Aug 2019 18:01:56 -0400, Aakash Jain  wrote:
> 
>> I want to update everyone with the further improvements I have made to new
>> EWS. As always, please feel free to provide any feedback (either by filing
>> bugs or contacting me directly). 
>> 
>> New Features:
>> 
>> [...]
> 
> Thanks a ton for all the improvements in the EWS! Since we moved the GTK/EWS
> builders to the new queues they have been faster and more reliable. After
> solving a couple of minor hiccups right after the switch, the builders have
> not needed any manual intervention since (with the old system we would need
> to clean stray files from old builds now and then, but not anymore!)

Glad to hear. New EWS has been more reliable and easier to maintain at our end 
as well.

> 
> \o/
> 
>> Only remaining queues on old EWS are windows, jsc and commit-queue, which I
>> will be working on next.
> 
> Today Xan López and I talked a bit about the JSCOnly queue. We have a many
> builders targeting different architectures, which we would like to have
> moved to the new system.
> 
> In the case of the JSC port, instead of grouping all of the builders under a
> single “jsc” status bubble it would be better to have a bubble for each target
> architecture e.g. “jsc-armv7”, “jsc-arm64”, and so on.

Yes, my plan is to have separate bubbles for different architectures. I will 
touch base with you when I work on those queues.

> 
> The motivation is that we think it makes sense to that the system considers
> that a patch cannot be landed if it breaks any of the supported JSC
> architectures. With a single status bubble grouping them, we can have
> situations where a builder for one architecture passes the build+checks
> but the patch may still break *some other* architecture (e.g. a patch
> that touches ARM code generation gets picked by a MIPS builder).
> 
> Is the above something that can supported? Let us know if you need more
> information and/or some support from our side for this.
> 
> Best regards,
> —Adrián

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Recent EWS improvements

2019-08-29 Thread Aakash Jain
Hi Everyone,

I want to update everyone with the further improvements I have made to new EWS. 
As always, please feel free to provide any feedback (either by filing bugs or 
contacting me directly). 

New Features:

- Moved following queues to from old to new EWS:
• iOS 12 Tests EWS
• macOS WK1 and WK2 Release Tests EWS
• macOS WK1 Debug Tests EWS
• GTK and WPE EWS
• WinCairo EWS
• Style (and watchlist) EWS

- Added a new 'services' EWS to run Buildbot unit tests 
https://webkit.org/b/14

- Separate status bubbles for builders and layout-testers


Infrastructure Improvements:

- EWS is now able to automatically recover (and retry the build) in case of 
certain git and network errors https://webkit.org/b/199722
- Some EWS bots now reboot periodically to ensure that start from clean state
- Added KillOldProcesses step before running API or Layout tests to increase 
robustness https://webkit.org/b/199592
- 'view layout test results' option is now displayed next to layout-test build 
step https://webkit.org/b/200048
- EWS now displays pre-existing Layout test failure names in the build summary 
https://webkit.org/b/199941
- Machine uptime is now reported in PrintConfiguration step 
https://webkit.org/b/200812
- Added EWS Buildbot support to Internal scripts to easily find/reconfigure bots
- Flakiness in API tests has been further reduced (thanks to multiple fixes by 
WebKit developers)
- Added more unit tests


Bug fixes:

- EWS status-bubbles are sometimes multi-row with scroll-bar 
https://webkit.org/b/199939
- EWS: Cannot see build status page when patch is waiting for tester 
https://webkit.org/b/200333
- EWS: Do not append additional '(failure)' string at the end of custom failure 
message in EWS Buildbot https://webkit.org/b/201140
- [ews] Status bubble should display only important messages in pop-over 
https://webkit.org/b/201308


Only remaining queues on old EWS are windows, jsc and commit-queue, which I 
will be working on next.

Thanks
Aakash

> On Jul 11, 2019, at 10:18 am, Aakash Jain  wrote:
> 
> Hi Everyone,
> 
> I want to update everyone with the further improvements I have made to new 
> EWS. As always, please feel encouraged to provide any feedback (either by 
> filing bugs or contacting me directly). 
> 
> New Features:
> - Launched 'Security EWS'
> - Added iOS-12 Builder queue on new EWS (moved from old to new EWS)
> - Added WPE and GTK queues on new EWS (moved from old to new EWS)
> - New EWS can now process large patches (larger than 640kb) 
> https://webkit.org/b/198851
> 
> 
> Infrastructure Improvements:
> - EWS now automatically retries builds in case of certain infrastructure 
> issues and ToT compile failures https://bugs.webkit.org/show_bug.cgi?id=199025
> - Improved stability by adding KillOldProcesses step before API and Layout 
> tests https://webkit.org/b/199592
> - Shared bots across queues to improve efficiency https://webkit.org/b/198370
> - Added check for duplicate workers in config.json https://webkit.org/b/199240
> - Patch link now opens the pretty-patch url https://webkit.org/b/199031
> - Triggered builds should use same revision as parent build (fixes a corner 
> case) https://webkit.org/b/198289
> - Display pre-existing API test failures in build summary to help 
> bot-watchers https://webkit.org/b/199525
> - Send email notifications for failures (for maintenance/bot-watching) 
> https://webkit.org/b/198919
> - Remove unused buildbot tabs for better readability 
> https://webkit.org/b/198108
> - Allow skipping uploading built product for few builders 
> https://webkit.org/b/199422
> - EWS should provide option to download layout test results zip file 
> https://webkit.org/b/199121
> - Retry Layout test in case of failures https://webkit.org/b/199194
> - Upload test results after running layout-tests https://webkit.org/b/199120
> - Improved error message on performing certain actions (like Rebuild) which 
> requires authentication
> - Added more unit-tests
> 
> 
> Bug fixes:
> - Results are clobbered in UploadTestResults and ExtractTestResults steps in 
> case of multiple layout test runs https://webkit.org/b/199178
> - New EWS: api-ios does not differentiate between patch compile failure and 
> ToT compile failure https://webkit.org/b/197850
> - Buildbot worker not pinged https://webkit.org/b/199438
> - Make PrintConfiguration platform aware https://webkit.org/b/196657
> - Status bubble should not turn orange when any build step is skipped 
> https://webkit.org/b/199079
> - Do not print worker environment variables in each build step 
> https://webkit.org/b/197319
> - Do not run unix commands for windows in PrintConfiguration 
> https://webkit.org/b/199605
> 
> Thanks
> Aakash
> 
>> On May 22, 2019, at 7:36 PM, Aakash Jain  wro

Re: [webkit-dev] EWS bubbles not loading for some users

2019-08-22 Thread Aakash Jain
The issue has been resolved. Please let me know if anyone still notice any 
issue with EWS.

Thanks
Aakash

> On Aug 22, 2019, at 12:57 pm, Aakash Jain  wrote:
> 
> Hi Everyone,
> 
> There were some DNS changes done yesterday for webkit.org 
> <http://webkit.org/>. It seems to be causing EWS bubbles not to load for some 
> users. For those users, https://ews.webkit.org <https://ews.webkit.org/> is 
> redirecting to bugs.webkit.org <http://bugs.webkit.org/> instead of 
> displaying 'EWS for WebKit.' This issue seems to be affecting new EWS, but 
> not the old EWS (so 'style', 'jsc' and 'win' bubbles might still show up).
> 
> We are actively looking into it.
> 
> Thanks
> Aakash

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] EWS bubbles not loading for some users

2019-08-22 Thread Aakash Jain
Hi Everyone,

There were some DNS changes done yesterday for webkit.org . 
It seems to be causing EWS bubbles not to load for some users. For those users, 
https://ews.webkit.org  is redirecting to 
bugs.webkit.org  instead of displaying 'EWS for 
WebKit.' This issue seems to be affecting new EWS, but not the old EWS (so 
'style', 'jsc' and 'win' bubbles might still show up).

We are actively looking into it.

Thanks
Aakash___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Recent EWS improvements

2019-07-11 Thread Aakash Jain
Hi Everyone,

I want to update everyone with the further improvements I have made to new EWS. 
As always, please feel encouraged to provide any feedback (either by filing 
bugs or contacting me directly). 

New Features:
- Launched 'Security EWS'
- Added iOS-12 Builder queue on new EWS (moved from old to new EWS)
- Added WPE and GTK queues on new EWS (moved from old to new EWS)
- New EWS can now process large patches (larger than 640kb) 
https://webkit.org/b/198851 <https://webkit.org/b/198851>


Infrastructure Improvements:
- EWS now automatically retries builds in case of certain infrastructure issues 
and ToT compile failures https://bugs.webkit.org/show_bug.cgi?id=199025 
<https://bugs.webkit.org/show_bug.cgi?id=199025>
- Improved stability by adding KillOldProcesses step before API and Layout 
tests https://webkit.org/b/199592 <https://webkit.org/b/199592>
- Shared bots across queues to improve efficiency https://webkit.org/b/198370 
<https://webkit.org/b/198370>
- Added check for duplicate workers in config.json https://webkit.org/b/199240 
<https://webkit.org/b/199240>
- Patch link now opens the pretty-patch url https://webkit.org/b/199031 
<https://webkit.org/b/199031>
- Triggered builds should use same revision as parent build (fixes a corner 
case) https://webkit.org/b/198289 <https://webkit.org/b/198289>
- Display pre-existing API test failures in build summary to help bot-watchers 
https://webkit.org/b/199525 <https://webkit.org/b/199525>
- Send email notifications for failures (for maintenance/bot-watching) 
https://webkit.org/b/198919 <https://webkit.org/b/198919>
- Remove unused buildbot tabs for better readability 
https://webkit.org/b/198108 <https://webkit.org/b/198108>
- Allow skipping uploading built product for few builders 
https://webkit.org/b/199422 <https://webkit.org/b/199422>
- EWS should provide option to download layout test results zip file 
https://webkit.org/b/199121 <https://webkit.org/b/199121>
- Retry Layout test in case of failures https://webkit.org/b/199194 
<https://webkit.org/b/199194>
- Upload test results after running layout-tests https://webkit.org/b/199120 
<https://webkit.org/b/199120>
- Improved error message on performing certain actions (like Rebuild) which 
requires authentication
- Added more unit-tests


Bug fixes:
- Results are clobbered in UploadTestResults and ExtractTestResults steps in 
case of multiple layout test runs https://webkit.org/b/199178 
<https://webkit.org/b/199178>
- New EWS: api-ios does not differentiate between patch compile failure and ToT 
compile failure https://webkit.org/b/197850 <https://webkit.org/b/197850>
- Buildbot worker not pinged https://webkit.org/b/199438 
<https://webkit.org/b/199438>
- Make PrintConfiguration platform aware https://webkit.org/b/196657 
<https://webkit.org/b/196657>
- Status bubble should not turn orange when any build step is skipped 
https://webkit.org/b/199079 <https://webkit.org/b/199079>
- Do not print worker environment variables in each build step 
https://webkit.org/b/197319 <https://webkit.org/b/197319>
- Do not run unix commands for windows in PrintConfiguration 
https://webkit.org/b/199605 <https://webkit.org/b/199605>

Thanks
Aakash

> On May 22, 2019, at 7:36 PM, Aakash Jain  wrote:
> 
> Hi Everyone,
> 
> I just wanted to update everyone with the recent improvements I have made to 
> new EWS. As always, please feel encouraged to provide any feedback (either by 
> filing bugs or contacting me directly).
> 
> New Features:
> EWS status-bubble now display position in queue while patch is waiting to be 
> processed
> Added webkitpy and bindings-tests EWS (moved from old to new EWS)
> Status bubbles for webkitpy and bindings-tests EWS now display the exact test 
> failures in hover-over message (https://webkit.org/b/197395 
> <https://webkit.org/b/197395>, https://webkit.org/b/197423 
> <https://webkit.org/b/197423>)
> Added support for 'new EWS' in webkit-patch tool
> Added 'EWS Build Archives' (similar to 'WebKit Build Archives' 
> https://webkit.org/blog/7978/introducing-webkit-build-archives/ 
> <https://webkit.org/blog/7978/introducing-webkit-build-archives/>). For every 
> patch uploaded to Bugzilla, EWS builders build the patch for various 
> platforms (currently macOS and iOS) and upload the archives to S3. These 
> archives are available to download by anyone (for 14 days). The S3 URL is in 
> corresponding build (e.g.: notice 'uploaded archive' link in 
> https://ews-build.webkit.org/#/builders/7/builds/2477 
> <https://ews-build.webkit.org/#/builders/7/builds/2477>). So, if for any 
> reason, you want to get a built archive for your patch, you can simply upload 
> the patch to Bugzilla. (Note that if there is interest in this, we can 
&g

[webkit-dev] wincairo EWS bot down?

2019-06-24 Thread Aakash Jain
wincairo-ews-001 bot seems to be stuck/down for last 2 days 
(https://webkit-queues.webkit.org/queue-status/wincairo-ews 
). There are ~43 
patch pending currently. Can someone take a look?

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Requesting feedback about EWS comments on Bugzilla bugs

2019-06-18 Thread Aakash Jain


> On Jun 16, 2019, at 2:14 PM, Darin Adler  wrote:
> 
>> On Jun 15, 2019, at 9:13 PM, Aakash Jain > <mailto:aakash_j...@apple.com>> wrote:
>> 
>> 1) Do not upload archive (for layout-test-results) on bugzilla, instead 
>> upload it to another server, unzip it and post a link to the results.html.
>> Pros:
>> a) Engineers won't have to download the attachment, unzip it, look for 
>> failures, and then delete it from their disk. They can simply click the url 
>> to view the results. 
>> b) This approach will also reduce 2 comments per failure to 1 comment. 
>> Currently there are two comments per failure, one for failure details, 
>> second for bugzilla attachment.
> 
> Great improvement to do this. The most confusing thing about build bot 
> comments is all the “creation of attachments” extra text with things like 
> “attachment number” and “patch".
> 
> However, it’s really nice that I can download a directory full of test 
> results easily. I’d like to see the EWS website still have that feature.

Sure, will keep an option to download the test results archive as well.

> 
>> 4) When a patch becomes 'obsolete', tag the corresponding EWS comments as 
>> 'obsolete', so that they will be hidden.
> 
> Incredibly valuable.
> 
>> 5) Do not comment on bugzilla bug at all
> 
> I think this makes sense. I don’t see a reason that test results need to be 
> comments. I think the “red bubble” in EWS already calls someone’s attention 
> to failures.

What do you think about comments for 'Style' failures, is that ok to keep, or 
should we remove them as well?

> If we want to augment it, we should think of what we are aiming at. I do find 
> it useful to see which tests are failing, and when I click on the red bubble 
> I don’t see that information. I have to click once to see the “log of 
> activities” then click on “results”, then see a confusing giant file with 
> lots of other information. At the bottom of that file the one thing I want to 
> know.
> 
> A better hierarchy is to put that “what new tests are failing” summary right 
> t the top and let the logs be fallbacks, not the primary place to see the 
> features.

This brings another interesting question, what page should be displayed on 
clicking an EWS bubble?

a) old EWS style page, e.g.: 
https://webkit-queues.webkit.org/patch/362845/ios-sim-ews 
<https://webkit-queues.webkit.org/patch/362845/ios-sim-ews>. This does have 
deficiencies like you mentioned.

b) Buildbot page, e.g.: https://ews-build.webkit.org/#/builders/3/builds/414 
<https://ews-build.webkit.org/#/builders/3/builds/414>, currently this is new 
EWS behavior (but this doesn't cover the case when there are multiple builds 
(e.g.: retries) for a patch on a given queue).

c) Newly designed page which shows the summary of failures, have link(s) to the 
Buildbot page(s), have link to download test result archives, and maybe summary 
of major build steps which were executed. More feedback regarding the design of 
this page would be useful.

> 
>> instead send email to the author of the patch.
> 
> Why? I don’t think this should send any emails at all, unless the person 
> requested it.

ok

> 
>> Pros: less noisy, also this will allow to include more detailed information 
>> about the failure in email.
> 
> I think the more detailed information should be on the webpage, not in an 
> email.
> 
>> Cons: reviewers would have to click status-bubbles to see the failures, 
>> failure information is not immediately present in the comments.
> 
> I think we should start with this approach, eliminating the comments entirely.

Sure, let's do this. I wouldn't change the old EWS (which is being replaced), 
but new EWS will have this behavior.

> 
> — Darin

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Requesting feedback about EWS comments on Bugzilla bugs

2019-06-18 Thread Aakash Jain


> On Jun 17, 2019, at 1:52 PM, Keith Rollin  wrote:
> 
>> On Jun 16, 2019, at 11:14, Darin Adler  wrote:
>> 
>> If we want to augment it, we should think of what we are aiming at. I do 
>> find it useful to see which tests are failing, and when I click on the red 
>> bubble I don’t see that information. I have to click once to see the “log of 
>> activities” then click on “results”, then see a confusing giant file with 
>> lots of other information. At the bottom of that file the one thing I want 
>> to know.
> 
> We might want to also start turning those failure into action items. We could 
> have an automatic mechanism that gathers the failures, records them in a 
> database, and then — with sufficient data — makes determinations about the 
> flakiness or other status of the test. It could then mark the test as flaky 
> or raise it as an issue to some responsible (and responsive) party.

Agree. This is the plan. Jonathan Bedard is working on an improved flakiness 
dashboard. Once we have that, EWS will start using it's API to get the test 
flakiness information. That should significantly reduce EWS's false positives 
(and also reduce the number of retries EWS has to do while trying to rule out 
flakiness).

> 
> We could also have a relatively manual process. The failures are surfaced in 
> Bugzilla or in a Bugzilla-accessible page. The engineer posting the patch 
> could then review the failures and mark them as “Flag as flaky”, “Flag as 
> failing and should be fixed by someone else”, “Flag as failing and should be 
> ignored”, etc. These responses could then be turned into action items for 
> some responsible (and responsive) party to address.
> 
> As Michael says, there’s a big issue with ignoring test results. Putting a 
> frictionless process in place to address test results would help make them 
> more effective. When I make a change to an Xcode project and Windows builds 
> throw up errors, that’s not something caused by my immediate patch, but I 
> would like to see the flaky test fixed.
> 
> — Keith
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Requesting feedback about EWS comments on Bugzilla bugs

2019-06-15 Thread Aakash Jain
Hi Everyone,

I am gathering feedback about EWS - specially about the comments which EWS 
makes on the Bugzilla bugs. Currently, the comments are not very 
user-friendly/polished/readable and are sometimes very noisy (e.g.: 72 comments 
and 36 attachments by EWS in https://bugs.webkit.org/show_bug.cgi?id=177484 
). I am working on improving 
them and looking for specific ideas/feedback.

Few ideas which I am considering:

1) Do not upload archive (for layout-test-results) on bugzilla, instead upload 
it to another server, unzip it and post a link to the results.html.
Pros:
a) Engineers won't have to download the attachment, unzip it, look for 
failures, and then delete it from their disk. They can simply click the url to 
view the results. 
b) This approach will also reduce 2 comments per failure to 1 comment. 
Currently there are two comments per failure, one for failure details, second 
for bugzilla attachment.

2) Aggregate comments from multiple queues.
Pros: less noise
Cons: comments would get delayed while waiting for results from other queues. 
(Also might be little complex to implement)

3) Improve the text of the comments to make them more readable (specific ideas 
are welcome).

4) When a patch becomes 'obsolete', tag the corresponding EWS comments as 
'obsolete', so that they will be hidden.

5) Do not comment on bugzilla bug at all, instead send email to the author of 
the patch.
Pros: less noisy, also this will allow to include more detailed information 
about the failure in email.
Cons: reviewers would have to click status-bubbles to see the failures, failure 
information is not immediately present in the comments.

What do you guys think?

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Recent EWS improvements

2019-05-22 Thread Aakash Jain
Hi Everyone,

I just wanted to update everyone with the recent improvements I have made to 
new EWS. As always, please feel encouraged to provide any feedback (either by 
filing bugs or contacting me directly).

New Features:
EWS status-bubble now display position in queue while patch is waiting to be 
processed
Added webkitpy and bindings-tests EWS (moved from old to new EWS)
Status bubbles for webkitpy and bindings-tests EWS now display the exact test 
failures in hover-over message (https://webkit.org/b/197395 
<https://webkit.org/b/197395>, https://webkit.org/b/197423 
<https://webkit.org/b/197423>)
Added support for 'new EWS' in webkit-patch tool
Added 'EWS Build Archives' (similar to 'WebKit Build Archives' 
https://webkit.org/blog/7978/introducing-webkit-build-archives/ 
<https://webkit.org/blog/7978/introducing-webkit-build-archives/>). For every 
patch uploaded to Bugzilla, EWS builders build the patch for various platforms 
(currently macOS and iOS) and upload the archives to S3. These archives are 
available to download by anyone (for 14 days). The S3 URL is in corresponding 
build (e.g.: notice 'uploaded archive' link in 
https://ews-build.webkit.org/#/builders/7/builds/2477 
<https://ews-build.webkit.org/#/builders/7/builds/2477>). So, if for any 
reason, you want to get a built archive for your patch, you can simply upload 
the patch to Bugzilla. (Note that if there is interest in this, we can enhance 
it further)

Infrastructure Improvements:
Flakiness in API tests has been reduced (thanks to many WebKit developers)
Infrastructure improvements to prevent build failure due to "worker not pinged" 
(e.g.: https://ews-build.webkit.org/#/builders/9/builds/332 
<https://ews-build.webkit.org/#/builders/9/builds/332>)
New EWS polls bugzilla more frequently https://webkit.org/b/197138 
<https://webkit.org/b/197138>
Configured DEBUG mode appropriately for Production and Development env 
https://webkit.org/b/197700 <https://webkit.org/b/197700>
Ensured that Buildbot worker logs are not lost on restarting worker
Do not run clean build by default on EWS builders (to improve efficiency) 
https://webkit.org/b/196897 <https://webkit.org/b/196897>
build.webkit.org <http://build.webkit.org/> and ews-build.webkit.org 
<http://ews-build.webkit.org/> starting sharing code (although very little as 
of now, however the plan is to share more code)
Added migrations file to repository https://webkit.org/b/197729 
<https://webkit.org/b/197729>
Added EWS bots information to Internal scripts to easily monitor bots
Added more unit-tests

Bug fixes:
Clicking 'submit to new ews' doesn't reload status-bubble 
https://webkit.org/b/196675 <https://webkit.org/b/196675>
Clicking on white bubble navigates to page with only bubbles 
https://webkit.org/b/197520 <https://webkit.org/b/197520>
Submit to EWS buttons are not aligned properly with status-bubbles 
https://webkit.org/b/197139 <https://webkit.org/b/197139>
Status bubble should turn orange when any build step fails 
https://webkit.org/b/197812 <https://webkit.org/b/197812>
Handle bug titles with unicode characters https://webkit.org/b/196802 
<https://webkit.org/b/196802>
Scripts using Buildbot API have CORS error https://webkit.org/b/196709 
<https://webkit.org/b/196709>
PrintConfiguration should display Xcode version instead of SDKVersion 
https://webkit.org/b/196780 <https://webkit.org/b/196780>
Trigger queues only after uploading the archive https://webkit.org/b/197180 
<https://webkit.org/b/197180>
Do not upload archive when Compile Fails https://webkit.org/b/196674 
<https://webkit.org/b/196674>
Exception while loading status-bubble when no build step has started 
https://webkit.org/b/196676 <https://webkit.org/b/196676>
Use singular verb in failure description in case of single api test failure 
https://webkit.org/b/197013 <https://webkit.org/b/197013>
EWS should clearly indicate flaky test failures https://webkit.org/b/196947 
<https://webkit.org/b/196947>
Use explicit imports instead of wildcard imports https://webkit.org/b/197194 
<https://webkit.org/b/197194>
New EWS: patches on recently added queues listed as #1 for older bugs 
https://webkit.org/b/197496 <https://webkit.org/b/197496>
Improved summary text for various build steps

Interesting info: Since last month, 'EWS for API tests' prevented API test 
breakage on 50+ patches 
(https://ews-build.webkit.org/api/v2/builders/3/builds?state_string__contains=new%20API%20Test=bug_id=-number
 
<https://ews-build.webkit.org/api/v2/builders/3/builds?state_string__contains=new%20API%20Test=bug_id=-number>).

Thanks
Aakash

> On Apr 4, 2019, at 10:00 PM, Aakash Jain  wrote:
> 
> Introducing brand new EWS
> 
> with EWS for API Tests (for macOS and iOS)
> 
> and EWS for WebKitPerl Tests!
> 
> 
> Starting today, whe

Re: [webkit-dev] Introducing brand new EWS

2019-04-11 Thread Aakash Jain
Hi Ryosuke,

Sorry for the delay. I will soon be adding the feature to display patch's 
position in queue in https://bugs.webkit.org/show_bug.cgi?id=196607 
<https://bugs.webkit.org/show_bug.cgi?id=196607>. Would that address your 
concern?

Thanks
Aakash

> On Apr 4, 2019, at 10:09 PM, Ryosuke Niwa  wrote:
> 
> The new bubbles seem to be working well, and having API tests running in EWS 
> is great!
> 
> Can we add the waterfall view to https://ews-build.webkit.org/#/builders 
> <https://ews-build.webkit.org/#/builders> so that we can see where our 
> patches are in the queue?
> 
> - R. Niwa
> 
> On Thu, Apr 4, 2019 at 6:59 PM Aakash Jain  <mailto:aakash_j...@apple.com>> wrote:
> Introducing brand new EWS
> 
> with EWS for API Tests (for macOS and iOS)
> 
> and EWS for WebKitPerl Tests!
> 
> 
> Starting today, when you upload a patch to bugs.webkit.org 
> <http://bugs.webkit.org/>, you will see few more bubbles (for API tests and 
> webkitperl tests). You might also see additional button 'Submit to new EWS' 
> (if the patch doesn't have r? flag).
> 
> The new EWS comes with many new features and there are lot more I want to 
> add. But, I don't want you guys to wait more to start getting the benefits. 
> That's why I am rolling it out in phases, starting with EWS for API Tests and 
> WebKitPerl Tests. These are tests which are not currently covered by the 
> existing EWS. Next step would be to move queues from existing EWS to new EWS 
> one by one, with the eventual goal of moving over everything to new EWS.
> 
> 
> Why new EWS?
> The existing EWS has certain architectural limitations. One of the prominent 
> limitation is that there is no concept of building and testing the patch on 
> different queues. If we have three queues: WK1 tests, WK2 tests and API 
> tests, all three queues would need to compile WebKit independently. So WebKit 
> would be compiled thrice instead of once. This is inefficient and thereby 
> require more hardware.
> 
> The new EWS has separate builder and tester queues. Builder queues build once 
> and upload the archive. Multiple tester queues download that same archive and 
> run tests on that. That way WebKit is compiled only once, and re-used on 
> multiple tester queues. This improves system efficiency and allows us to add 
> new test queues with substantially less hardware.
> 
> The new EWS uses Buildbot at the back-end, which is a production-level CI 
> system. It is easier to maintain and automatically provide various features 
> like historical build logs, real-time log streaming, easier bot management, 
> ability to retry a build etc. Plus, it’s a system most of you are already 
> familiar with (build.webkit.org <http://build.webkit.org/>).
> 
> 
> How can you contribute:
> If you are interested in contributing, the source code is located at:
> ews-build (Buildbot): Tools/BuildSlaveSupport/ews-build
> ews-app (web-app): Tools/BuildSlaveSupport/ews-app
> 
> Detailed instructions are at: 
> https://trac.webkit.org/wiki/EarlyWarningSystem#ContributingtoEarlyWarningSystem
>  
> <https://trac.webkit.org/wiki/EarlyWarningSystem#ContributingtoEarlyWarningSystem>
> 
> 
> Upcoming features:
> - status-bubble should display position in queue 
> https://bugs.webkit.org/show_bug.cgi?id=196607 
> <https://bugs.webkit.org/show_bug.cgi?id=196607>
> - EWS should comment on Bugzilla bugs about failures 
> https://bugs.webkit.org/show_bug.cgi?id=196598 
> <https://bugs.webkit.org/show_bug.cgi?id=196598>
> - EWS should have a way to retry a patch 
> https://bugs.webkit.org/show_bug.cgi?id=196599 
> <https://bugs.webkit.org/show_bug.cgi?id=196599>
> - Security EWS https://bugs.webkit.org/show_bug.cgi?id=196605 
> <https://bugs.webkit.org/show_bug.cgi?id=196605>
> 
> 
> If you notice any issue, please feel free to file bugs (and assign to me).
> 
> Thanks & Regards
> Aakash Jain
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
> https://lists.webkit.org/mailman/listinfo/webkit-dev 
> <https://lists.webkit.org/mailman/listinfo/webkit-dev>

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Introducing brand new EWS

2019-04-04 Thread Aakash Jain
Introducing brand new EWS

with EWS for API Tests (for macOS and iOS)

and EWS for WebKitPerl Tests!


Starting today, when you upload a patch to bugs.webkit.org 
, you will see few more bubbles (for API tests and 
webkitperl tests). You might also see additional button 'Submit to new EWS' (if 
the patch doesn't have r? flag).

The new EWS comes with many new features and there are lot more I want to add. 
But, I don't want you guys to wait more to start getting the benefits. That's 
why I am rolling it out in phases, starting with EWS for API Tests and 
WebKitPerl Tests. These are tests which are not currently covered by the 
existing EWS. Next step would be to move queues from existing EWS to new EWS 
one by one, with the eventual goal of moving over everything to new EWS.


Why new EWS?
The existing EWS has certain architectural limitations. One of the prominent 
limitation is that there is no concept of building and testing the patch on 
different queues. If we have three queues: WK1 tests, WK2 tests and API tests, 
all three queues would need to compile WebKit independently. So WebKit would be 
compiled thrice instead of once. This is inefficient and thereby require more 
hardware.

The new EWS has separate builder and tester queues. Builder queues build once 
and upload the archive. Multiple tester queues download that same archive and 
run tests on that. That way WebKit is compiled only once, and re-used on 
multiple tester queues. This improves system efficiency and allows us to add 
new test queues with substantially less hardware.

The new EWS uses Buildbot at the back-end, which is a production-level CI 
system. It is easier to maintain and automatically provide various features 
like historical build logs, real-time log streaming, easier bot management, 
ability to retry a build etc. Plus, it’s a system most of you are already 
familiar with (build.webkit.org ).


How can you contribute:
If you are interested in contributing, the source code is located at:
ews-build (Buildbot): Tools/BuildSlaveSupport/ews-build
ews-app (web-app): Tools/BuildSlaveSupport/ews-app

Detailed instructions are at: 
https://trac.webkit.org/wiki/EarlyWarningSystem#ContributingtoEarlyWarningSystem
 



Upcoming features:
- status-bubble should display position in queue 
https://bugs.webkit.org/show_bug.cgi?id=196607 

- EWS should comment on Bugzilla bugs about failures 
https://bugs.webkit.org/show_bug.cgi?id=196598 

- EWS should have a way to retry a patch 
https://bugs.webkit.org/show_bug.cgi?id=196599 

- Security EWS https://bugs.webkit.org/show_bug.cgi?id=196605 



If you notice any issue, please feel free to file bugs (and assign to me).

Thanks & Regards
Aakash Jain___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] iOS EWS bots sick?

2018-11-20 Thread Aakash Jain
I am having a look. Seems like an ongoing intermittent issue. Tracked in 
https://bugs.webkit.org/show_bug.cgi?id=191864 


Thanks
Aakash

> On Nov 19, 2018, at 10:39 PM, Ryosuke Niwa  wrote:
> 
> e.g.
> 
> https://webkit-queues.webkit.org/results/10084446 
> 
> RuntimeError raised: Timed out while waiting for 
> F856DA6C-99A9-4F53-A577-FF6214181E63 to shut down
> Traceback (most recent call last):
>   File 
> "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py",
>  line 85, in main
> run_details = run(port, options, args, stderr)
>   File 
> "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py",
>  line 448, in run
> run_details = manager.run(args)
>   File 
> "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py",
>  line 249, in run
> initial_results, retry_results, enabled_pixel_tests_in_retry = 
> self._run_test_subset(default_device_tests, tests_to_skip)
>   File 
> "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py",
>  line 293, in _run_test_subset
> self._clean_up_run()
>   File 
> "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py",
>  line 342, in _clean_up_run
> self._port.clean_up_test_run()
>   File 
> "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/port/ios_simulator.py", line 
> 103, in clean_up_test_run
> SimulatedDeviceManager.tear_down(self.host)
>   File 
> "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/xcode/simulated_device.py", 
> line 444, in tear_down
> device.platform_device._tear_down(deadline - time.time())
>   File 
> "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/xcode/simulated_device.py", 
> line 538, in _tear_down
> self._shut_down(deadline - time.time())
>   File 
> "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/xcode/simulated_device.py", 
> line 524, in _shut_down
> raise RuntimeError('Timed out while waiting for {} to shut 
> down'.format(self.udid))
> RuntimeError: Timed out while waiting for 
> F856DA6C-99A9-4F53-A577-FF6214181E63 to shut down
> Stopping HTTP server ...
> Stopping WebSocket server ...
> Stopping Web Platform Test server ...
> 
> - R. Niwa
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] WinCairo EWS bots down

2018-11-19 Thread Aakash Jain
Hi Don, Sony Folks,

WinCairo bots seems to be down for last 2 days. Can someone please have a look?

https://webkit-queues.webkit.org/queue-status/wincairo-ews 


Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Buildbot upgrade on build.webkit.org

2018-01-08 Thread Aakash Jain
Hi Everyone,

I filed the Buildbot bug regarding Waterfall view not displaying steps 
information by default: https://github.com/buildbot/buildbot/issues/3884 
"Waterfall view should display steps information without hovering over".

I got the response that the issue with displaying all the steps information by 
default in waterfall view is that it would require a lot of API calls, and so 
it wouldn't be scalable.

Can you guys look at the new console and grid view to see if they can serve as 
an alternative?
e.g.:
https://nine.buildbot.net/#/console
https://nine.buildbot.net/#/grid


Here is the complete response:

"I agree the new waterfall is not very useful.
It is not really a problem of UI change, it is rather a problem of the number 
of rest API needed to get the information needed. In order to display the old 
waterfall, you need to get the whole details for every builds (steps, logs), 
that would be dozens of REST api calls, which would take a lot of time.

The old waterfall was not very scalable too, on my old buildbot eight at work, 
we disabled it, because it was stalling the the master for 10s every time one 
user would look at it.

The question you have to ask your users is what is the information that is 
really needed.
Maybe we can load the steps only for the failed builds?

We find the console_view and grid_view much more useful? They don't have 
details of which step failed, but the details of which builder is failing 
sorted by change, and usually people have been doing one builder per "kind of 
test"

You can see the example here, and the categorisation of builders, which I think 
could be quite useful as a replacement for https://build.webkit.org/waterfall

https://nine.buildbot.net/#/console

What user might request is probably the reason for failure, which can be done 
without having to fetch more information (by using the build results_string)
In the current version you can still click on a build, and have all the 
information for that build, including tail log of the failing steps (without 
changing web page)."


-Aakash


> On Dec 13, 2017, at 6:51 PM, Aakash Jain <aakash_j...@apple.com> wrote:
> 
> 
> 
>> On Dec 11, 2017, at 1:10 AM, Konstantin Tokarev <annu...@yandex.ru 
>> <mailto:annu...@yandex.ru>> wrote:
>> 
>> 
>> 
>> 11.12.2017, 02:49, "Carlos Alberto Lopez Perez" <clo...@igalia.com 
>> <mailto:clo...@igalia.com>>:
>>> On 07/12/17 21:47, Aakash Jain wrote:
>>>>  For people using build.webkit.org <http://build.webkit.org/> 
>>>> <http://build.webkit.org/ <http://build.webkit.org/>>, I would
>>>>  like to know what pages you use most of the time (e.g.: builder page,
>>>>  console view etc.) and what are your primary use-cases (purpose to
>>>>  visit build.webkit.org <http://build.webkit.org/> 
>>>> <http://build.webkit.org/ <http://build.webkit.org/>>). Also if you have
>>>>  any feedback/concern about new Buildbot please let me know.
>>> 
>>> I use 99% of the time the waterfall view, and I think the new waterfall
>>> view of buildbot 9 is a step back in terms of usefulness.
>>> 
>>> More details on about why I think this here:
>>> https://bugs.webkit.org/show_bug.cgi?id=175056#c1 
>>> <https://bugs.webkit.org/show_bug.cgi?id=175056#c1>
>> 
>> I also use waterfall 99% of time, and new waterfall totally sucks.
>> 
>> I wonder if it's possible to get rid of stupid sidebar, if not it's a 
>> complete
>> disaster.
> 
> The sidebar can be collapsed by clicking on the pin button on top ().
> 
> Are there any other suggestions for changes we could try in new buildbot 
> waterfall view to make it suitable?
> 
> Since we are on a very old version of buildbot, at some point we would have 
> to upgrade. It would be nice if we could identify changes which can be made 
> either to buildbot or to our workflow to better suit our requirements. We can 
> also consider passing on the feedback directly to Buildbot by filing bugs at 
> https://github.com/buildbot/buildbot/issues 
> <https://github.com/buildbot/buildbot/issues>, so that Buildbot can fix them.
> 
> Thanks
> Aakash
> 
>> 
>>> 
>>> ,
>>> 
>>> ___
>>> webkit-dev mailing list
>>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
>>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>> 
>> -- 
>> Regards,
>> Konstantin
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Buildbot upgrade on build.webkit.org

2017-12-13 Thread Aakash Jain


> On Dec 11, 2017, at 1:10 AM, Konstantin Tokarev <annu...@yandex.ru> wrote:
> 
> 
> 
> 11.12.2017, 02:49, "Carlos Alberto Lopez Perez" <clo...@igalia.com>:
>> On 07/12/17 21:47, Aakash Jain wrote:
>>>  For people using build.webkit.org <http://build.webkit.org/>, I would
>>>  like to know what pages you use most of the time (e.g.: builder page,
>>>  console view etc.) and what are your primary use-cases (purpose to
>>>  visit build.webkit.org <http://build.webkit.org/>). Also if you have
>>>  any feedback/concern about new Buildbot please let me know.
>> 
>> I use 99% of the time the waterfall view, and I think the new waterfall
>> view of buildbot 9 is a step back in terms of usefulness.
>> 
>> More details on about why I think this here:
>> https://bugs.webkit.org/show_bug.cgi?id=175056#c1
> 
> I also use waterfall 99% of time, and new waterfall totally sucks.
> 
> I wonder if it's possible to get rid of stupid sidebar, if not it's a complete
> disaster.

The sidebar can be collapsed by clicking on the pin button on top ().

Are there any other suggestions for changes we could try in new buildbot 
waterfall view to make it suitable?

Since we are on a very old version of buildbot, at some point we would have to 
upgrade. It would be nice if we could identify changes which can be made either 
to buildbot or to our workflow to better suit our requirements. We can also 
consider passing on the feedback directly to Buildbot by filing bugs at 
https://github.com/buildbot/buildbot/issues 
<https://github.com/buildbot/buildbot/issues>, so that Buildbot can fix them.

Thanks
Aakash

> 
>> 
>> ,
>> 
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
> 
> -- 
> Regards,
> Konstantin
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Buildbot upgrade on build.webkit.org

2017-12-07 Thread Aakash Jain
Hello WebKittens,

We have been using an ancient (5 years old) version of Buildbot on 
build.webkit.org . I am starting to work on upgrading 
to latest Buildbot (0.9.x). New buildbot would look like this: 
https://nine.buildbot.net .

For people using build.webkit.org , I would like to 
know what pages you use most of the time (e.g.: builder page, console view 
etc.) and what are your primary use-cases (purpose to visit build.webkit.org 
). Also if you have any feedback/concern about new 
Buildbot please let me know.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] EWS improvements and new EWSes

2017-11-09 Thread Aakash Jain
Hi All,

We have added webkitpy EWS this week. This should help in making sure that we 
don't accidentally break webkitpy tests. This EWS will run only if your patch 
has webkitpy related changes. Please avoid landing the patch if webkitpy EWS 
fails for your patch.

Don Olmstead added WinCairo EWS. He will send out separate email with more 
details.


Also, we made few other improvements in EWS in last few months:

- EWS reports more information about the patch status 
(https://trac.webkit.org/changeset/221649 
)
- EWS displays individual web-pages for each bubble 
(https://trac.webkit.org/changeset/220434 
)

If you face any issue with EWS or have any ideas for improvements, please free 
to let me know or file a bug.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Formatting style for inline comments in Python code

2017-11-03 Thread Aakash Jain


> On Nov 3, 2017, at 9:20 AM, Maciej Stachowiak <m...@apple.com> wrote:
> 
> 
> 
>> On Nov 3, 2017, at 8:59 AM, Aakash Jain <aakash_j...@apple.com 
>> <mailto:aakash_j...@apple.com>> wrote:
>> 
>> 
>> 
>>> On Nov 2, 2017, at 8:45 PM, Maciej Stachowiak <m...@apple.com 
>>> <mailto:m...@apple.com>> wrote:
>>> 
>>> 
>>> 
>>>> On Nov 2, 2017, at 5:41 PM, Aakash Jain <aakash_j...@apple.com 
>>>> <mailto:aakash_j...@apple.com>> wrote:
>>>> 
>>>> 
>>>> 
>>>>> On Oct 26, 2017, at 10:21 AM, Maciej Stachowiak <m...@apple.com 
>>>>> <mailto:m...@apple.com>> wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Oct 26, 2017, at 10:20 AM, Eric Carlson <eric.carl...@apple.com 
>>>>>> <mailto:eric.carl...@apple.com>> wrote:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> On Oct 26, 2017, at 9:50 AM, Brian Burg <bb...@apple.com 
>>>>>>> <mailto:bb...@apple.com>> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> 2017/10/26 午前9:21、Alexey Proskuryakov <a...@webkit.org 
>>>>>>>> <mailto:a...@webkit.org>>のメール:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> 25 окт. 2017 г., в 18:21, Michael Catanzaro <mcatanz...@igalia.com 
>>>>>>>>> <mailto:mcatanz...@igalia.com>> написал(а):
>>>>>>>>> 
>>>>>>>>> On Wed, Oct 25, 2017 at 4:58 PM, Aakash Jain <aakash_j...@apple.com 
>>>>>>>>> <mailto:aakash_j...@apple.com>> wrote:
>>>>>>>>>> Does anyone else has any opinion/preference for this?
>>>>>>>>> 
>>>>>>>>> The number of spaces before a comment really does not matter, but my 
>>>>>>>>> $0.02: PEP8 is an extremely common style for Python programs that all 
>>>>>>>>> Python developers are familiar with. I would follow that, and forget 
>>>>>>>>> about trying to adapt WebKit C++ style to an unrelated language. 
>>>>>>>>> Trying to adapt the style checker to ignore particular PEP8 rules 
>>>>>>>>> seems like wasted effort.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> There is definitely a number of PEP8 rules that we want to follow. But 
>>>>>>>> I don't think that there is anything about the two space before 
>>>>>>>> comment rule that makes it particularly fitting for Python.
>>>>>>> 
>>>>>>> This is entirely subjective, so: why differ from the vast majority of 
>>>>>>> all other Python code in existence, just to be different? What's the 
>>>>>>> point? PEP8 adherence is nearly universal among projects on PyPi, at 
>>>>>>> least among those that run style linters.
>>>>>>> 
>>>>>>>> I think that we should target WebKit developers with the coding style 
>>>>>>>> as much as possible, not Python developers. As we all agree on the one 
>>>>>>>> space rule elsewhere, why make a part of the code base uncomfortably 
>>>>>>>> different for most WebKit developers?
>>>>>>> 
>>>>>>> I don't understand the distinction between WebKit developers and Python 
>>>>>>> developers. Am I not a C++ developer and web developer as well?
>>>>>>> 
>>>>>>> If "WebKit developers" want to write Python code, perhaps they should 
>>>>>>> learn the Pythonic idioms of the language, just as they would use 
>>>>>>> idioms of Perl, JavaScript, and C++. For better or worse, PEP8 encodes 
>>>>>>> many of these idioms.
>>>>>>> 
>>>>>>> If someone already knows Python, they will be tripped up by this 
>>>>>>> divergence and waste some minutes trying to satisfy the style checker, 
>>>>>>> or just ignore it. If they don't know Python well, then they are being 
>>>>>>> conditioned to follow some variant that has no benefit and is differe

Re: [webkit-dev] Formatting style for inline comments in Python code

2017-11-03 Thread Aakash Jain


> On Nov 2, 2017, at 8:45 PM, Maciej Stachowiak <m...@apple.com> wrote:
> 
> 
> 
>> On Nov 2, 2017, at 5:41 PM, Aakash Jain <aakash_j...@apple.com 
>> <mailto:aakash_j...@apple.com>> wrote:
>> 
>> 
>> 
>>> On Oct 26, 2017, at 10:21 AM, Maciej Stachowiak <m...@apple.com 
>>> <mailto:m...@apple.com>> wrote:
>>> 
>>> 
>>> 
>>>> On Oct 26, 2017, at 10:20 AM, Eric Carlson <eric.carl...@apple.com 
>>>> <mailto:eric.carl...@apple.com>> wrote:
>>>> 
>>>> 
>>>> 
>>>>> On Oct 26, 2017, at 9:50 AM, Brian Burg <bb...@apple.com 
>>>>> <mailto:bb...@apple.com>> wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>>> 2017/10/26 午前9:21、Alexey Proskuryakov <a...@webkit.org 
>>>>>> <mailto:a...@webkit.org>>のメール:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> 25 окт. 2017 г., в 18:21, Michael Catanzaro <mcatanz...@igalia.com 
>>>>>>> <mailto:mcatanz...@igalia.com>> написал(а):
>>>>>>> 
>>>>>>> On Wed, Oct 25, 2017 at 4:58 PM, Aakash Jain <aakash_j...@apple.com 
>>>>>>> <mailto:aakash_j...@apple.com>> wrote:
>>>>>>>> Does anyone else has any opinion/preference for this?
>>>>>>> 
>>>>>>> The number of spaces before a comment really does not matter, but my 
>>>>>>> $0.02: PEP8 is an extremely common style for Python programs that all 
>>>>>>> Python developers are familiar with. I would follow that, and forget 
>>>>>>> about trying to adapt WebKit C++ style to an unrelated language. Trying 
>>>>>>> to adapt the style checker to ignore particular PEP8 rules seems like 
>>>>>>> wasted effort.
>>>>>> 
>>>>>> 
>>>>>> There is definitely a number of PEP8 rules that we want to follow. But I 
>>>>>> don't think that there is anything about the two space before comment 
>>>>>> rule that makes it particularly fitting for Python.
>>>>> 
>>>>> This is entirely subjective, so: why differ from the vast majority of all 
>>>>> other Python code in existence, just to be different? What's the point? 
>>>>> PEP8 adherence is nearly universal among projects on PyPi, at least among 
>>>>> those that run style linters.
>>>>> 
>>>>>> I think that we should target WebKit developers with the coding style as 
>>>>>> much as possible, not Python developers. As we all agree on the one 
>>>>>> space rule elsewhere, why make a part of the code base uncomfortably 
>>>>>> different for most WebKit developers?
>>>>> 
>>>>> I don't understand the distinction between WebKit developers and Python 
>>>>> developers. Am I not a C++ developer and web developer as well?
>>>>> 
>>>>> If "WebKit developers" want to write Python code, perhaps they should 
>>>>> learn the Pythonic idioms of the language, just as they would use idioms 
>>>>> of Perl, JavaScript, and C++. For better or worse, PEP8 encodes many of 
>>>>> these idioms.
>>>>> 
>>>>> If someone already knows Python, they will be tripped up by this 
>>>>> divergence and waste some minutes trying to satisfy the style checker, or 
>>>>> just ignore it. If they don't know Python well, then they are being 
>>>>> conditioned to follow some variant that has no benefit and is different 
>>>>> from what they would see in any other Python code.
>>>>> 
>>>>> I see no value in adding arbitrary barriers to new contributions in 
>>>>> Python code. The code has enough problems as-is, we don't need to make up 
>>>>> our own for some pretense of consistency. We import other Python projects 
>>>>> into the tree, and they follow PEP8, so what was proposed is to make the 
>>>>> Python code in the tree *less* internally consistent.
>>>>> 
>>>> 
>>>> +1
>>> 
>> 
>> 
>>> I'm very used to WebKit style for C++, and I agree that we should use PEP8 
>>> style for Python even where it differs from our C++ style.
>> 
>> I personally prefer following PEP8 while writing python.
>> 
>> Since people have opinions for both C++ style as well as PEP8 style (and 
>> comment spacing is anyways a minor thing), I am going to go with Maciej and 
>> use PEP8 style for Python (which is the style we have already been following 
>> in webkitpy).
> 
> I mean, I agree with this approach, but don't do it just because I said it. 
> :-) These days, I code less C++ and less Python in WebKit than most people on 
> this thread.

I am not doing it only because you said this. I discussed it with Alexey 
yesterday, and he was fine either way. I personally prefer PEP8. Brian Burg, 
Michael Catanzaro and Eric Carlson also supported this. That makes most of us 
(who expressed their opinion) favor this approach.

> 
> Different number of spaces before a same-line comment has never really fazed 
> me, the fact that the comment delimiter is different is much more noticeable.
> 
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Formatting style for inline comments in Python code

2017-11-02 Thread Aakash Jain


> On Oct 26, 2017, at 10:21 AM, Maciej Stachowiak <m...@apple.com> wrote:
> 
> 
> 
>> On Oct 26, 2017, at 10:20 AM, Eric Carlson <eric.carl...@apple.com 
>> <mailto:eric.carl...@apple.com>> wrote:
>> 
>> 
>> 
>>> On Oct 26, 2017, at 9:50 AM, Brian Burg <bb...@apple.com 
>>> <mailto:bb...@apple.com>> wrote:
>>> 
>>> 
>>> 
>>>> 2017/10/26 午前9:21、Alexey Proskuryakov <a...@webkit.org 
>>>> <mailto:a...@webkit.org>>のメール:
>>>> 
>>>> 
>>>> 
>>>>> 25 окт. 2017 г., в 18:21, Michael Catanzaro <mcatanz...@igalia.com 
>>>>> <mailto:mcatanz...@igalia.com>> написал(а):
>>>>> 
>>>>> On Wed, Oct 25, 2017 at 4:58 PM, Aakash Jain <aakash_j...@apple.com 
>>>>> <mailto:aakash_j...@apple.com>> wrote:
>>>>>> Does anyone else has any opinion/preference for this?
>>>>> 
>>>>> The number of spaces before a comment really does not matter, but my 
>>>>> $0.02: PEP8 is an extremely common style for Python programs that all 
>>>>> Python developers are familiar with. I would follow that, and forget 
>>>>> about trying to adapt WebKit C++ style to an unrelated language. Trying 
>>>>> to adapt the style checker to ignore particular PEP8 rules seems like 
>>>>> wasted effort.
>>>> 
>>>> 
>>>> There is definitely a number of PEP8 rules that we want to follow. But I 
>>>> don't think that there is anything about the two space before comment rule 
>>>> that makes it particularly fitting for Python.
>>> 
>>> This is entirely subjective, so: why differ from the vast majority of all 
>>> other Python code in existence, just to be different? What's the point? 
>>> PEP8 adherence is nearly universal among projects on PyPi, at least among 
>>> those that run style linters.
>>> 
>>>> I think that we should target WebKit developers with the coding style as 
>>>> much as possible, not Python developers. As we all agree on the one space 
>>>> rule elsewhere, why make a part of the code base uncomfortably different 
>>>> for most WebKit developers?
>>> 
>>> I don't understand the distinction between WebKit developers and Python 
>>> developers. Am I not a C++ developer and web developer as well?
>>> 
>>> If "WebKit developers" want to write Python code, perhaps they should learn 
>>> the Pythonic idioms of the language, just as they would use idioms of Perl, 
>>> JavaScript, and C++. For better or worse, PEP8 encodes many of these idioms.
>>> 
>>> If someone already knows Python, they will be tripped up by this divergence 
>>> and waste some minutes trying to satisfy the style checker, or just ignore 
>>> it. If they don't know Python well, then they are being conditioned to 
>>> follow some variant that has no benefit and is different from what they 
>>> would see in any other Python code.
>>> 
>>> I see no value in adding arbitrary barriers to new contributions in Python 
>>> code. The code has enough problems as-is, we don't need to make up our own 
>>> for some pretense of consistency. We import other Python projects into the 
>>> tree, and they follow PEP8, so what was proposed is to make the Python code 
>>> in the tree *less* internally consistent.
>>> 
>> 
>> +1
> 


> I'm very used to WebKit style for C++, and I agree that we should use PEP8 
> style for Python even where it differs from our C++ style.

I personally prefer following PEP8 while writing python.

Since people have opinions for both C++ style as well as PEP8 style (and 
comment spacing is anyways a minor thing), I am going to go with Maciej and use 
PEP8 style for Python (which is the style we have already been following in 
webkitpy).

-Aakash

> 
>  - Maciej
> 
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
> https://lists.webkit.org/mailman/listinfo/webkit-dev 
> <https://lists.webkit.org/mailman/listinfo/webkit-dev>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Dead code in webkitpy runtests.py

2017-11-01 Thread Aakash Jain
Hi Everyone,

Inside webkitpy, in tool/steps/runtests.py there is code to run various kind of 
tests (JSC, bindings, webkitpy, webkitperl, layout-tests) which seems like dead 
code. This code is not used by EWS (since ews pass --non-interactive argument 
to webkit-patch). 

I believe the original intention of this code was to have a single command to 
execute all our test-suites. Is there anyone who uses this functionality, by 
manually running webkit-patch command with "--build-and-test --test" arguments 
with an intention of running all possible test-suites?

If not, I am considering to remove this dead code to clean-up webkitpy.

References: https://bugs.webkit.org/show_bug.cgi?id=178608 
, 
https://bugs.webkit.org/show_bug.cgi?id=178599 


Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Formatting style for inline comments in Python code

2017-10-25 Thread Aakash Jain
As Ryosuke said, we can modify check-webkit-style and in-fact I will update 
that if we decide to use Webkit style for this case.

Does anyone else has any opinion/preference for this?

Thanks
Aakash


> On Oct 25, 2017, at 12:22 PM, Brian Burg <bb...@apple.com> wrote:
> 
> In this case, I always prefer the PEP8 rules, because check-webkit-style will 
> complain if you don't do so.
> 
> Brian
> 
>> 2017/10/25 午後0:13、Aakash Jain <aakash_j...@apple.com 
>> <mailto:aakash_j...@apple.com>>のメール:
>> 
>> Hi All,
>> 
>> There is one case in which Webkit style guidelines and Python style 
>> guidelines do not match. This is about spacing before inline comments.
>> 
>> WebKit style guidelines 
>> (https://webkit.org/code-style-guidelines/#comments-eol 
>> <https://webkit.org/code-style-guidelines/#comments-eol>) says: "Use only 
>> one space before end of line comments and in between sentences in comments."
>> 
>> Python PEP8 style guidelines 
>> (https://www.python.org/dev/peps/pep-0008/#inline-comments 
>> <https://www.python.org/dev/peps/pep-0008/#inline-comments>) says: "Inline 
>> comments should be separated by at least two spaces from the statement."
>> 
>> Should we use "one space" or "two spaces" before the inline comments in 
>> python code inside Webkit?
>> 
>> Thanks
>> Aakash
>> 
>> Reference: https://bugs.webkit.org/show_bug.cgi?id=171506 
>> <https://bugs.webkit.org/show_bug.cgi?id=171506>___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Formatting style for inline comments in Python code

2017-10-25 Thread Aakash Jain
Hi All,

There is one case in which Webkit style guidelines and Python style guidelines 
do not match. This is about spacing before inline comments.

WebKit style guidelines (https://webkit.org/code-style-guidelines/#comments-eol 
) says: "Use only one 
space before end of line comments and in between sentences in comments."

Python PEP8 style guidelines 
(https://www.python.org/dev/peps/pep-0008/#inline-comments 
) says: "Inline 
comments should be separated by at least two spaces from the statement."

Should we use "one space" or "two spaces" before the inline comments in python 
code inside Webkit?

Thanks
Aakash

Reference: https://bugs.webkit.org/show_bug.cgi?id=171506
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] EWS bots don't like patches with too many test failures

2017-10-17 Thread Aakash Jain
Hi All,

This issue is now resolved after the fix in 
https://trac.webkit.org/changeset/223572/webkit

Please let us know if anyone notices any issue with EWS bots.

Thanks
Aakash


> On Oct 16, 2017, at 10:03 PM, Aakash Jain <aakash_j...@apple.com> wrote:
> 
> Hi All,
> 
> We recently noticed that bots on some of the EWS queues get stuck while 
> running layout-tests on some particular patches. This results in EWS queues 
> not able to process new patches until the bots are manually fixed.
> 
> This happens with patches which have many layout-test failures (30+). e.g.: 
> https://bugs.webkit.org/show_bug.cgi?id=178332 
> <https://bugs.webkit.org/show_bug.cgi?id=178332>
> 
> We are working on fixing the root cause. Meanwhile I'll try to keep an eye on 
> the bots and will fix them as required. Please let me know if you notice any 
> issues.
> 
> Thanks
> Aakash
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] EWS bots don't like patches with too many test failures

2017-10-16 Thread Aakash Jain
Hi All,

We recently noticed that bots on some of the EWS queues get stuck while running 
layout-tests on some particular patches. This results in EWS queues not able to 
process new patches until the bots are manually fixed.

This happens with patches which have many layout-test failures (30+). e.g.: 
https://bugs.webkit.org/show_bug.cgi?id=178332

We are working on fixing the root cause. Meanwhile I'll try to keep an eye on 
the bots and will fix them as required. Please let me know if you notice any 
issues.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] EWS queues seem stuck

2017-10-08 Thread Aakash Jain
Many bots were stuck processing 
https://bug-178007-attachments.webkit.org/attachment.cgi?id=323010 
 (during 
build-and-test) and mac-debug queue bots was stuck on  
https://bug-178054-attachments.webkit.org/attachment.cgi?id=323106 


I have  brought them all back online. They are all processing patches now. 
Should catch up with the pending patches soon.

I will try to analyze further why they got stuck in the first place.

Thanks
Aakash

> On Oct 7, 2017, at 11:04 PM, Darin Adler  wrote:
> 
> A couple of my patches have been sitting around all day and some of the EWS 
> servers still say they are 15 patches behind. Are they stuck? Can someone 
> bring them back to life?
> 
> — Darin
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] "webkit-patch upload --no-review" submits to EWS by default

2017-08-23 Thread Aakash Jain
We can have "--wip" flag equivalent to " --no-review --no-ews".

But I feel that it might not be clear to many people that it also means skip 
EWS. Many people might expect EWS to be run while passing --wip flag. The name 
"wip" doesn't clear imply skipping EWS. Maybe we can come up with a better 
name, or probably the current --no-ews flag is good enough.

-Aakash

> On Aug 23, 2017, at 11:56 AM, Ryosuke Niwa  wrote:
> 
> On Wed, Aug 23, 2017 at 9:11 AM, Andy Estes  wrote:
>> 
>> 
>> On Aug 22, 2017, at 8:10 PM, Keith Miller  wrote:
>> 
>> Does it make sense to have a --wip option that’s basically --no-review /
>> don’t run EWS? There are times I upload clearly broken patches for early
>> analysis that don’t need to be run on EWS.
>> 
>> 
>> `webkit-patch --no-review --no-ews` should do what you want.
> 
> I think Keith was asking about adding a shorthand for that combination.
> 
> I'm not certain if the most common workflow of uploading a WIP patch
> to Bugzilla involves not triggering EWS. I feel like I upload WIP
> patches to test out EWS but that could be just me.
> 
> - R. Niwa
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Buildbot 0.9

2017-08-02 Thread Aakash Jain
Looking at the configuration, I believe you are running few build-slaves 
connecting to build.webkit.org

Please feel free to upgrade the slaves after buildbot master is upgraded to 0.9 
in https://bugs.webkit.org/show_bug.cgi?id=175056. It wouldn't be mandatory for 
you to upgrade, but would be good to do it for consistency.

Thanks
Aakash


> On Aug 2, 2017, at 12:37 PM, Olmstead, Don  wrote:
> 
> We’re running https://build.webkit.org/buildslaves/wincairo-2 
>  . Its all in a docker 
> container so its pretty painless to upgrade to 0.9 I’m just not sure when we 
> should plan to do the work. So we could update our build slave to 0.9 right 
> now?
>  
> From: aakash_j...@apple.com  
> [mailto:aakash_j...@apple.com ] 
> Sent: Wednesday, August 2, 2017 12:26 PM
> To: Olmstead, Don >
> Cc: WebKit-Dev Development (webkit-dev@lists.webkit.org 
> )  >
> Subject: Re: [webkit-dev] Buildbot 0.9
>  
> Hi Olmstead,
>  
> The commit adding buildbot 0.9 support is 
> https://trac.webkit.org/changeset/220120 
> . This is for bot watcher's 
> dashboard http://build.webkit.org/dashboard/ 
> . The code now have support for both 
> buildbot 0.8 and 0.9. However, we might drop the support for buildbot 0.8 in 
> near future (after we fix https://bugs.webkit.org/show_bug.cgi?id=175056 
> ).
>  
> I would be able to answer your question better if I know what kind of 
> buildbot setup you are using. Is your setup using code from 
> https://svn.webkit.org/repository/webkit/trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/
>  
> 
>  ?
>  
> Thanks
> Aakash
>  
>  
> On Aug 2, 2017, at 11:27 AM, Olmstead, Don  > wrote:
>  
> There was a commit, https://trac.webkit.org/changeset/220139/webkit 
>  , referencing a Buildbot 
> 0.9 dashboard. I’m wondering if we need to upgrade our setup to 0.9 and if so 
> what the timeframe is for the upgrade.
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org 
> https://lists.webkit.org/mailman/listinfo/webkit-dev 
> 
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Buildbot 0.9

2017-08-02 Thread Aakash Jain
Hi Olmstead,

The commit adding buildbot 0.9 support is 
https://trac.webkit.org/changeset/220120. This is for bot watcher's dashboard 
http://build.webkit.org/dashboard/. The code now have support for both buildbot 
0.8 and 0.9. However, we might drop the support for buildbot 0.8 in near future 
(after we fix https://bugs.webkit.org/show_bug.cgi?id=175056).

I would be able to answer your question better if I know what kind of buildbot 
setup you are using. Is your setup using code from 
https://svn.webkit.org/repository/webkit/trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/
 ?

Thanks
Aakash


> On Aug 2, 2017, at 11:27 AM, Olmstead, Don  wrote:
> 
> There was a commit, https://trac.webkit.org/changeset/220139/webkit 
>  , referencing a Buildbot 
> 0.9 dashboard. I’m wondering if we need to upgrade our setup to 0.9 and if so 
> what the timeframe is for the upgrade.
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org 
> https://lists.webkit.org/mailman/listinfo/webkit-dev 
> 
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] SVN trouble

2017-02-27 Thread Aakash Jain
EWS should be up as of yesterday. Please let me know if you notice any issues.

-Aakash

> On Feb 25, 2017, at 8:40 AM, Simon Fraser  wrote:
> 
> EWS is still down. Do we have an ETA?
> 
> Simon
> 
>> On Feb 24, 2017, at 10:25 PM, Alexey Proskuryakov > > wrote:
>> 
>> 
>>> 24 февр. 2017 г., в 19:50, Chris Dumez >> > написал(а):
>>> 
>>> 
>>> 
>>> 
 On Feb 24, 2017, at 11:41 AM, Alexey Proskuryakov > wrote:
 
 I believe that all infrastructure has recovered. We are currently looking 
 into one unrelated issue with webkit-queues, so EWS and commit queue don't 
 work.
 
 - Alexey
>>> 
>>> 
>>> It looks like EWS is still down. Did it break again or is it just not fixed 
>>> yet?
>> 
>> 
>> It did work for a period of time, but no conclusive fix yet.
>> 
>> - Alexey
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] webkitpy - patches landed with large formatting changes

2016-06-22 Thread Aakash Jain
Hi All,

I landed two patches which fixes various formatting issues in webkitpy 
(http://trac.webkit.org/changeset/202362 
 , 
http://trac.webkit.org/changeset/202319 
). These patches touches a lot of 
files in webkitpy. If you are woking on webkitpy, you might want to update your 
local copy and make sure there are no merge conflicts.

Thanks
Aakash

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Improved CrashLog collection in Webkit Tests

2015-10-27 Thread Aakash Jain
Hi All,

In the webkit test results page, we have added a section "Other Crashes" 
(through https://bugs.webkit.org/show_bug.cgi?id=150056). After the tests are 
complete, we go through the crashlog directory on the machine and scan all the 
crashlogs generated during test run. We then check if these crashes are already 
associated with any test by webkitpy, if not, we will list them in "Other 
Crashes" section.

It would enable us to notice the crashes which were previously being hidden. 
For e.g. this run shows a backboardd crash:
https://build.webkit.org/results/Apple%20iOS%209%20Simulator%20Release%20WK2%20(Tests)/r191612%20(645)/results.html

Please let me know if anyone notices any issue or have suggestions for further 
improvement.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Switching servers for EWS and flakiness dashboard

2015-08-02 Thread Aakash Jain
I am looking into it.

Thanks
Aakash

 On Aug 2, 2015, at 1:56 AM, Osztrogonác Csaba o...@inf.u-szeged.hu wrote:
 
 Something is still wrong, the commit queue, the
 style queue,the GTK and EFL EWS are down again.
 
 Commit Queue
 17 hours, 46 minutes ago
 Status: Starting Queue
 2 pending
 
 Style Queue
 5 hours, 35 minutes ago
 Status: Stopping Queue, reason: Delegate terminated queue.
 6 pending
 
 Gtk WK2 EWS
 35 minutes ago
 Status: Idle
 0 pending
 
 Efl WK2 EWS
 16 hours, 29 minutes ago
 Status: Starting Queue
 17 pending
 
 
 Alexey Proskuryakov írta:
 I tried running the style queue from command line, and it processed some 
 patches, errored out on some others, and then hit a different error. I 
 restarted the queue normally then, and it has processed all patches except 
 for https://bugs.webkit.org/attachment.cgi?id=257920action=prettypatch 
 https://bugs.webkit.org/attachment.cgi?id=257920action=prettypatch. We 
 probably need to find a way to enable more logging to find the problem(s). 
 Looks like the bot has trouble talking to bugzilla.
 
 ...
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Switching servers for EWS and flakiness dashboard

2015-08-02 Thread Aakash Jain
I have restarted the commit, style queues. Seems to be working now. I don’t 
have access to EFL/GTK, so someone would need to restart them. I will look more 
to figure out the rootcause.

Thanks
Aakash

 On Aug 2, 2015, at 8:42 AM, Aakash Jain aakash_j...@apple.com wrote:
 
 I am looking into it.
 
 Thanks
 Aakash
 
 On Aug 2, 2015, at 1:56 AM, Osztrogonác Csaba o...@inf.u-szeged.hu wrote:
 
 Something is still wrong, the commit queue, the
 style queue,the GTK and EFL EWS are down again.
 
 Commit Queue
 17 hours, 46 minutes ago
 Status: Starting Queue
 2 pending
 
 Style Queue
 5 hours, 35 minutes ago
 Status: Stopping Queue, reason: Delegate terminated queue.
 6 pending
 
 Gtk WK2 EWS
 35 minutes ago
 Status: Idle
 0 pending
 
 Efl WK2 EWS
 16 hours, 29 minutes ago
 Status: Starting Queue
 17 pending
 
 
 Alexey Proskuryakov írta:
 I tried running the style queue from command line, and it processed some 
 patches, errored out on some others, and then hit a different error. I 
 restarted the queue normally then, and it has processed all patches except 
 for https://bugs.webkit.org/attachment.cgi?id=257920action=prettypatch 
 https://bugs.webkit.org/attachment.cgi?id=257920action=prettypatch. We 
 probably need to find a way to enable more logging to find the problem(s). 
 Looks like the bot has trouble talking to bugzilla.
 
 ...
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 https://lists.webkit.org/mailman/listinfo/webkit-dev
 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Switching servers for EWS and flakiness dashboard [Caution: Message contains Suspicious URL content]

2015-07-31 Thread Aakash Jain
/webkitpy/tool/commands/queues.py,
  line 490, in command_passed
 self._update_status(message, patch=patch)
   File 
 /Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkitpy/tool/commands/queues.py,
  line 211, in _update_status
 return self._tool.status_server.update_status(self.name, message, patch, 
 results_file)
   File 
 /Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkitpy/common/net/statusserver.py,
  line 160, in update_status
 return NetworkTransaction().run(lambda: 
 self._post_status_to_server(queue_name, status, patch, results_file))
   File 
 /Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkitpy/common/net/networktransaction.py,
  line 53, in run
 return request()
   File 
 /Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkitpy/common/net/statusserver.py,
  line 160, in lambda
 return NetworkTransaction().run(lambda: 
 self._post_status_to_server(queue_name, status, patch, results_file))
   File 
 /Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkitpy/common/net/statusserver.py,
  line 93, in _post_status_to_server
 return self._browser.submit().read()  # This is the id of the newly 
 created status object.
   File 
 /Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_mechanize.py,
  line 541, in submit
 return self.open(self.click(*args, **kwds))
   File 
 /Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_mechanize.py,
  line 203, in open
 return self._mech_open(url, data, timeout=timeout)
   File 
 /Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_mechanize.py,
  line 230, in _mech_open
 response = UserAgentBase.open(self, request, data)
   File 
 /Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_opener.py,
  line 193, in open
 response = urlopen(self, req, data)
   File 
 /Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_urllib2_fork.py,
  line 344, in _open
 '_open', req)
   File 
 /Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_urllib2_fork.py,
  line 332, in _call_chain
 result = func(*args)
   File 
 /Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_urllib2_fork.py,
  line 1142, in http_open
 return self.do_open(httplib.HTTPConnection, req)
   File 
 /Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_urllib2_fork.py,
  line 1118, in do_open
 raise URLError(err)
 URLError: urlopen error [Errno 54] Connection reset by peer
 Exception while preparing queue Sleeping until 2015-07-31 10:56:59 (120 
 seconds).
 
 
 - Alexey
 
 
 31 июля 2015 г., в 10:07, Osztrogonác Csaba o...@inf.u-szeged.hu 
 mailto:o...@inf.u-szeged.hu написал(а):
 
 Style queue is still down, otherwise everything looks good.
 
 br,
 Ossy
 
 Aakash Jain írta:
 The upgrade was a success!  EWS and the Flakiness Dashboard are running 
 smoothly. Please reply to this thread if you notice any issues.
 New urls are:
 https://webkit-queues.webkit.org/ https://webkit-queues.webkit.org/
 https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html
 Thanks
 Aakash
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org mailto:webkit-dev@lists.webkit.org
 https://lists.webkit.org/mailman/listinfo/webkit-dev
 
 
 
 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Switching servers for EWS and flakiness dashboard

2015-07-30 Thread Aakash Jain
The upgrade was a success!  EWS and the Flakiness Dashboard are running 
smoothly. Please reply to this thread if you notice any issues.

New urls are:
https://webkit-queues.webkit.org/ https://webkit-queues.webkit.org/
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html 
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html

Thanks
Aakash

 On Jul 30, 2015, at 12:10 AM, Aakash Jain aakash_j...@apple.com wrote:
 
 Hi All,
 
 We are planning to switch to new servers for two of the apps: 
 EWS(webkit-queues.appspot.com) and flakiness dashboard 
 (http://webkit-test-results.appspot.com/dashboards/flakiness_dashboard.html). 
 It would be good to restart all the bots communicating with these apps to 
 ensure that they switch to new server. We will be handling most of the bots, 
 except the ones which we don't have the admin access to (e.g.: efl-wk2-ews 
 and gtk-wk2-ews). It would be great if the admins for these bots can do the 
 necessary.
 
 We plan to commit the patch to switch the servers tomorrow early afternoon 
 (July 30, PST timezone) and restart the bots soon after. You can see more 
 details at https://bugs.webkit.org/show_bug.cgi?id=147178
 
 For those who are interested in knowing more, we are switching the servers 
 since Google is depreciating one of the datastore model : master/slave 
 datastore (https://cloud.google.com/appengine/docs/deprecations/ms_datastore) 
 which these Apps were using. Going forward, we will be using AppScale which 
 is an open-source alternative to Google App Engine.
 
 Thanks
 Aakash
 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Switching servers for EWS and flakiness dashboard

2015-07-30 Thread Aakash Jain
Hi All,

We are planning to switch to new servers for two of the apps: 
EWS(webkit-queues.appspot.com) and flakiness dashboard 
(http://webkit-test-results.appspot.com/dashboards/flakiness_dashboard.html). 
It would be good to restart all the bots communicating with these apps to 
ensure that they switch to new server. We will be handling most of the bots, 
except the ones which we don't have the admin access to (e.g.: efl-wk2-ews and 
gtk-wk2-ews). It would be great if the admins for these bots can do the 
necessary.

We plan to commit the patch to switch the servers tomorrow early afternoon 
(July 30, PST timezone) and restart the bots soon after. You can see more 
details at https://bugs.webkit.org/show_bug.cgi?id=147178

For those who are interested in knowing more, we are switching the servers 
since Google is depreciating one of the datastore model : master/slave 
datastore (https://cloud.google.com/appengine/docs/deprecations/ms_datastore) 
which these Apps were using. Going forward, we will be using AppScale which is 
an open-source alternative to Google App Engine.

Thanks
Aakash

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev