Re: [webkit-dev] Please be careful with webkitpy changes!
The Windows EWS bots process patches fairly quickly. Once I corrected the problem today, it managed to process about 97 patches in about an hour. I do think one bottleneck is due to individual EWS bots “locking” patches. The first bot to reach a patch locks the patch against other bots handling it. If the patch happens to be ‘consumed’ be a bot with some kind of problem (e.g., bad local configuration, a full disk drive, etc.), that patch will not be touched again — even if the other eight EWS bots are sitting dormant. Is there some other processing metric you are concerned about? Brent Fulgham - Apple Inc. On Apr 1, 2015, at 2:26 PM, Maciej Stachowiak m...@apple.com wrote: Is it possible to make EWS start processing changes more promptly? On Apr 1, 2015, at 12:42 PM, Brent Fulgham bfulg...@apple.com wrote: Hi Everyone, We lost Windows EWS coverage for the past 36 hours due to a very benign-appearing change to some webkitpy code. I haven’t yet figured out why this particular set of changes caused the Windows bots to start failing, but it has to do with various differences between the Cygwin Python 2.7.8 build and the versions used on our other EWS bots. This does not seem like something developers SHOULD have to worry about, but it’s an unfortunately reality that they really do need to. To make matters worse, the patch that introduced the problem passed EWS. This is because the EWS bots only really begin using changes to webkitpy when they restart processing (about once every 10-13 build iterations). To help combat this problem, I’d like to request that when making changes to webkitpy, please keep an eye on the various EWS bots to make sure they continue processing. If they do start failing, please roll the patch back out and we can work together to resolve the issue. I apologize for how manual and inconvenient this needs to be (at least for now), but keeping the EWS up and running is critical to the smooth function of this project. If you have any questions, please don’t hesitate to e-mail me or look for me on IRC. Thanks! -Brent ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Please be careful with webkitpy changes!
On Apr 1, 2015 9:29 PM, Brent Fulgham bfulg...@apple.com wrote: The Windows EWS bots process patches fairly quickly. Once I corrected the problem today, it managed to process about 97 patches in about an hour. I do think one bottleneck is due to individual EWS bots “locking” patches. The first bot to reach a patch locks the patch against other bots handling it. If the patch happens to be ‘consumed’ be a bot with some kind of problem (e.g., bad local configuration, a full disk drive, etc.), that patch will not be touched again — even if the other eight EWS bots are sitting dormant. It might be worth adding the ability for a bot to unlock a patch when it detects a problem with itself. We didn't have that originally because we were paranoid about handling the case of an unclean bot shutdown and wanted to make sure we got a lot of experience with that case. At this point, it's probably an overly paranoid design. Adam Is there some other processing metric you are concerned about? Brent Fulgham - Apple Inc. On Apr 1, 2015, at 2:26 PM, Maciej Stachowiak m...@apple.com wrote: Is it possible to make EWS start processing changes more promptly? On Apr 1, 2015, at 12:42 PM, Brent Fulgham bfulg...@apple.com wrote: Hi Everyone, We lost Windows EWS coverage for the past 36 hours due to a very benign-appearing change to some webkitpy code. I haven’t yet figured out why this particular set of changes caused the Windows bots to start failing, but it has to do with various differences between the Cygwin Python 2.7.8 build and the versions used on our other EWS bots. This does not seem like something developers SHOULD have to worry about, but it’s an unfortunately reality that they really do need to. To make matters worse, the patch that introduced the problem passed EWS. This is because the EWS bots only really begin using changes to webkitpy when they restart processing (about once every 10-13 build iterations). To help combat this problem, I’d like to request that when making changes to webkitpy, please keep an eye on the various EWS bots to make sure they continue processing. If they do start failing, please roll the patch back out and we can work together to resolve the issue. I apologize for how manual and inconvenient this needs to be (at least for now), but keeping the EWS up and running is critical to the smooth function of this project. If you have any questions, please don’t hesitate to e-mail me or look for me on IRC. Thanks! -Brent ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
[webkit-dev] Please be careful with webkitpy changes!
Hi Everyone, We lost Windows EWS coverage for the past 36 hours due to a very benign-appearing change to some webkitpy code. I haven’t yet figured out why this particular set of changes caused the Windows bots to start failing, but it has to do with various differences between the Cygwin Python 2.7.8 build and the versions used on our other EWS bots. This does not seem like something developers SHOULD have to worry about, but it’s an unfortunately reality that they really do need to. To make matters worse, the patch that introduced the problem passed EWS. This is because the EWS bots only really begin using changes to webkitpy when they restart processing (about once every 10-13 build iterations). To help combat this problem, I’d like to request that when making changes to webkitpy, please keep an eye on the various EWS bots to make sure they continue processing. If they do start failing, please roll the patch back out and we can work together to resolve the issue. I apologize for how manual and inconvenient this needs to be (at least for now), but keeping the EWS up and running is critical to the smooth function of this project. If you have any questions, please don’t hesitate to e-mail me or look for me on IRC. Thanks! -Brent ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Please be careful with webkitpy changes!
Is it possible to make EWS start processing changes more promptly? On Apr 1, 2015, at 12:42 PM, Brent Fulgham bfulg...@apple.com wrote: Hi Everyone, We lost Windows EWS coverage for the past 36 hours due to a very benign-appearing change to some webkitpy code. I haven’t yet figured out why this particular set of changes caused the Windows bots to start failing, but it has to do with various differences between the Cygwin Python 2.7.8 build and the versions used on our other EWS bots. This does not seem like something developers SHOULD have to worry about, but it’s an unfortunately reality that they really do need to. To make matters worse, the patch that introduced the problem passed EWS. This is because the EWS bots only really begin using changes to webkitpy when they restart processing (about once every 10-13 build iterations). To help combat this problem, I’d like to request that when making changes to webkitpy, please keep an eye on the various EWS bots to make sure they continue processing. If they do start failing, please roll the patch back out and we can work together to resolve the issue. I apologize for how manual and inconvenient this needs to be (at least for now), but keeping the EWS up and running is critical to the smooth function of this project. If you have any questions, please don’t hesitate to e-mail me or look for me on IRC. Thanks! -Brent ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
