@Vandana: Yes the backlog would need to be addressed, but I've been very encouraged by your and cclauss's recent work in this area. Thanks a ton for the effort, I think it's really improving the quality.
On Thu, Aug 16, 2018 at 8:15 PM Marco de Abreu <[email protected]> wrote: > These are great ideas, sounds very good to me. Thanks for your efforts > around the user experience! > > -Marco > > Vandana Kannan <[email protected]> schrieb am Do., 16. Aug. 2018, > 18:27: > > > @Marco: Most of the examples are standalone applications and not > > notebooks. We could start off by enabling tests for examples that are > > either notebooks or those that can be executed with command line options. > > > > Some of the problems that we may come across are with examples that load > > large datasets (we could probably use a smaller dataset, but it depends > on > > the example), or examples that have a long training time (we could > probably > > execute them for 1 epoch). Straight Dope nightly tests cap the execution > > time to 10 minutes per notebook. We could do the same here wherever > > possible. > > > > @Kellen: Good idea to include flake8/pylint checks for examples in > nightly > > builds - we would need to fix the backlog ( > > https://github.com/apache/incubator-mxnet/issues/12205) before enabling. > > Yes, any contribution from the community on any of these TODO items, > would > > be great. > > > > - Vandana > > > > On 2018/08/16 06:40:56, kellen sunderland <[email protected]> > > wrote: > > > I think it would be very beneficial to start fleshing out the nighties. > > > They provide a lot of value at a relatively small cost. Any > > contributions > > > from the community would be appreciated. > > > > > > Things I could see being beneficial: > > > * Long running tests > > > * In depth flake8/pylint linters > > > * cudamemcheck builds > > > * Asan builds. > > > > > > On Thu, Aug 16, 2018, 12:24 AM Marco de Abreu > > > <[email protected]> wrote: > > > > > > > Hello, > > > > > > > > I think this is a great idea! Thanks a lot for fixing all the > problems > > in > > > > our examples. > > > > > > > > Do you see any problems that could come up if we just run them one by > > one? > > > > We already got a pipeline that allows us to verify jupyter notebooks, > > but > > > > afaik the examples are standalone files with a main function, right? > > > > > > > > Best regards, > > > > Marco > > > > > > > > Vandana Kannan <[email protected]> schrieb am Mi., 15. Aug. 2018, > > > > 23:44: > > > > > > > > > Hi All, > > > > > > > > > > Recently we saw that there were quite a few Pylint > undefined-variable > > > > > errors in MXNet code, a majority of them in the example folder. > These > > > > > errors highlight that there are paths in the examples that are > > broken and > > > > > these have not been caught during testing. > > > > > > > > > > These errors are later reported as issues by users. For example, > > > > > https://github.com/apache/incubator-mxnet/issues/11278. > > > > > > > > > > It would be a good idea to include tests for all examples in the > > nightly > > > > > builds (Similar to > > > > > incubator-mxnet/tests/nightly/test_image_classification.sh). Side > > note: > > > > We > > > > > might have to see what to do about examples that include special > > setup > > > > > instructions or take too long to execute. > > > > > > > > > > Any thoughts or suggestions on writing these tests, feasibility or > > > > > previous attempts at including these tests, would be helpful. > > > > > > > > > > Thanks, > > > > > Vandana > > > > > > > > > > > > > > > > >
