Re: demise of dmlc.ml domain & new Julia docs hosting
Great, thanks for the links and for working on the plan. I think the community should align on the requirements of the new website as long as it's still in development. We can then do an analysis in how far the current state of the beta website differs from the requirements and make that part of the transition plan. Would you mind leading the technical requirement gathering? Best regards, Marco On Thu, Jul 18, 2019 at 11:22 PM Aaron Markham wrote: > What I have written doesn't directly align with the beta site. The site was > created by Mu, and several people have contributed to updating it with > content. The repo is here https://github.com/mli/new-docs > There's no architecture doc afaik. > > I have docs for a redesign and new ux in the wiki: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103089084 > I've tried to apply some of the IA to the beta site, but I'm constrained by > the Sphinx template. > > There's decent progress in the beta site, but it has the drawback of > lock-in with Sphinx again. And covers only Python and R. > > I will be writing a doc to cover a transition plan. I will try to layer it, > so we have the bare minimum all the way to the optimal route. That being > said, any plan requires resources. It's not going to be easy. > > On Thu, Jul 18, 2019, 14:01 Marco de Abreu > wrote: > > > I see. Would you mind linking the thread/doc where we discussed the > > requirements for the new website? > > > > -Marco > > > > On Thu, Jul 18, 2019 at 9:49 PM Aaron Markham > > > wrote: > > > > > I agree with you. I'd like the website to be fairly standard Jekyll and > > > normal front end stuff, rather than a Sphinx template. Docs would be > docs > > > built by doxygen, Sphinx, Scala, java, etc, based on their native docs > > > generation, and we'd get a microsite for each language binding. If one > > > breaks it doesn't break the whole website. You can build them > separately > > or > > > build a particular version. I made the settings system accomplish that > > with > > > our Sphinx build, but the template we have tied with the old versions > > keeps > > > us locked into a complex build. > > > > > > If we must support old versions then this notion needs to be part of > the > > > new beta Python site, and as far as I know, that's not planned. It is > not > > > supported in the beta site now. > > > > > > On Thu, Jul 18, 2019, 12:31 Marco de Abreu > > > wrote: > > > > > > > I'm not entirely sure about dropping docs for old versions to be > > honest. > > > > The root cause in my opinion is the fact that the build is wy to > > > > complicated. If it would have less dependencies and be more > > > > straight-forward, it should be a matter of seconds. The key here > could > > be > > > > to separate the docs from the website. Doc generation should be a > > simple > > > > matter of "run doxygen" (or whatever we use). Right now, the website > > > build > > > > process intermingles every aspect of it. If each version (and > language) > > > > would just output a docs folder that has no outgoing references, they > > > could > > > > be created entirely independent of another. Right now, we regenerate > > > every > > > > version for every single build - that's not necessary. > > > > > > > > Considering how many people are unable to use the latest version of > > > MXNet, > > > > I'd prefer if we provide the docs as long as the release binaries are > > > > available. > > > > > > > > -Marco > > > > > > > > Aaron Markham schrieb am Do., 18. Juli > > 2019, > > > > 21:20: > > > > > > > > > I'd have it check signatures. Ssl is optional I think. China users > > seem > > > > to > > > > > have issues with SSL... That may have changed, but I think we might > > > have > > > > to > > > > > work around that for background downloads for models. > > > > > > > > > > With regard to the current links to dmlc.ml, I asked ivy to just > get > > > rid > > > > > of > > > > > them with # if necessary. Better to 404 than go to malware. This > > > requires > > > > > going back into previous release branches and purging those > > references. > > > > > > > > > > The website is publishing regularly again. I had trouble getting > > Julia > > > to > > > > > work in CI for generating the Julia docs, but I have it working > now. > > > > > > > > > > As we move forward with yet another release, plus the potential > > launch > > > of > > > > > the new Python docs site we need to consider reducing complexity of > > the > > > > > build and prune old versions. I think we should only host master or > > the > > > > > most recent release, and provide instructions on how to generate > > other > > > > > versions. Otherwise the transition will be too difficult and > > > disruptive. > > > > > > > > > > On Tue, Jul 16, 2019, 16:41 Marco de Abreu < > marco.g.ab...@gmail.com> > > > > > wrote: > > > > > > > > > > > While we are at it, we should maybe also take about SSL to avoid > > > these > > > > > > kinds of downloads in future.
Re: demise of dmlc.ml domain & new Julia docs hosting
What I have written doesn't directly align with the beta site. The site was created by Mu, and several people have contributed to updating it with content. The repo is here https://github.com/mli/new-docs There's no architecture doc afaik. I have docs for a redesign and new ux in the wiki: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103089084 I've tried to apply some of the IA to the beta site, but I'm constrained by the Sphinx template. There's decent progress in the beta site, but it has the drawback of lock-in with Sphinx again. And covers only Python and R. I will be writing a doc to cover a transition plan. I will try to layer it, so we have the bare minimum all the way to the optimal route. That being said, any plan requires resources. It's not going to be easy. On Thu, Jul 18, 2019, 14:01 Marco de Abreu wrote: > I see. Would you mind linking the thread/doc where we discussed the > requirements for the new website? > > -Marco > > On Thu, Jul 18, 2019 at 9:49 PM Aaron Markham > wrote: > > > I agree with you. I'd like the website to be fairly standard Jekyll and > > normal front end stuff, rather than a Sphinx template. Docs would be docs > > built by doxygen, Sphinx, Scala, java, etc, based on their native docs > > generation, and we'd get a microsite for each language binding. If one > > breaks it doesn't break the whole website. You can build them separately > or > > build a particular version. I made the settings system accomplish that > with > > our Sphinx build, but the template we have tied with the old versions > keeps > > us locked into a complex build. > > > > If we must support old versions then this notion needs to be part of the > > new beta Python site, and as far as I know, that's not planned. It is not > > supported in the beta site now. > > > > On Thu, Jul 18, 2019, 12:31 Marco de Abreu > > wrote: > > > > > I'm not entirely sure about dropping docs for old versions to be > honest. > > > The root cause in my opinion is the fact that the build is wy to > > > complicated. If it would have less dependencies and be more > > > straight-forward, it should be a matter of seconds. The key here could > be > > > to separate the docs from the website. Doc generation should be a > simple > > > matter of "run doxygen" (or whatever we use). Right now, the website > > build > > > process intermingles every aspect of it. If each version (and language) > > > would just output a docs folder that has no outgoing references, they > > could > > > be created entirely independent of another. Right now, we regenerate > > every > > > version for every single build - that's not necessary. > > > > > > Considering how many people are unable to use the latest version of > > MXNet, > > > I'd prefer if we provide the docs as long as the release binaries are > > > available. > > > > > > -Marco > > > > > > Aaron Markham schrieb am Do., 18. Juli > 2019, > > > 21:20: > > > > > > > I'd have it check signatures. Ssl is optional I think. China users > seem > > > to > > > > have issues with SSL... That may have changed, but I think we might > > have > > > to > > > > work around that for background downloads for models. > > > > > > > > With regard to the current links to dmlc.ml, I asked ivy to just get > > rid > > > > of > > > > them with # if necessary. Better to 404 than go to malware. This > > requires > > > > going back into previous release branches and purging those > references. > > > > > > > > The website is publishing regularly again. I had trouble getting > Julia > > to > > > > work in CI for generating the Julia docs, but I have it working now. > > > > > > > > As we move forward with yet another release, plus the potential > launch > > of > > > > the new Python docs site we need to consider reducing complexity of > the > > > > build and prune old versions. I think we should only host master or > the > > > > most recent release, and provide instructions on how to generate > other > > > > versions. Otherwise the transition will be too difficult and > > disruptive. > > > > > > > > On Tue, Jul 16, 2019, 16:41 Marco de Abreu > > > > wrote: > > > > > > > > > While we are at it, we should maybe also take about SSL to avoid > > these > > > > > kinds of downloads in future. > > > > > > > > > > -Marco > > > > > > > > > > Aaron Markham schrieb am Mi., 17. Juli > > > 2019, > > > > > 01:20: > > > > > > > > > > > Hi Hen, > > > > > > We still have all of the these models that are referenced in this > > > > > > issue: https://github.com/apache/incubator-mxnet/issues/15410 > > > > > > I think it's a pretty big security breach to have systems that do > > > > > > automatic downloading point to a malware site. Anything that > > fetches > > > > > > binary data should be well secured and it really lowers trust if > > > > > > assets send customers to malware sites. > > > > > > There might be more. > > > > > > > > > > > > On Sat, Jul 13, 2019 at 1:00 PM Hen wrote: > > > > > > > > > > > > > > Nice
Re: demise of dmlc.ml domain & new Julia docs hosting
I see. Would you mind linking the thread/doc where we discussed the requirements for the new website? -Marco On Thu, Jul 18, 2019 at 9:49 PM Aaron Markham wrote: > I agree with you. I'd like the website to be fairly standard Jekyll and > normal front end stuff, rather than a Sphinx template. Docs would be docs > built by doxygen, Sphinx, Scala, java, etc, based on their native docs > generation, and we'd get a microsite for each language binding. If one > breaks it doesn't break the whole website. You can build them separately or > build a particular version. I made the settings system accomplish that with > our Sphinx build, but the template we have tied with the old versions keeps > us locked into a complex build. > > If we must support old versions then this notion needs to be part of the > new beta Python site, and as far as I know, that's not planned. It is not > supported in the beta site now. > > On Thu, Jul 18, 2019, 12:31 Marco de Abreu > wrote: > > > I'm not entirely sure about dropping docs for old versions to be honest. > > The root cause in my opinion is the fact that the build is wy to > > complicated. If it would have less dependencies and be more > > straight-forward, it should be a matter of seconds. The key here could be > > to separate the docs from the website. Doc generation should be a simple > > matter of "run doxygen" (or whatever we use). Right now, the website > build > > process intermingles every aspect of it. If each version (and language) > > would just output a docs folder that has no outgoing references, they > could > > be created entirely independent of another. Right now, we regenerate > every > > version for every single build - that's not necessary. > > > > Considering how many people are unable to use the latest version of > MXNet, > > I'd prefer if we provide the docs as long as the release binaries are > > available. > > > > -Marco > > > > Aaron Markham schrieb am Do., 18. Juli 2019, > > 21:20: > > > > > I'd have it check signatures. Ssl is optional I think. China users seem > > to > > > have issues with SSL... That may have changed, but I think we might > have > > to > > > work around that for background downloads for models. > > > > > > With regard to the current links to dmlc.ml, I asked ivy to just get > rid > > > of > > > them with # if necessary. Better to 404 than go to malware. This > requires > > > going back into previous release branches and purging those references. > > > > > > The website is publishing regularly again. I had trouble getting Julia > to > > > work in CI for generating the Julia docs, but I have it working now. > > > > > > As we move forward with yet another release, plus the potential launch > of > > > the new Python docs site we need to consider reducing complexity of the > > > build and prune old versions. I think we should only host master or the > > > most recent release, and provide instructions on how to generate other > > > versions. Otherwise the transition will be too difficult and > disruptive. > > > > > > On Tue, Jul 16, 2019, 16:41 Marco de Abreu > > > wrote: > > > > > > > While we are at it, we should maybe also take about SSL to avoid > these > > > > kinds of downloads in future. > > > > > > > > -Marco > > > > > > > > Aaron Markham schrieb am Mi., 17. Juli > > 2019, > > > > 01:20: > > > > > > > > > Hi Hen, > > > > > We still have all of the these models that are referenced in this > > > > > issue: https://github.com/apache/incubator-mxnet/issues/15410 > > > > > I think it's a pretty big security breach to have systems that do > > > > > automatic downloading point to a malware site. Anything that > fetches > > > > > binary data should be well secured and it really lowers trust if > > > > > assets send customers to malware sites. > > > > > There might be more. > > > > > > > > > > On Sat, Jul 13, 2019 at 1:00 PM Hen wrote: > > > > > > > > > > > > Nice work :) > > > > > > > > > > > > What other links does the project have that are outside Apache > > Infra > > > > > > control and have this risk? > > > > > > > > > > > > Outside of github.com/dmlc which is well known and iiuc in > process > > > for > > > > > > resolving. > > > > > > > > > > > > Hen > > > > > > > > > > > > > > > > > > On Tue, Jul 9, 2019 at 4:46 PM Aaron Markham < > > > > aaron.s.mark...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > The PR has passed CI. Please take a look. > > > > > > > https://github.com/apache/incubator-mxnet/pull/15454 > > > > > > > We really need to get rid of those malware links, and this does > > > that, > > > > > > > restores the Julia docs with a local build, has CI coverage, > and > > a > > > > new > > > > > > > Ubuntu guide (since I had to figure out how to use Julia and > > found > > > > our > > > > > > > docs for that were kind of broken). > > > > > > > Cheers, > > > > > > > Aaron > > > > > > > > > > > > > > On Thu, Jul 4, 2019 at 8:09 PM Iblis Lin > > > > > wrote: > > > > > > > > > > > > > > > > Hi, > > > > > >
Re: demise of dmlc.ml domain & new Julia docs hosting
I agree with you. I'd like the website to be fairly standard Jekyll and normal front end stuff, rather than a Sphinx template. Docs would be docs built by doxygen, Sphinx, Scala, java, etc, based on their native docs generation, and we'd get a microsite for each language binding. If one breaks it doesn't break the whole website. You can build them separately or build a particular version. I made the settings system accomplish that with our Sphinx build, but the template we have tied with the old versions keeps us locked into a complex build. If we must support old versions then this notion needs to be part of the new beta Python site, and as far as I know, that's not planned. It is not supported in the beta site now. On Thu, Jul 18, 2019, 12:31 Marco de Abreu wrote: > I'm not entirely sure about dropping docs for old versions to be honest. > The root cause in my opinion is the fact that the build is wy to > complicated. If it would have less dependencies and be more > straight-forward, it should be a matter of seconds. The key here could be > to separate the docs from the website. Doc generation should be a simple > matter of "run doxygen" (or whatever we use). Right now, the website build > process intermingles every aspect of it. If each version (and language) > would just output a docs folder that has no outgoing references, they could > be created entirely independent of another. Right now, we regenerate every > version for every single build - that's not necessary. > > Considering how many people are unable to use the latest version of MXNet, > I'd prefer if we provide the docs as long as the release binaries are > available. > > -Marco > > Aaron Markham schrieb am Do., 18. Juli 2019, > 21:20: > > > I'd have it check signatures. Ssl is optional I think. China users seem > to > > have issues with SSL... That may have changed, but I think we might have > to > > work around that for background downloads for models. > > > > With regard to the current links to dmlc.ml, I asked ivy to just get rid > > of > > them with # if necessary. Better to 404 than go to malware. This requires > > going back into previous release branches and purging those references. > > > > The website is publishing regularly again. I had trouble getting Julia to > > work in CI for generating the Julia docs, but I have it working now. > > > > As we move forward with yet another release, plus the potential launch of > > the new Python docs site we need to consider reducing complexity of the > > build and prune old versions. I think we should only host master or the > > most recent release, and provide instructions on how to generate other > > versions. Otherwise the transition will be too difficult and disruptive. > > > > On Tue, Jul 16, 2019, 16:41 Marco de Abreu > > wrote: > > > > > While we are at it, we should maybe also take about SSL to avoid these > > > kinds of downloads in future. > > > > > > -Marco > > > > > > Aaron Markham schrieb am Mi., 17. Juli > 2019, > > > 01:20: > > > > > > > Hi Hen, > > > > We still have all of the these models that are referenced in this > > > > issue: https://github.com/apache/incubator-mxnet/issues/15410 > > > > I think it's a pretty big security breach to have systems that do > > > > automatic downloading point to a malware site. Anything that fetches > > > > binary data should be well secured and it really lowers trust if > > > > assets send customers to malware sites. > > > > There might be more. > > > > > > > > On Sat, Jul 13, 2019 at 1:00 PM Hen wrote: > > > > > > > > > > Nice work :) > > > > > > > > > > What other links does the project have that are outside Apache > Infra > > > > > control and have this risk? > > > > > > > > > > Outside of github.com/dmlc which is well known and iiuc in process > > for > > > > > resolving. > > > > > > > > > > Hen > > > > > > > > > > > > > > > On Tue, Jul 9, 2019 at 4:46 PM Aaron Markham < > > > aaron.s.mark...@gmail.com> > > > > > wrote: > > > > > > > > > > > The PR has passed CI. Please take a look. > > > > > > https://github.com/apache/incubator-mxnet/pull/15454 > > > > > > We really need to get rid of those malware links, and this does > > that, > > > > > > restores the Julia docs with a local build, has CI coverage, and > a > > > new > > > > > > Ubuntu guide (since I had to figure out how to use Julia and > found > > > our > > > > > > docs for that were kind of broken). > > > > > > Cheers, > > > > > > Aaron > > > > > > > > > > > > On Thu, Jul 4, 2019 at 8:09 PM Iblis Lin > > > wrote: > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > I will add +1 to the micro-site approach. > > > > > > > Since I have tried to take the generated MD outputs from > > > > > > > Julia's doc system, but the syntax is incompatible with > > > > > > > the MD plugin of Sphinx. > > > > > > > > > > > > > > Iblis Lin > > > > > > > 林峻頤 > > > > > > > > > > > > > > On 7/4/19 12:12 PM, Aaron Markham wrote: > > > > > > > > Hi dev@, > > > > > > > > In case you
Re: demise of dmlc.ml domain & new Julia docs hosting
I'm not entirely sure about dropping docs for old versions to be honest. The root cause in my opinion is the fact that the build is wy to complicated. If it would have less dependencies and be more straight-forward, it should be a matter of seconds. The key here could be to separate the docs from the website. Doc generation should be a simple matter of "run doxygen" (or whatever we use). Right now, the website build process intermingles every aspect of it. If each version (and language) would just output a docs folder that has no outgoing references, they could be created entirely independent of another. Right now, we regenerate every version for every single build - that's not necessary. Considering how many people are unable to use the latest version of MXNet, I'd prefer if we provide the docs as long as the release binaries are available. -Marco Aaron Markham schrieb am Do., 18. Juli 2019, 21:20: > I'd have it check signatures. Ssl is optional I think. China users seem to > have issues with SSL... That may have changed, but I think we might have to > work around that for background downloads for models. > > With regard to the current links to dmlc.ml, I asked ivy to just get rid > of > them with # if necessary. Better to 404 than go to malware. This requires > going back into previous release branches and purging those references. > > The website is publishing regularly again. I had trouble getting Julia to > work in CI for generating the Julia docs, but I have it working now. > > As we move forward with yet another release, plus the potential launch of > the new Python docs site we need to consider reducing complexity of the > build and prune old versions. I think we should only host master or the > most recent release, and provide instructions on how to generate other > versions. Otherwise the transition will be too difficult and disruptive. > > On Tue, Jul 16, 2019, 16:41 Marco de Abreu > wrote: > > > While we are at it, we should maybe also take about SSL to avoid these > > kinds of downloads in future. > > > > -Marco > > > > Aaron Markham schrieb am Mi., 17. Juli 2019, > > 01:20: > > > > > Hi Hen, > > > We still have all of the these models that are referenced in this > > > issue: https://github.com/apache/incubator-mxnet/issues/15410 > > > I think it's a pretty big security breach to have systems that do > > > automatic downloading point to a malware site. Anything that fetches > > > binary data should be well secured and it really lowers trust if > > > assets send customers to malware sites. > > > There might be more. > > > > > > On Sat, Jul 13, 2019 at 1:00 PM Hen wrote: > > > > > > > > Nice work :) > > > > > > > > What other links does the project have that are outside Apache Infra > > > > control and have this risk? > > > > > > > > Outside of github.com/dmlc which is well known and iiuc in process > for > > > > resolving. > > > > > > > > Hen > > > > > > > > > > > > On Tue, Jul 9, 2019 at 4:46 PM Aaron Markham < > > aaron.s.mark...@gmail.com> > > > > wrote: > > > > > > > > > The PR has passed CI. Please take a look. > > > > > https://github.com/apache/incubator-mxnet/pull/15454 > > > > > We really need to get rid of those malware links, and this does > that, > > > > > restores the Julia docs with a local build, has CI coverage, and a > > new > > > > > Ubuntu guide (since I had to figure out how to use Julia and found > > our > > > > > docs for that were kind of broken). > > > > > Cheers, > > > > > Aaron > > > > > > > > > > On Thu, Jul 4, 2019 at 8:09 PM Iblis Lin > > wrote: > > > > > > > > > > > > Hi, > > > > > > > > > > > > I will add +1 to the micro-site approach. > > > > > > Since I have tried to take the generated MD outputs from > > > > > > Julia's doc system, but the syntax is incompatible with > > > > > > the MD plugin of Sphinx. > > > > > > > > > > > > Iblis Lin > > > > > > 林峻頤 > > > > > > > > > > > > On 7/4/19 12:12 PM, Aaron Markham wrote: > > > > > > > Hi dev@, > > > > > > > In case you missed the issues with the dmlc.ml domain, it was > > let > > > go > > > > > > > or sniped and now goes to a malware site. [1] Several assets > like > > > > > > > models and the Julia documentation were hosted there. > > > > > > > > > > > > > > I made some progress getting the Julia docs generated as part > of > > > the > > > > > > > regular website build flow. [2] It'll work as long as you have > > > Julia > > > > > > > installed and configured with MXNet. I don't imagine you can > use > > > it to > > > > > > > build the website natively on your mac or windows box at this > > point > > > > > > > because all of the CI and related instructions appear to be > only > > > for > > > > > > > Ubuntu. That being said, I added an option to dev_menu.py so > you > > > can > > > > > > > build the Julia docs with docker on whatever host you're on. > > > > > > > > > > > > > > I tried to bring in the markdown files and have the website > theme > > > > > > > applied to them. Many things were then broken
Re: demise of dmlc.ml domain & new Julia docs hosting
I'd have it check signatures. Ssl is optional I think. China users seem to have issues with SSL... That may have changed, but I think we might have to work around that for background downloads for models. With regard to the current links to dmlc.ml, I asked ivy to just get rid of them with # if necessary. Better to 404 than go to malware. This requires going back into previous release branches and purging those references. The website is publishing regularly again. I had trouble getting Julia to work in CI for generating the Julia docs, but I have it working now. As we move forward with yet another release, plus the potential launch of the new Python docs site we need to consider reducing complexity of the build and prune old versions. I think we should only host master or the most recent release, and provide instructions on how to generate other versions. Otherwise the transition will be too difficult and disruptive. On Tue, Jul 16, 2019, 16:41 Marco de Abreu wrote: > While we are at it, we should maybe also take about SSL to avoid these > kinds of downloads in future. > > -Marco > > Aaron Markham schrieb am Mi., 17. Juli 2019, > 01:20: > > > Hi Hen, > > We still have all of the these models that are referenced in this > > issue: https://github.com/apache/incubator-mxnet/issues/15410 > > I think it's a pretty big security breach to have systems that do > > automatic downloading point to a malware site. Anything that fetches > > binary data should be well secured and it really lowers trust if > > assets send customers to malware sites. > > There might be more. > > > > On Sat, Jul 13, 2019 at 1:00 PM Hen wrote: > > > > > > Nice work :) > > > > > > What other links does the project have that are outside Apache Infra > > > control and have this risk? > > > > > > Outside of github.com/dmlc which is well known and iiuc in process for > > > resolving. > > > > > > Hen > > > > > > > > > On Tue, Jul 9, 2019 at 4:46 PM Aaron Markham < > aaron.s.mark...@gmail.com> > > > wrote: > > > > > > > The PR has passed CI. Please take a look. > > > > https://github.com/apache/incubator-mxnet/pull/15454 > > > > We really need to get rid of those malware links, and this does that, > > > > restores the Julia docs with a local build, has CI coverage, and a > new > > > > Ubuntu guide (since I had to figure out how to use Julia and found > our > > > > docs for that were kind of broken). > > > > Cheers, > > > > Aaron > > > > > > > > On Thu, Jul 4, 2019 at 8:09 PM Iblis Lin > wrote: > > > > > > > > > > Hi, > > > > > > > > > > I will add +1 to the micro-site approach. > > > > > Since I have tried to take the generated MD outputs from > > > > > Julia's doc system, but the syntax is incompatible with > > > > > the MD plugin of Sphinx. > > > > > > > > > > Iblis Lin > > > > > 林峻頤 > > > > > > > > > > On 7/4/19 12:12 PM, Aaron Markham wrote: > > > > > > Hi dev@, > > > > > > In case you missed the issues with the dmlc.ml domain, it was > let > > go > > > > > > or sniped and now goes to a malware site. [1] Several assets like > > > > > > models and the Julia documentation were hosted there. > > > > > > > > > > > > I made some progress getting the Julia docs generated as part of > > the > > > > > > regular website build flow. [2] It'll work as long as you have > > Julia > > > > > > installed and configured with MXNet. I don't imagine you can use > > it to > > > > > > build the website natively on your mac or windows box at this > point > > > > > > because all of the CI and related instructions appear to be only > > for > > > > > > Ubuntu. That being said, I added an option to dev_menu.py so you > > can > > > > > > build the Julia docs with docker on whatever host you're on. > > > > > > > > > > > > I tried to bring in the markdown files and have the website theme > > > > > > applied to them. Many things were then broken - in large part due > > to > > > > > > some bugs in the website code related to post processing and > > injection > > > > > > of dom elements. This would require a rewrite of the Julia docs > to > > > > > > workaround the existing website bugs. Rather than do this, I just > > took > > > > > > the Julia site output, which has its own look and feel and nested > > in > > > > > > Julia's API directory. This is much like how the scala docs and > the > > > > > > java docs are - using their own look and feel as a micro-site. I > > feel > > > > > > that this is the better approach for now. > > > > > > > > > > > > The PR for the Julia docs is here: > > > > > > https://github.com/apache/incubator-mxnet/pull/15454 > > > > > > > > > > > > Ivy and I created several new issues to cover the broken links > that > > > > > > will need fixing. Models that need to be recovered or recreated > and > > > > > > uploaded to a new location. I have an s3 bucket that I've been > > using > > > > > > for some public assets like this, but I can't make progress on > > fixing > > > > > > those links when the models just don't exist. And I don't have >
Re: demise of dmlc.ml domain & new Julia docs hosting
While we are at it, we should maybe also take about SSL to avoid these kinds of downloads in future. -Marco Aaron Markham schrieb am Mi., 17. Juli 2019, 01:20: > Hi Hen, > We still have all of the these models that are referenced in this > issue: https://github.com/apache/incubator-mxnet/issues/15410 > I think it's a pretty big security breach to have systems that do > automatic downloading point to a malware site. Anything that fetches > binary data should be well secured and it really lowers trust if > assets send customers to malware sites. > There might be more. > > On Sat, Jul 13, 2019 at 1:00 PM Hen wrote: > > > > Nice work :) > > > > What other links does the project have that are outside Apache Infra > > control and have this risk? > > > > Outside of github.com/dmlc which is well known and iiuc in process for > > resolving. > > > > Hen > > > > > > On Tue, Jul 9, 2019 at 4:46 PM Aaron Markham > > wrote: > > > > > The PR has passed CI. Please take a look. > > > https://github.com/apache/incubator-mxnet/pull/15454 > > > We really need to get rid of those malware links, and this does that, > > > restores the Julia docs with a local build, has CI coverage, and a new > > > Ubuntu guide (since I had to figure out how to use Julia and found our > > > docs for that were kind of broken). > > > Cheers, > > > Aaron > > > > > > On Thu, Jul 4, 2019 at 8:09 PM Iblis Lin wrote: > > > > > > > > Hi, > > > > > > > > I will add +1 to the micro-site approach. > > > > Since I have tried to take the generated MD outputs from > > > > Julia's doc system, but the syntax is incompatible with > > > > the MD plugin of Sphinx. > > > > > > > > Iblis Lin > > > > 林峻頤 > > > > > > > > On 7/4/19 12:12 PM, Aaron Markham wrote: > > > > > Hi dev@, > > > > > In case you missed the issues with the dmlc.ml domain, it was let > go > > > > > or sniped and now goes to a malware site. [1] Several assets like > > > > > models and the Julia documentation were hosted there. > > > > > > > > > > I made some progress getting the Julia docs generated as part of > the > > > > > regular website build flow. [2] It'll work as long as you have > Julia > > > > > installed and configured with MXNet. I don't imagine you can use > it to > > > > > build the website natively on your mac or windows box at this point > > > > > because all of the CI and related instructions appear to be only > for > > > > > Ubuntu. That being said, I added an option to dev_menu.py so you > can > > > > > build the Julia docs with docker on whatever host you're on. > > > > > > > > > > I tried to bring in the markdown files and have the website theme > > > > > applied to them. Many things were then broken - in large part due > to > > > > > some bugs in the website code related to post processing and > injection > > > > > of dom elements. This would require a rewrite of the Julia docs to > > > > > workaround the existing website bugs. Rather than do this, I just > took > > > > > the Julia site output, which has its own look and feel and nested > in > > > > > Julia's API directory. This is much like how the scala docs and the > > > > > java docs are - using their own look and feel as a micro-site. I > feel > > > > > that this is the better approach for now. > > > > > > > > > > The PR for the Julia docs is here: > > > > > https://github.com/apache/incubator-mxnet/pull/15454 > > > > > > > > > > Ivy and I created several new issues to cover the broken links that > > > > > will need fixing. Models that need to be recovered or recreated and > > > > > uploaded to a new location. I have an s3 bucket that I've been > using > > > > > for some public assets like this, but I can't make progress on > fixing > > > > > those links when the models just don't exist. And I don't have the > > > > > bandwidth to regenerate the model zoo and validate the models. > > > > > > > > > > Any thoughts or suggestions are appreciated. > > > > > Happy 4th of July! > > > > > > > > > > [1] https://github.com/apache/incubator-mxnet/issues/15410 > > > > > [2] https://github.com/apache/incubator-mxnet/pull/15454 > > > > > > > > >
Re: demise of dmlc.ml domain & new Julia docs hosting
Hi Hen, We still have all of the these models that are referenced in this issue: https://github.com/apache/incubator-mxnet/issues/15410 I think it's a pretty big security breach to have systems that do automatic downloading point to a malware site. Anything that fetches binary data should be well secured and it really lowers trust if assets send customers to malware sites. There might be more. On Sat, Jul 13, 2019 at 1:00 PM Hen wrote: > > Nice work :) > > What other links does the project have that are outside Apache Infra > control and have this risk? > > Outside of github.com/dmlc which is well known and iiuc in process for > resolving. > > Hen > > > On Tue, Jul 9, 2019 at 4:46 PM Aaron Markham > wrote: > > > The PR has passed CI. Please take a look. > > https://github.com/apache/incubator-mxnet/pull/15454 > > We really need to get rid of those malware links, and this does that, > > restores the Julia docs with a local build, has CI coverage, and a new > > Ubuntu guide (since I had to figure out how to use Julia and found our > > docs for that were kind of broken). > > Cheers, > > Aaron > > > > On Thu, Jul 4, 2019 at 8:09 PM Iblis Lin wrote: > > > > > > Hi, > > > > > > I will add +1 to the micro-site approach. > > > Since I have tried to take the generated MD outputs from > > > Julia's doc system, but the syntax is incompatible with > > > the MD plugin of Sphinx. > > > > > > Iblis Lin > > > 林峻頤 > > > > > > On 7/4/19 12:12 PM, Aaron Markham wrote: > > > > Hi dev@, > > > > In case you missed the issues with the dmlc.ml domain, it was let go > > > > or sniped and now goes to a malware site. [1] Several assets like > > > > models and the Julia documentation were hosted there. > > > > > > > > I made some progress getting the Julia docs generated as part of the > > > > regular website build flow. [2] It'll work as long as you have Julia > > > > installed and configured with MXNet. I don't imagine you can use it to > > > > build the website natively on your mac or windows box at this point > > > > because all of the CI and related instructions appear to be only for > > > > Ubuntu. That being said, I added an option to dev_menu.py so you can > > > > build the Julia docs with docker on whatever host you're on. > > > > > > > > I tried to bring in the markdown files and have the website theme > > > > applied to them. Many things were then broken - in large part due to > > > > some bugs in the website code related to post processing and injection > > > > of dom elements. This would require a rewrite of the Julia docs to > > > > workaround the existing website bugs. Rather than do this, I just took > > > > the Julia site output, which has its own look and feel and nested in > > > > Julia's API directory. This is much like how the scala docs and the > > > > java docs are - using their own look and feel as a micro-site. I feel > > > > that this is the better approach for now. > > > > > > > > The PR for the Julia docs is here: > > > > https://github.com/apache/incubator-mxnet/pull/15454 > > > > > > > > Ivy and I created several new issues to cover the broken links that > > > > will need fixing. Models that need to be recovered or recreated and > > > > uploaded to a new location. I have an s3 bucket that I've been using > > > > for some public assets like this, but I can't make progress on fixing > > > > those links when the models just don't exist. And I don't have the > > > > bandwidth to regenerate the model zoo and validate the models. > > > > > > > > Any thoughts or suggestions are appreciated. > > > > Happy 4th of July! > > > > > > > > [1] https://github.com/apache/incubator-mxnet/issues/15410 > > > > [2] https://github.com/apache/incubator-mxnet/pull/15454 > > > > > >
Re: demise of dmlc.ml domain & new Julia docs hosting
Nice work :) What other links does the project have that are outside Apache Infra control and have this risk? Outside of github.com/dmlc which is well known and iiuc in process for resolving. Hen On Tue, Jul 9, 2019 at 4:46 PM Aaron Markham wrote: > The PR has passed CI. Please take a look. > https://github.com/apache/incubator-mxnet/pull/15454 > We really need to get rid of those malware links, and this does that, > restores the Julia docs with a local build, has CI coverage, and a new > Ubuntu guide (since I had to figure out how to use Julia and found our > docs for that were kind of broken). > Cheers, > Aaron > > On Thu, Jul 4, 2019 at 8:09 PM Iblis Lin wrote: > > > > Hi, > > > > I will add +1 to the micro-site approach. > > Since I have tried to take the generated MD outputs from > > Julia's doc system, but the syntax is incompatible with > > the MD plugin of Sphinx. > > > > Iblis Lin > > 林峻頤 > > > > On 7/4/19 12:12 PM, Aaron Markham wrote: > > > Hi dev@, > > > In case you missed the issues with the dmlc.ml domain, it was let go > > > or sniped and now goes to a malware site. [1] Several assets like > > > models and the Julia documentation were hosted there. > > > > > > I made some progress getting the Julia docs generated as part of the > > > regular website build flow. [2] It'll work as long as you have Julia > > > installed and configured with MXNet. I don't imagine you can use it to > > > build the website natively on your mac or windows box at this point > > > because all of the CI and related instructions appear to be only for > > > Ubuntu. That being said, I added an option to dev_menu.py so you can > > > build the Julia docs with docker on whatever host you're on. > > > > > > I tried to bring in the markdown files and have the website theme > > > applied to them. Many things were then broken - in large part due to > > > some bugs in the website code related to post processing and injection > > > of dom elements. This would require a rewrite of the Julia docs to > > > workaround the existing website bugs. Rather than do this, I just took > > > the Julia site output, which has its own look and feel and nested in > > > Julia's API directory. This is much like how the scala docs and the > > > java docs are - using their own look and feel as a micro-site. I feel > > > that this is the better approach for now. > > > > > > The PR for the Julia docs is here: > > > https://github.com/apache/incubator-mxnet/pull/15454 > > > > > > Ivy and I created several new issues to cover the broken links that > > > will need fixing. Models that need to be recovered or recreated and > > > uploaded to a new location. I have an s3 bucket that I've been using > > > for some public assets like this, but I can't make progress on fixing > > > those links when the models just don't exist. And I don't have the > > > bandwidth to regenerate the model zoo and validate the models. > > > > > > Any thoughts or suggestions are appreciated. > > > Happy 4th of July! > > > > > > [1] https://github.com/apache/incubator-mxnet/issues/15410 > > > [2] https://github.com/apache/incubator-mxnet/pull/15454 > > > >
Re: demise of dmlc.ml domain & new Julia docs hosting
The PR has passed CI. Please take a look. https://github.com/apache/incubator-mxnet/pull/15454 We really need to get rid of those malware links, and this does that, restores the Julia docs with a local build, has CI coverage, and a new Ubuntu guide (since I had to figure out how to use Julia and found our docs for that were kind of broken). Cheers, Aaron On Thu, Jul 4, 2019 at 8:09 PM Iblis Lin wrote: > > Hi, > > I will add +1 to the micro-site approach. > Since I have tried to take the generated MD outputs from > Julia's doc system, but the syntax is incompatible with > the MD plugin of Sphinx. > > Iblis Lin > 林峻頤 > > On 7/4/19 12:12 PM, Aaron Markham wrote: > > Hi dev@, > > In case you missed the issues with the dmlc.ml domain, it was let go > > or sniped and now goes to a malware site. [1] Several assets like > > models and the Julia documentation were hosted there. > > > > I made some progress getting the Julia docs generated as part of the > > regular website build flow. [2] It'll work as long as you have Julia > > installed and configured with MXNet. I don't imagine you can use it to > > build the website natively on your mac or windows box at this point > > because all of the CI and related instructions appear to be only for > > Ubuntu. That being said, I added an option to dev_menu.py so you can > > build the Julia docs with docker on whatever host you're on. > > > > I tried to bring in the markdown files and have the website theme > > applied to them. Many things were then broken - in large part due to > > some bugs in the website code related to post processing and injection > > of dom elements. This would require a rewrite of the Julia docs to > > workaround the existing website bugs. Rather than do this, I just took > > the Julia site output, which has its own look and feel and nested in > > Julia's API directory. This is much like how the scala docs and the > > java docs are - using their own look and feel as a micro-site. I feel > > that this is the better approach for now. > > > > The PR for the Julia docs is here: > > https://github.com/apache/incubator-mxnet/pull/15454 > > > > Ivy and I created several new issues to cover the broken links that > > will need fixing. Models that need to be recovered or recreated and > > uploaded to a new location. I have an s3 bucket that I've been using > > for some public assets like this, but I can't make progress on fixing > > those links when the models just don't exist. And I don't have the > > bandwidth to regenerate the model zoo and validate the models. > > > > Any thoughts or suggestions are appreciated. > > Happy 4th of July! > > > > [1] https://github.com/apache/incubator-mxnet/issues/15410 > > [2] https://github.com/apache/incubator-mxnet/pull/15454 > >
Re: demise of dmlc.ml domain & new Julia docs hosting
Hi, I will add +1 to the micro-site approach. Since I have tried to take the generated MD outputs from Julia's doc system, but the syntax is incompatible with the MD plugin of Sphinx. Iblis Lin 林峻頤 On 7/4/19 12:12 PM, Aaron Markham wrote: Hi dev@, In case you missed the issues with the dmlc.ml domain, it was let go or sniped and now goes to a malware site. [1] Several assets like models and the Julia documentation were hosted there. I made some progress getting the Julia docs generated as part of the regular website build flow. [2] It'll work as long as you have Julia installed and configured with MXNet. I don't imagine you can use it to build the website natively on your mac or windows box at this point because all of the CI and related instructions appear to be only for Ubuntu. That being said, I added an option to dev_menu.py so you can build the Julia docs with docker on whatever host you're on. I tried to bring in the markdown files and have the website theme applied to them. Many things were then broken - in large part due to some bugs in the website code related to post processing and injection of dom elements. This would require a rewrite of the Julia docs to workaround the existing website bugs. Rather than do this, I just took the Julia site output, which has its own look and feel and nested in Julia's API directory. This is much like how the scala docs and the java docs are - using their own look and feel as a micro-site. I feel that this is the better approach for now. The PR for the Julia docs is here: https://github.com/apache/incubator-mxnet/pull/15454 Ivy and I created several new issues to cover the broken links that will need fixing. Models that need to be recovered or recreated and uploaded to a new location. I have an s3 bucket that I've been using for some public assets like this, but I can't make progress on fixing those links when the models just don't exist. And I don't have the bandwidth to regenerate the model zoo and validate the models. Any thoughts or suggestions are appreciated. Happy 4th of July! [1] https://github.com/apache/incubator-mxnet/issues/15410 [2] https://github.com/apache/incubator-mxnet/pull/15454
