Re: xindy, texlive, and concurrency
On Sat, May 18, 2019 at 5:46 PM Kevin Fenzi wrote: > Also, If it's just a lack of funds stopping you from getting to Hungary, > we do have sponsorship, so might be work putting in for that. (Or course > it could be any number of other things, but thought I would mention it > if it's just funds. :) Good grief, I never replied to this. Sorry. My email inbox is in a state of perpetual disarray. Thanks for the tip. I'll have to talk this over with my spouse and boss, the two people who will care most if I'm gone for a few days. :-) -- Jerry James http://www.jamezone.org/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: xindy, texlive, and concurrency
On 5/17/19 2:37 AM, Miro Hrončok wrote: > On 17. 05. 19 3:54, Jerry James wrote: >> And, going off on a really steep tangent, I was just reading about >> Flock and wishing I could go. I've been hanging around the Fedora >> community for something on the order of 14 years now, believe it or >> not, and I have yet to meet a single other Fedora contributor in >> person. There's no way I'm going to make it to Hungary, I'm afraid. >> What is coming up in North America in the next year or so that will >> have significant numbers of Fedorans present? I would love to put >> some faces to the names I've been seeing on my screen all these years. > > Would love to meet you Jerry! Me too! > I think that next Flock after Hungary is supposed to happen in NA. > > It normally rotates between NA and Europe every year, but a shift in > schedule was needed, hence 2019 is in Europe again, so 2020 can be in NA. Also, If it's just a lack of funds stopping you from getting to Hungary, we do have sponsorship, so might be work putting in for that. (Or course it could be any number of other things, but thought I would mention it if it's just funds. :) kevin signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: xindy, texlive, and concurrency
On 17. 05. 19 3:54, Jerry James wrote: And, going off on a really steep tangent, I was just reading about Flock and wishing I could go. I've been hanging around the Fedora community for something on the order of 14 years now, believe it or not, and I have yet to meet a single other Fedora contributor in person. There's no way I'm going to make it to Hungary, I'm afraid. What is coming up in North America in the next year or so that will have significant numbers of Fedorans present? I would love to put some faces to the names I've been seeing on my screen all these years. Would love to meet you Jerry! I think that next Flock after Hungary is supposed to happen in NA. It normally rotates between NA and Europe every year, but a shift in schedule was needed, hence 2019 is in Europe again, so 2020 can be in NA. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: xindy, texlive, and concurrency
On Thu, May 16, 2019 at 5:55 AM Stephen John Smoogen wrote: > I want to say ++ to this. I have found every one of Mr James articles > explaining all the things he has done to work on, debug, dead-ends, etc to be > insightful and incredibly useful to track down in other software. If you have > a chance, please look up some previous ones. > > Thank you again Jerry James. Your work and analysis is deeply appreciated. Awww, you guys are making me blush. Thank you for the kind words. I actually enjoy ferreting out problems in software, which probably makes me some kind of oddball. Heck, the rest of you are probably oddballs, too. :-) And, going off on a really steep tangent, I was just reading about Flock and wishing I could go. I've been hanging around the Fedora community for something on the order of 14 years now, believe it or not, and I have yet to meet a single other Fedora contributor in person. There's no way I'm going to make it to Hungary, I'm afraid. What is coming up in North America in the next year or so that will have significant numbers of Fedorans present? I would love to put some faces to the names I've been seeing on my screen all these years. Regards, -- Jerry James http://www.jamezone.org/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: xindy, texlive, and concurrency
On Thu, 16 May 2019 at 01:51, Vít Ondruch wrote:
> Dear Jerry,
>
> Although I have no idea what xindy is, I enjoyed reading your analysis.
> Thx for your insightful post.
>
>
>
I want to say ++ to this. I have found every one of Mr James articles
explaining all the things he has done to work on, debug, dead-ends, etc to
be insightful and incredibly useful to track down in other software. If you
have a chance, please look up some previous ones.
Thank you again Jerry James. Your work and analysis is deeply appreciated.
> Vít
>
>
> Dne 16. 05. 19 v 3:45 Jerry James napsal(a):
> > I finally found some time to look at the xindy failure to build.
> > First, let me say that I've got a workaround for the problem, which
> > resulted in the beautiful green colors in this xindy-enabled scratch
> > build of texlive-base:
> >
> > https://koji.fedoraproject.org/koji/taskinfo?taskID=34877270
> >
> > When the build process reached the xindy part of the build, it would
> > successfully build xindy itself, then go to work on some
> > documentation. This involved invoking latex on several input files.
> > This marked the first (and possibly only) point in the build where
> > latex was invoked, so latex.fmt had not yet been generated. Since we
> > build with %{?_smp_mflags}, several independent threads invoked latex
> > at approximately the same time. Every one of those invocations
> > detected that latex.fmt was missing and tried to generate it.
> >
> > If you got lucky, each one of those threads would generate latex.fmt,
> > then quickly consume it as it ran on its appointed input file. If you
> > didn't get lucky, one or more threads would start reading latex.fmt
> > after some other thread started writing it, but before it finished
> > writing it all the way. That's why xindy would sometimes build and
> > sometimes fail to build: the build process had a race condition.
> >
> > It is unfortunate that texlive is smart enough to detect missing
> > format files and generate them, but not smart enough to stop that from
> > happening multiple times concurrently. Anyway, poor xindy has been
> > maligned: none of this was xindy's fault, nor clisp's. We, the Fedora
> > packagers, threw concurrency at a job that lacked concurrency control.
> > And the whole xindy_arches thing is useless: this is not an
> > arch-specific problem, so removing random arches from the build
> > accomplishes nothing.
> >
> > The workaround is to invoke latex on a dummy input file early in
> > %build. This causes latex.fmt to be generated, and then everything is
> > hunky-dory when xindy is built later.
> >
> > My recommendation is to remove the xindy_arches conditional from the
> > texlive-base and texlive spec files. Make it unconditionallly on
> > again. Then insert something like this at the top of %build:
> >
> > # Make texlive generate latex.fmt, so that multiple threads do not race
> to
> > # make it during the xindy build.
> > cat > dummy.tex << EOF
> > \documentclass{article}
> > \begin{document}
> > This is a document.
> > \end{document}
> > EOF
> > latex dummy.tex
> > rm -f dummy.*
> >
> > That is the substance of this pull request:
> >
> > https://src.fedoraproject.org/rpms/texlive-base/pull-request/6
> >
> > Also, I should be able to push a clisp build with s390x support to F30
> > stable tomorrow. I, personally, would really appreciate having xindy
> > reappear in F30, due to Sphinx assuming it is available.
> >
> > Regards,
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>
--
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: xindy, texlive, and concurrency
On Thu, May 16, 2019 at 9:41 AM Zbigniew Jędrzejewski-Szmek wrote: > > On Thu, May 16, 2019 at 09:11:21AM +0200, Dridi Boukelmoune wrote: > Let me also add my +1 to the write-up itself, a very nice story. > > Zbyszek I may have accidentally skipped the +1 step and jumped straight to suggestions on how to improve the situation, shifting the blame away from _smp_mflags. But I agree this is a very relatable investigation! Dridi ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: xindy, texlive, and concurrency
On Thu, May 16, 2019 at 09:11:21AM +0200, Dridi Boukelmoune wrote:
> On Thu, May 16, 2019 at 3:46 AM Jerry James wrote:
> >
> > I finally found some time to look at the xindy failure to build.
> > First, let me say that I've got a workaround for the problem, which
> > resulted in the beautiful green colors in this xindy-enabled scratch
> > build of texlive-base:
> >
> > https://koji.fedoraproject.org/koji/taskinfo?taskID=34877270
> >
> > When the build process reached the xindy part of the build, it would
> > successfully build xindy itself, then go to work on some
> > documentation. This involved invoking latex on several input files.
> > This marked the first (and possibly only) point in the build where
> > latex was invoked, so latex.fmt had not yet been generated. Since we
> > build with %{?_smp_mflags}, several independent threads invoked latex
> > at approximately the same time. Every one of those invocations
> > detected that latex.fmt was missing and tried to generate it.
> >
> > If you got lucky, each one of those threads would generate latex.fmt,
> > then quickly consume it as it ran on its appointed input file. If you
> > didn't get lucky, one or more threads would start reading latex.fmt
> > after some other thread started writing it, but before it finished
> > writing it all the way. That's why xindy would sometimes build and
> > sometimes fail to build: the build process had a race condition.
>
> So this is a build system bug from upstream. If concurrency introduces
> a race condition then source-target dependencies are lacking in the
> build system. Depending on the build system it may not be upstream's
> fault, but the tooling itself.
>
> A workaround for the concurrency problem would be an atomic write of
> the generated file. This way even when it is generated multiple times
> while others try to read it they either see a complete or missing file.
>
> The only way I'm aware of for this workaround would be to write to a
> temp file on the same filesystem, and then use mv to rename it
> atomically.
Yes! Check-if-exists, write-to-tmpfile, atomic-rename. If the rename
fails, someone else won the race, so discard the tmpfile, continue
as usual.
So this isn't really a bug in the build system, but a bug in latex
which bungles creation of what is essentially a cache file.
(One expects that a tool like latex may be invoked more than once
at the same time and anything it does internally is its own business).
Let me also add my +1 to the write-up itself, a very nice story.
Zbyszek
> > It is unfortunate that texlive is smart enough to detect missing
> > format files and generate them, but not smart enough to stop that from
> > happening multiple times concurrently. Anyway, poor xindy has been
> > maligned: none of this was xindy's fault, nor clisp's. We, the Fedora
> > packagers, threw concurrency at a job that lacked concurrency control.
> > And the whole xindy_arches thing is useless: this is not an
> > arch-specific problem, so removing random arches from the build
> > accomplishes nothing.
> >
> > The workaround is to invoke latex on a dummy input file early in
> > %build. This causes latex.fmt to be generated, and then everything is
> > hunky-dory when xindy is built later.
> >
> > My recommendation is to remove the xindy_arches conditional from the
> > texlive-base and texlive spec files. Make it unconditionallly on
> > again. Then insert something like this at the top of %build:
> >
> > # Make texlive generate latex.fmt, so that multiple threads do not race to
> > # make it during the xindy build.
> > cat > dummy.tex << EOF
> > \documentclass{article}
> > \begin{document}
> > This is a document.
> > \end{document}
> > EOF
> > latex dummy.tex
> > rm -f dummy.*
> >
> > That is the substance of this pull request:
> >
> > https://src.fedoraproject.org/rpms/texlive-base/pull-request/6
> >
> > Also, I should be able to push a clisp build with s390x support to F30
> > stable tomorrow. I, personally, would really appreciate having xindy
> > reappear in F30, due to Sphinx assuming it is available.
> >
> > Regards,
> > --
> > Jerry James
> > http://www.jamezone.org/
> > ___
> > devel mailing list -- devel@lists.fedoraproject.org
> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
>
Re: xindy, texlive, and concurrency
On Thu, May 16, 2019 at 3:46 AM Jerry James wrote:
>
> I finally found some time to look at the xindy failure to build.
> First, let me say that I've got a workaround for the problem, which
> resulted in the beautiful green colors in this xindy-enabled scratch
> build of texlive-base:
>
> https://koji.fedoraproject.org/koji/taskinfo?taskID=34877270
>
> When the build process reached the xindy part of the build, it would
> successfully build xindy itself, then go to work on some
> documentation. This involved invoking latex on several input files.
> This marked the first (and possibly only) point in the build where
> latex was invoked, so latex.fmt had not yet been generated. Since we
> build with %{?_smp_mflags}, several independent threads invoked latex
> at approximately the same time. Every one of those invocations
> detected that latex.fmt was missing and tried to generate it.
>
> If you got lucky, each one of those threads would generate latex.fmt,
> then quickly consume it as it ran on its appointed input file. If you
> didn't get lucky, one or more threads would start reading latex.fmt
> after some other thread started writing it, but before it finished
> writing it all the way. That's why xindy would sometimes build and
> sometimes fail to build: the build process had a race condition.
So this is a build system bug from upstream. If concurrency introduces
a race condition then source-target dependencies are lacking in the
build system. Depending on the build system it may not be upstream's
fault, but the tooling itself.
A workaround for the concurrency problem would be an atomic write of
the generated file. This way even when it is generated multiple times
while others try to read it they either see a complete or missing file.
The only way I'm aware of for this workaround would be to write to a
temp file on the same filesystem, and then use mv to rename it
atomically.
> It is unfortunate that texlive is smart enough to detect missing
> format files and generate them, but not smart enough to stop that from
> happening multiple times concurrently. Anyway, poor xindy has been
> maligned: none of this was xindy's fault, nor clisp's. We, the Fedora
> packagers, threw concurrency at a job that lacked concurrency control.
> And the whole xindy_arches thing is useless: this is not an
> arch-specific problem, so removing random arches from the build
> accomplishes nothing.
>
> The workaround is to invoke latex on a dummy input file early in
> %build. This causes latex.fmt to be generated, and then everything is
> hunky-dory when xindy is built later.
>
> My recommendation is to remove the xindy_arches conditional from the
> texlive-base and texlive spec files. Make it unconditionallly on
> again. Then insert something like this at the top of %build:
>
> # Make texlive generate latex.fmt, so that multiple threads do not race to
> # make it during the xindy build.
> cat > dummy.tex << EOF
> \documentclass{article}
> \begin{document}
> This is a document.
> \end{document}
> EOF
> latex dummy.tex
> rm -f dummy.*
>
> That is the substance of this pull request:
>
> https://src.fedoraproject.org/rpms/texlive-base/pull-request/6
>
> Also, I should be able to push a clisp build with s390x support to F30
> stable tomorrow. I, personally, would really appreciate having xindy
> reappear in F30, due to Sphinx assuming it is available.
>
> Regards,
> --
> Jerry James
> http://www.jamezone.org/
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: xindy, texlive, and concurrency
On Thursday, 16 May 2019 at 07:51, Vít Ondruch wrote: > Dear Jerry, > > Although I have no idea what xindy is, I enjoyed reading your analysis. > Thx for your insightful post. +1, great analysis and write-up! Regards, Dominik -- Fedora https://getfedora.org | RPM Fusion http://rpmfusion.org There should be a science of discontent. People need hard times and oppression to develop psychic muscles. -- from "Collected Sayings of Muad'Dib" by the Princess Irulan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: xindy, texlive, and concurrency
Dear Jerry,
Although I have no idea what xindy is, I enjoyed reading your analysis.
Thx for your insightful post.
Vít
Dne 16. 05. 19 v 3:45 Jerry James napsal(a):
> I finally found some time to look at the xindy failure to build.
> First, let me say that I've got a workaround for the problem, which
> resulted in the beautiful green colors in this xindy-enabled scratch
> build of texlive-base:
>
> https://koji.fedoraproject.org/koji/taskinfo?taskID=34877270
>
> When the build process reached the xindy part of the build, it would
> successfully build xindy itself, then go to work on some
> documentation. This involved invoking latex on several input files.
> This marked the first (and possibly only) point in the build where
> latex was invoked, so latex.fmt had not yet been generated. Since we
> build with %{?_smp_mflags}, several independent threads invoked latex
> at approximately the same time. Every one of those invocations
> detected that latex.fmt was missing and tried to generate it.
>
> If you got lucky, each one of those threads would generate latex.fmt,
> then quickly consume it as it ran on its appointed input file. If you
> didn't get lucky, one or more threads would start reading latex.fmt
> after some other thread started writing it, but before it finished
> writing it all the way. That's why xindy would sometimes build and
> sometimes fail to build: the build process had a race condition.
>
> It is unfortunate that texlive is smart enough to detect missing
> format files and generate them, but not smart enough to stop that from
> happening multiple times concurrently. Anyway, poor xindy has been
> maligned: none of this was xindy's fault, nor clisp's. We, the Fedora
> packagers, threw concurrency at a job that lacked concurrency control.
> And the whole xindy_arches thing is useless: this is not an
> arch-specific problem, so removing random arches from the build
> accomplishes nothing.
>
> The workaround is to invoke latex on a dummy input file early in
> %build. This causes latex.fmt to be generated, and then everything is
> hunky-dory when xindy is built later.
>
> My recommendation is to remove the xindy_arches conditional from the
> texlive-base and texlive spec files. Make it unconditionallly on
> again. Then insert something like this at the top of %build:
>
> # Make texlive generate latex.fmt, so that multiple threads do not race to
> # make it during the xindy build.
> cat > dummy.tex << EOF
> \documentclass{article}
> \begin{document}
> This is a document.
> \end{document}
> EOF
> latex dummy.tex
> rm -f dummy.*
>
> That is the substance of this pull request:
>
> https://src.fedoraproject.org/rpms/texlive-base/pull-request/6
>
> Also, I should be able to push a clisp build with s390x support to F30
> stable tomorrow. I, personally, would really appreciate having xindy
> reappear in F30, due to Sphinx assuming it is available.
>
> Regards,
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
xindy, texlive, and concurrency
I finally found some time to look at the xindy failure to build.
First, let me say that I've got a workaround for the problem, which
resulted in the beautiful green colors in this xindy-enabled scratch
build of texlive-base:
https://koji.fedoraproject.org/koji/taskinfo?taskID=34877270
When the build process reached the xindy part of the build, it would
successfully build xindy itself, then go to work on some
documentation. This involved invoking latex on several input files.
This marked the first (and possibly only) point in the build where
latex was invoked, so latex.fmt had not yet been generated. Since we
build with %{?_smp_mflags}, several independent threads invoked latex
at approximately the same time. Every one of those invocations
detected that latex.fmt was missing and tried to generate it.
If you got lucky, each one of those threads would generate latex.fmt,
then quickly consume it as it ran on its appointed input file. If you
didn't get lucky, one or more threads would start reading latex.fmt
after some other thread started writing it, but before it finished
writing it all the way. That's why xindy would sometimes build and
sometimes fail to build: the build process had a race condition.
It is unfortunate that texlive is smart enough to detect missing
format files and generate them, but not smart enough to stop that from
happening multiple times concurrently. Anyway, poor xindy has been
maligned: none of this was xindy's fault, nor clisp's. We, the Fedora
packagers, threw concurrency at a job that lacked concurrency control.
And the whole xindy_arches thing is useless: this is not an
arch-specific problem, so removing random arches from the build
accomplishes nothing.
The workaround is to invoke latex on a dummy input file early in
%build. This causes latex.fmt to be generated, and then everything is
hunky-dory when xindy is built later.
My recommendation is to remove the xindy_arches conditional from the
texlive-base and texlive spec files. Make it unconditionallly on
again. Then insert something like this at the top of %build:
# Make texlive generate latex.fmt, so that multiple threads do not race to
# make it during the xindy build.
cat > dummy.tex << EOF
\documentclass{article}
\begin{document}
This is a document.
\end{document}
EOF
latex dummy.tex
rm -f dummy.*
That is the substance of this pull request:
https://src.fedoraproject.org/rpms/texlive-base/pull-request/6
Also, I should be able to push a clisp build with s390x support to F30
stable tomorrow. I, personally, would really appreciate having xindy
reappear in F30, due to Sphinx assuming it is available.
Regards,
--
Jerry James
http://www.jamezone.org/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
