Re: help with bash script to translate @ref{} items in translated manuals
Wait, I didn't give any input.. Now I've run: ./node.pl *.itely and it prints everything. Anyway I think I'll have to figure it out by myself with something I know how to use. Il giorno mar 25 apr 2017 alle 7:58, Federico Bruniha scritto: Hi Andrew Thank you, but it hangs forever and nothing happens. Il giorno sab 22 apr 2017 alle 5:37, Andrew Bernard ha scritto: Hi Federico, I believe you are trying to automate a set of translations, correct? If so, here's a way to do it in perl that avoids all the shell convolutions. I assume you do know perl. If not, always worth knowing for this sort of quick work. Just add the translations to the hash table in the script. The virtue of this small tool is that it will tell you if you missed any. Reads file from stdin. Outputs to stdout. You can figure how to process the whole directory. There's a hundred ways to do this. Some people like to open all the files in the perl script. I prefer to keep it simple. Hope this may be useful for now and in the future. Andrew == snip #!/usr/bin/perl use strict; use warnings; my @ref; # translation table my %translations = ( 'Automatic beams' => 'Automatic Beams in Italian', 'Stems' => 'Stems in Italian', ); while (<>) { if (@ref = /\@ref\{(.+)\}/) { if (exists $translations{$ref[0]}) { s/$ref[0]/$translations{$ref[0]}/; } else { print STDERR "no translation for $ref[0]", "\n"; } } print; } == snip ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
Hi Andrew Thank you, but it hangs forever and nothing happens. Il giorno sab 22 apr 2017 alle 5:37, Andrew Bernardha scritto: Hi Federico, I believe you are trying to automate a set of translations, correct? If so, here's a way to do it in perl that avoids all the shell convolutions. I assume you do know perl. If not, always worth knowing for this sort of quick work. Just add the translations to the hash table in the script. The virtue of this small tool is that it will tell you if you missed any. Reads file from stdin. Outputs to stdout. You can figure how to process the whole directory. There's a hundred ways to do this. Some people like to open all the files in the perl script. I prefer to keep it simple. Hope this may be useful for now and in the future. Andrew == snip #!/usr/bin/perl use strict; use warnings; my @ref; # translation table my %translations = ( 'Automatic beams' => 'Automatic Beams in Italian', 'Stems' => 'Stems in Italian', ); while (<>) { if (@ref = /\@ref\{(.+)\}/) { if (exists $translations{$ref[0]}) { s/$ref[0]/$translations{$ref[0]}/; } else { print STDERR "no translation for $ref[0]", "\n"; } } print; } == snip ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
Federico Bruni: > Il giorno ven 21 apr 2017 alle 7:08, Federico Bruni >ha scritto: > > I guess I'll have to revert to my almost manual replacement I've used > > so far. > > Or I may try a different approach and script a replacement from a file > like: > > @ref{Different editions from one source},@ref{Edizioni diverse da un > unico sorgente} > @ref{Dimensions},@ref{Dimensioni} > @ref{Direction and placement},@ref{Direzione e posizionamento} > > that replace first instance of @ref with the second instance (after the > comma) Well, that is about the same as putting the translation in the po file and running the attached script. Then if and when gettext is working again for doc., as someone said it has been, then you don't have to convert your replacement file to po style. $ read_po.pl Documentation/po/it.po .itely | head File: Documentation/essay/engraving.itely @ref{Music engraving} -> @ref{} @ref{The LilyPond story} -> @ref{} @ref{Engraving details} -> @ref{} @ref{Music fonts} -> @ref{} @ref{Optical spacing} -> @ref{} @ref{Ledger lines} -> @ref{} @ref{Optical sizing} -> @ref{} @ref{Why work so hard?} -> @ref{} @ref{Automated engraving} -> @ref{} $ Regards, /Karl Hammar --- Aspö Data Lilla Aspö 148 S-742 94 Östhammar Sweden +46 173 140 57 #!/usr/bin/perl -w use strict; use Data::Dumper; my %file_list; sub read_po() { my $po_file = $ARGV[0]; my @arr; open(FH, $po_file) || die("cannot open $po_file"); my $nxt = [ [], "", "" ]; while () { chomp; s/^\s*//; s/\s*$//; if (m/^$/) { my %file; my @lst; for my $r (@{$$nxt[0]}) { next if ($r =~ m|Documentation/cs/|); if ($r =~ m| in (Documentation/.*\.itely)$|) { $file{$1} = 1; } push @lst, $r; } my @Keys = sort keys %file; if (@Keys) { $$nxt[0] = [ @lst ]; push @arr, $nxt; for my $f (@Keys) { if (!defined($file_list{$f})) { $file_list{$f} = []; } my $r = [ $$nxt[1], $$nxt[2] ]; push @{$file_list{$f}}, $r; } } $nxt = [ [], "", "" ]; next; } if (m/^\#/) { push @{$$nxt[0]}, $_; next; } if (m/^msgid\s*\"(.*)\"$/) { my $str = $1; $$nxt[1] = $str; next; } if (m/^msgstr\s*\"(.*)\"$/) { my $str = $1; $$nxt[2] = $str; next; } if (m/^\"(.*)\"$/) { my $str = $1; if ($$nxt[2] eq "") { $$nxt[1] .= $str; } else { $$nxt[2] .= $str; } next; } warn("unhandled data: <$_>"); } close(FH); @arr; } sub print_arr(@) { my @arr = @_; for my $r (@arr) { print join("\n", @{$$r[0]}), "\n"; print "msgid \"$$r[1]\"\n"; print "msgstr \"$$r[2]\"\n"; print "\n"; } } sub mk_subst(@) { my @arr = @_; #print Dumper(\%file_list); my @k = sort keys %file_list; for my $k (@k) { my @val = @{$file_list{$k}}; print "File: $k\n"; for my $r (@val) { print " \@ref{$$r[0]} -> \@ref{$$r[1]}\n"; } print "\n"; } } sub main() { my @arr = read_po(); #print_arr(@arr); mk_subst(); } main();___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
Hi Federico, I believe you are trying to automate a set of translations, correct? If so, here's a way to do it in perl that avoids all the shell convolutions. I assume you do know perl. If not, always worth knowing for this sort of quick work. Just add the translations to the hash table in the script. The virtue of this small tool is that it will tell you if you missed any. Reads file from stdin. Outputs to stdout. You can figure how to process the whole directory. There's a hundred ways to do this. Some people like to open all the files in the perl script. I prefer to keep it simple. Hope this may be useful for now and in the future. Andrew == snip #!/usr/bin/perl use strict; use warnings; my @ref; # translation table my %translations = ( 'Automatic beams' => 'Automatic Beams in Italian', 'Stems' => 'Stems in Italian', ); while (<>) { if (@ref = /\@ref\{(.+)\}/) { if (exists $translations{$ref[0]}) { s/$ref[0]/$translations{$ref[0]}/; } else { print STDERR "no translation for $ref[0]", "\n"; } } print; } == snip ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
> From: Federico Bruni> To: Dev > Date: Thu, 20 Apr 2017 08:20:26 +0200 > Subject: help with bash script to translate @ref{} items in translated > manuals > Hi all > > I recently realized that @ref{} links should be translated otherwise PDF > links are broken (while HTML files still work fine without @ref{} > translated). > > Yesterday I started to manually translate all the @ref{} instances in the > italian web, usage and learning manuals. > Now I've come to the notation manual, which has too many items. > > [notation (translation %)]$ grep -oh -e @ref{.*} *.itely | sort -u | wc -l > 257 > [notation (translation %)]$ grep -oh -e @ref{.*} *.itely | wc -l > 702 > > 257 different @ref and a total of 702 @ref > > Not all these 257 ref are translated (as translation of this manual is not > complete yet), but most are. > So I decided to try a bash script, but I have a problem with a regexp, > which works fine in the terminal but not in the bash script. > > Enter Documentation/it/notation and run this: > > #!/bin/bash > LIST="$(grep -oh -e @ref{.*} *.itely | sort -u)" > for i in $LIST; do >echo $i > #echo -n "Replace" $i "with the translated node: " > #read NODE > #if $NODE=""; then exit > #else > #sed "s|$i|$NODE|g" *.itely > #fi > done > > You'll get something like: > > @ref{Tuplets} > @ref{Turkish > classical > music} > @ref{Typesetting > Gregorian > chant} > > > The problem is the space character, which is not matched in the bash > script. > Why? > > Thanks in advance > Federico > Not sure if you'e found a solution in bash yet. As someone else mentioned, this is pretty straightforward in Perl: #!/usr/bin/perl # Usage: findRefs.pl # # Find all references, from the files specified on the command line, # where references are defined as the contents inside strings like "@ref{}". # # Print the sorted, unique contents. my $file ; my $contents ; my $match ; my @matches ; my %refs ; my $ref ; local $/ = undef; foreach $file ( @ARGV ) { open my $fh, '<', $file or die "Couldn't open file: $!" ; $contents = <$fh> ; close $fh ; (@matches) = ( $contents =~ /\@ref\{(.*)\}/g ) ; foreach $match (@matches) { unless ( defined $refs{$match} ) { $refs{$match} = 'defined' ; } } @matches = () ; } foreach my $ref (sort keys %refs) { print $ref . "\n" ; } If you save that to a file findRefs.pl and make it executable, you might invoke it as: findRefs.pl Documentation/it/notation HTH, David Elaine Alt 415 . 341 .4954 "*Confusion is highly underrated*" ela...@flaminghakama.com self-immolation.info skype: flaming_hakama Producer ~ Composer ~ Instrumentalist -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
Le 21/04/2017 à 17:59, Federico Bruni a écrit : Il giorno ven 21 apr 2017 alle 15:54, k...@aspodata.se ha scritto: You already have such an replacement file in Documentation/po/it.po: $ fgrep -C2 'Different editions from one source' Documentation/po/it.po #. @node in Documentation/notation/input.itely #. @subsection in Documentation/notation/input.itely msgid "Different editions from one source" msgstr "" Maybe that could be put to use. The translation stuff is quite a mess. IIRC those po files are used only to translate a few strings of the website, but most of the strings are useless. In fact they are empty and nobody complains about it :) It has been broken during the GDP process. I've a tarball of the 2.12.3 docs (January 2010) and we even had variables' name and comments translated in the @lilypond thanks to those po files. Cheers, Jean-Charles ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
Il giorno ven 21 apr 2017 alle 15:54, k...@aspodata.se ha scritto: It is expecting something from stdin. You didn't tell us where the replacement data comes from, so I guessed you run your script as cat repl.txt | your_script Actually in my first attempt I thought I'd have entered the replacement interactively during the loop. Now I've changed my mind: as I wrote in a previous email, I'll write the replacement file and then run a single script. ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
Il giorno ven 21 apr 2017 alle 15:54, k...@aspodata.se ha scritto: Federico Bruni: Il giorno ven 21 apr 2017 alle 7:08, Federico Bruniha scritto: > I guess I'll have to revert to my almost manual replacement I've used > so far. Or I may try a different approach and script a replacement from a file like: @ref{Different editions from one source},@ref{Edizioni diverse da un unico sorgente} @ref{Dimensions},@ref{Dimensioni} @ref{Direction and placement},@ref{Direzione e posizionamento} that replace first instance of @ref with the second instance (after the comma) You already have such an replacement file in Documentation/po/it.po: $ fgrep -C2 'Different editions from one source' Documentation/po/it.po #. @node in Documentation/notation/input.itely #. @subsection in Documentation/notation/input.itely msgid "Different editions from one source" msgstr "" Maybe that could be put to use. The translation stuff is quite a mess. IIRC those po files are used only to translate a few strings of the website, but most of the strings are useless. In fact they are empty and nobody complains about it :) I don't know if it is up to speed yet, but have a look at http://po4a.alioth.debian.org/man/man7/po4a.7.php There is some talk about texinfo and gettext in: https://www.gnu.org/software/texinfo/manual/texinfo/html_node/Internationalization-of-Document-Strings.html That manual is about texi2any, but we are still forced to use texi2html. I once tried to find some projects using texi2any and gettext. I could not find even one. Can't remember where or who I asked. I remember a person replied, but he was using the same method we are already using (diff), see: http://lists.gnu.org/archive/html/help-texinfo/2013-08/msg1.html ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
Federico Bruni: > Il giorno ven 21 apr 2017 alle 7:08, Federico Bruni >ha scritto: > > I guess I'll have to revert to my almost manual replacement I've used > > so far. > > Or I may try a different approach and script a replacement from a file > like: > > @ref{Different editions from one source},@ref{Edizioni diverse da un > unico sorgente} > @ref{Dimensions},@ref{Dimensioni} > @ref{Direction and placement},@ref{Direzione e posizionamento} > > that replace first instance of @ref with the second instance (after the > comma) You already have such an replacement file in Documentation/po/it.po: $ fgrep -C2 'Different editions from one source' Documentation/po/it.po #. @node in Documentation/notation/input.itely #. @subsection in Documentation/notation/input.itely msgid "Different editions from one source" msgstr "" Maybe that could be put to use. I don't know if it is up to speed yet, but have a look at http://po4a.alioth.debian.org/man/man7/po4a.7.php There is some talk about texinfo and gettext in: https://www.gnu.org/software/texinfo/manual/texinfo/html_node/Internationalization-of-Document-Strings.html Regards, /Karl Hammar --- Aspö Data Lilla Aspö 148 S-742 94 Östhammar Sweden +46 173 140 57 ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
Frederico Bruni: > Il giorno gio 20 apr 2017 alle 11:22, k...@aspodata.se ha scritto: > > Frederico Bruni: > > ... > >> Enter Documentation/it/notation and run this: > >> > >> #!/bin/bash > >> LIST="$(grep -oh -e @ref{.*} *.itely | sort -u)" > >> for i in $LIST; do > >> echo $i > >> #echo -n "Replace" $i "with the translated node: " > >> #read NODE > >> #if $NODE=""; then exit > >> #else > >> #sed "s|$i|$NODE|g" *.itely > >> #fi > >> done ... > > $ cat tt > > #!/bin/sh > > > > # for some reason mapfile doesnt work on pipes, hence the tmpfile > > grep -oh -e "@ref{.*}" *.itely | sort -u > tmpfile > > mapfile -t org < tmpfile > > > > len=${#org[@]} > > for (( ix=0; ix < $len; ix++ )) > > do > > read NODE > > printf "%4d: %s %s\n" $ix "${org[ix]}" "$NODE" > > done > > Unfortunately this script hangs forever. ... It is expecting something from stdin. You didn't tell us where the replacement data comes from, so I guessed you run your script as cat repl.txt | your_script Regards, /Karl Hammar --- Aspö Data Lilla Aspö 148 S-742 94 Östhammar Sweden +46 173 140 57 ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
Il giorno ven 21 apr 2017 alle 7:08, Federico Bruniha scritto: I guess I'll have to revert to my almost manual replacement I've used so far. Or I may try a different approach and script a replacement from a file like: @ref{Different editions from one source},@ref{Edizioni diverse da un unico sorgente} @ref{Dimensions},@ref{Dimensioni} @ref{Direction and placement},@ref{Direzione e posizionamento} that replace first instance of @ref with the second instance (after the comma) ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
Il giorno gio 20 apr 2017 alle 11:22, k...@aspodata.se ha scritto: Frederico Bruni: ... Enter Documentation/it/notation and run this: #!/bin/bash LIST="$(grep -oh -e @ref{.*} *.itely | sort -u)" for i in $LIST; do echo $i #echo -n "Replace" $i "with the translated node: " #read NODE #if $NODE=""; then exit #else #sed "s|$i|$NODE|g" *.itely #fi done This doesn't work: $ LIST="$(grep -oh -e @ref{.*} *.itely | sort -u)" $ echo $LIST This is better but messes with newlines: $ LIST=$(grep -oh -e "@ref{.*}" *.itely | sort -u) $ echo $LIST @ref{Accidentals} @ref{Align} @ref{Aligning lyrics to a melody [...] The "for i in $LIST" also messes with newlines. This will rid you of the newline problem: $ grep -oh -e "@ref{.*}" *.itely | sort -u | while read i; do echo $i; done @ref{Accidentals} @ref{Align} ... but now "read NODE" won't read from script stdin. Trying with /dev/stdin doesn't help since stdin is from the pipe, not the script stdin. yes, this is a problem... $ grep -oh -e "@ref{.*}" *.itely | sort -u | while read i; do echo $i; read NODE < /dev/stdin; echo $b; break; done @ref{Accidentals} @ref{Align} you culd solve that with arrays: $ cat tt #!/bin/sh # for some reason mapfile doesnt work on pipes, hence the tmpfile grep -oh -e "@ref{.*}" *.itely | sort -u > tmpfile mapfile -t org < tmpfile len=${#org[@]} for (( ix=0; ix < $len; ix++ )) do read NODE printf "%4d: %s %s\n" $ix "${org[ix]}" "$NODE" done Unfortunately this script hangs forever. I guess I'll have to revert to my almost manual replacement I've used so far. $ ./tt | head -5 a d 0: @ref{Accidentals} a d cd g 1: @ref{Align} cd g gv 2: @ref{Aligning lyrics to a melody} gv xcvb rge 3: @ref{Aligning objects} xcvb rge rtdfg gr 4: @ref{Ambitus} rtdfg gr 2 $ you could also do the for loop like: for i in "${org[@]}" do read NODE echo $i $NODE done Regards, /Karl Hammar --- Aspö Data Lilla Aspö 148 S-742 94 Östhammar Sweden +46 173 140 57 ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
Hi All, All good answers, but so much easier in perl! Andrew ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
Frederico Bruni: ... > Enter Documentation/it/notation and run this: > > #!/bin/bash > LIST="$(grep -oh -e @ref{.*} *.itely | sort -u)" > for i in $LIST; do > echo $i > #echo -n "Replace" $i "with the translated node: " > #read NODE > #if $NODE=""; then exit > #else > #sed "s|$i|$NODE|g" *.itely > #fi > done This doesn't work: $ LIST="$(grep -oh -e @ref{.*} *.itely | sort -u)" $ echo $LIST This is better but messes with newlines: $ LIST=$(grep -oh -e "@ref{.*}" *.itely | sort -u) $ echo $LIST @ref{Accidentals} @ref{Align} @ref{Aligning lyrics to a melody [...] The "for i in $LIST" also messes with newlines. This will rid you of the newline problem: $ grep -oh -e "@ref{.*}" *.itely | sort -u | while read i; do echo $i; done @ref{Accidentals} @ref{Align} ... but now "read NODE" won't read from script stdin. Trying with /dev/stdin doesn't help since stdin is from the pipe, not the script stdin. $ grep -oh -e "@ref{.*}" *.itely | sort -u | > while read i; do echo $i; read NODE < /dev/stdin; echo $b; break; done @ref{Accidentals} @ref{Align} you culd solve that with arrays: $ cat tt #!/bin/sh # for some reason mapfile doesnt work on pipes, hence the tmpfile grep -oh -e "@ref{.*}" *.itely | sort -u > tmpfile mapfile -t org < tmpfile len=${#org[@]} for (( ix=0; ix < $len; ix++ )) do read NODE printf "%4d: %s %s\n" $ix "${org[ix]}" "$NODE" done $ ./tt | head -5 a d 0: @ref{Accidentals} a d cd g 1: @ref{Align} cd g gv 2: @ref{Aligning lyrics to a melody} gv xcvb rge 3: @ref{Aligning objects} xcvb rge rtdfg gr 4: @ref{Ambitus} rtdfg gr 2 $ you could also do the for loop like: for i in "${org[@]}" do read NODE echo $i $NODE done Regards, /Karl Hammar --- Aspö Data Lilla Aspö 148 S-742 94 Östhammar Sweden +46 173 140 57 ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: help with bash script to translate @ref{} items in translated manuals
Federico Bruniwrites: > #!/bin/bash > LIST="$(grep -oh -e @ref{.*} *.itely | sort -u)" > for i in $LIST; do This splits $LIST at separators contained in the shell variable IFS . One can temporarily override it to just split on newlines, but it is more straightforward to write grep -oh -e @ref{.*} *.itely | sort -u | while read i; do rather than tamper with shell syntax temporarily. >echo $i > #echo -n "Replace" $i "with the translated node: " > #read NODE > #if $NODE=""; then exit > #else > #sed "s|$i|$NODE|g" *.itely > #fi > done > > You'll get something like: > > @ref{Tuplets} > @ref{Turkish > classical > music} > @ref{Typesetting > Gregorian > chant} > > > The problem is the space character, which is not matched in the bash > script. > Why? > > Thanks in advance > Federico > -- David Kastrup ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel