Re: [Amforth] Reference Card page now missing

[Mark Roth]

Okay, I got a little bit of a chance to play with this tonight. I wanted to
hack out the $prevline variables and forcibly replace them with the actual
beginning lines from the file. This then shows all the asm files in
common/words that have issues that need to be dealt with. It's at best a
hack before really fixing the script, but I wanted to see if that was how
those prevlines were working. I'll attach the hacked up make-refcard-rst
below. Once it is in place it can be run after which building the htdocs
with make htdocs will create the new refcard.html file. As you can see
there are a number of errors. Looking at the common/asm files shows them to
be ones with non-conformant comment lines at the top. The files with the 3
comment lines at the top, even if they have the ".if cpu_" stuff in them
now generate correctly though. I'll try to make more progress during the
week, but bear in mind that I've never looked at a Perl script before a day
or two ago. :P

make-refcard-rst == START

#!/usr/bin/perl
use strict;

# local hashes
my %XT;
my %VOC;
my %ASM;
my %USEDBY;
my %DESCRIPTION;
my %DSTACK;
my %RSTACK;
my %CSTACK;
my %CATEGORY;
my %TYPEOF;

# current version. There should be a version variable
# stored somewhere in the source tree that could be pulled
# by any of the build tools. TODO.
my $version="6.9";

# relative location of refcard.rst output file for building
my $texdir="../doc/source/TG";

# location of the core words
# currently the core .asm files are parsed incorrectly
# due to adding various platforms. The platform ones work.
# TODO: add platform specific output to refcard.
#my $asmdir="../avr8/words";
my $asmdir="../common/words";

sub readASM {
my ($filename) = @_;
open(ASM, $filename) or die ("$filename: $!\n");
my @ASM = ;
close(ASM);
my $ASM = "";
my ($lbl, $state, $voc, $xt, $dstack, $rstack, $cstack, $category,
$typeof);
my ($prevline1, $prevline2, $prevline3, $description) = ("","","", "");

# Added to try and remove the prevline issues
# before fixing correctly.
my $line1 = $ASM[0];
my $line2 = $ASM[1];
my $line3 = $ASM[2];

# From this point on all prevline vars are now:
# prevline3 is now $line1
# prevline2 is now $line2
# prevline1 is now $line3

foreach my $line (@ASM) {
chomp($line);
#
next if $line=~/\.if/;
$line =~ s/_VE_HEAD/VE_HEAD/;
$ASM .= $line;
if($line=~/^VE_(.*):/) {
   # start a new definition
   $ASM = "";
   $lbl = "XT_$1";
   $state = "new_header_found";
   $voc   = "(unnamed)";
   $category = "unclassified";
   $dstack = "( -- )";
   $dstack = "($1)" if $line1=~/\([S]?:?([^\)]+)/;
   $rstack = "";
   $rstack = "(R: $1)" if $line1=~/R:\s+([^)]+)\)/;
   $cstack = "";
   $cstack = "(C: $1)" if $line1=~/C:\s+([^)]+)\)/;
$description = $1 if $line3=~/^;(.*)/;
   if( $line2=~/;(.+)$/) {
   $category = $1;
   }
#print "***\n$prevline3\n$prevline2\n$prevline1\n$line\n***\n";
#print "$prevline3 : $category\n";
#die;
   next;
}
if($line=~/^;VE_(.*):/) {
   # start a new definition
   $ASM = "";
   $lbl = "XT_$1";
   $state = "new_header_found";
   $voc   = "(hidden)";
   $dstack = $line1;
   $dstack = "( -- )";
   $dstack = "($1)" if $line1=~/\([S]?:?([^\)]+)/;
   $rstack = "";
   $rstack = "R($1)" if $line1=~/R:\s+(.+)\)/;
   $cstack = "";
   $cstack = "(C: $1)" if $line1=~/C:\s+(.+)\)/;
$description = $1 if $line3=~/^;(.*)/;
   if( $line2=~/;(.+)$/) {
   $category = $1;
   }
   $category = "internal/hidden";
   next;
}
if($state =~ /new_header_found/ && $line=~/.dw\s*(.*)/) {
   $state = "new_voc_header";
   next;
}
if ($state =~ /new_voc_header/ && $line=~/.db\s*(.*)/) {
   my @voc = split/,/, $1;
   my $i=0;
   $voc = "";
   foreach my $v (@voc) {
# next if $i++ == 0;
print "[$v]";
$voc .= chr(hex($1)) if $v=~/\$([\da-fA-F]+)/;
$voc .= $1 if $v=~/"(\S+)"/;
   }
   $state = "vocabulary entry found";
   next;
}
if($line=~/^XT_(.*)/){
   $state = "xt_found";
   next;
}
if($state=~/xt_found/ && $line=~/.dw\s+(\w+)/) {
   $xt = $1;
   $state = "header_complete";
   next;
}
if($state =~ /header_complete/) {
   $DSTACK{$lbl} = $dstack;
   $RSTACK{$lbl} = $rstack;
   $CSTACK{$lbl} = $cstack;

   $XT{$lbl} = $xt;
   $VOC{$lbl} = $voc;
$DESCRIPTION{$lbl} = $description;
   push @{$CATEGORY{$category}}, $lbl;
   $state = "parsing_body";
   next;
}
$prevline3 = $prevline2;
$prevline2 = $prevline1;
$prevline1 = $line;

if($state =~ /parsing_body/) {
   $ASM{$lbl} = $ASM if $ASM=~/\w/;
}
}
}

sub _head {
my ($title) = @_;
my ($r);
open(I, "refcard-head.rst") or die "refcard header not found";
while() {
  s/\*VERSION\*/$version/g;
  $r .= $_
}
close(I);
return $r;
}

sub _foot {
}

sub printLaTeX {
my ($title) = @_;
open(LATEX, ">$texdir/refcard.rst") or die "$!\n";;
print LATEX _head($title);
foreach my $category (sort keys %CATEGORY) {
 

Re: [Amforth] Reference Card page now missing

[Mark Roth]

Actually not really impressive, yet at least. Mostly I just changed the
locations for:

my $texdir="../doc/source/TG";
#my $asmdir="../avr8/words";
my $asmdir="../common/words";

so they pointed at the right place to be built. Then I yanked some old asm
files from release/5.1 to find out why the new ones were failing. Had I
tried the avr8 ones first I could have skipped that step. Pretty much from
that point I just put one file that worked and one that didn't next to each
other in common/words (the existing 2swap.asm and the old one of the same
name but with a "2" added to the end of the filename and inside the file).
Then I'd fire off make-refcard-rst and make htdocs from their respective
locations. Examine output, look up Perl term, lather rinse repeat. If I
manage to make some actual progress with the Perl file I'll post it here.


> I'm still fairly fluent in perl, so I expect this can be fixed
> to produce a refcard again. And yes I also expect to fix up the
> source .asm files. Regarding those headers: IFF the files were
> indeed generated from forth code, then I would like to include
> said forth code in the comment section, and add a line, how it
> was generated (I think this is a must for generated files
> anyhow).
>

This bit is what I was really looking for advice on. After playing around
with g4 a bit to find out how to include my own files in the hex build I
realized that the asm files in common/words and avr8/words look very
similar to g4's output. Maybe they were just generated from AmForth's
compiler? In any case, if they are end-of -he-road files and not something
repeatedly regenerated, fixing them to be compliant to whatever they should
be would be easy enough.  The format that the avr8 ones use was what the
older versions had as well. That is to say, Three comment lines in order at
the top. Then whatever else will be added as far as comments go, finally
followed by the actual code. This is what I'm spelunking in the script for.
It most certainly needs to be much more robust. I'll almost certainly need
a hand to do multiple directories though.

So the takeaway would be to define a format of the first 3 lines having to
be how they are, or some other way to tag the stack--effect, category,
description and processor flavor. The first way is easier and the first 3
lines could just be jammed into what is now this clever, but touchy setup.

$prevline3 = $prevline2;
> $prevline2 = $prevline1;
> $prevline1 = $line;


That is why everything is broken.  It almost seems if the idea was to be
able to have the 4 lines of comment, comment, comment, VE_NAME: be able to
be placed anywhere. Almost everything in the readASM subroutine keys off of
those. The only thing that does any other work is finding the actual name
from the .db "name",0 line, making sure everything closes, then shoving the
whole mess into a hash to be dealt with. At least I think that is what is
going on.

The decision to make is what should the header of the asm file have in it
and make them all conform. Like I said, I'll keep at it this week and post
here if I have anything helpful to add. As far as putting all the Forth
code in comments of the asm file, I don't know. Wouldn't it be better to
just have a forth source file somewhere in the tree? The asm files are nice
and tight and easy to read. If there is a bunch of extra info in there it
could make it messier to look through. I'm thinking more like when you have
a .c file and an .o file next to each other. They don't hurt each other but
since one can be generated from the other it would be easy enough to
automate the whole thing if there was a newer assembler and you wanted to
regenerate all of the asm files. Come to think of it, I guess that is where
the windows exe comes in to play.

In any case, define the header for the asm and I'll help out wherever (or
whenever) I can and have the time.

Mark

___
Amforth-devel mailing list for http://amforth.sf.net/
Amforth-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amforth-devel

Re: [Amforth] AmForth Weekend 1 (2020-06-27/28)

[Ian Jefferson]

Thanks for the thought stream Erich.

I have one other wacky idea for the list.  Maybe for next year when life in 
theory will settle or possibly over the 2020-2021 winter.  

I’d like to re-engineer a device called the CNC FROG.  It was originally a kind 
of 1.5 or 2.5 axis CNC gizmo but really it was just a flexible stepper motor 
controller and driver built into a single quite ingenious UI.  I think it sold 
for around $USD 150 with some bits of hardware for Sherline and Taig lathes.  
You could do ordinary turning and threading with it.

As it turned out though what it was really good at was just driving random bits 
of machinery via stepper motor.  The UI (a generous term) was very clever and 
flexible so you could configure it to operate in any units you liked after a 
short setup period.  So you could advance a machine in thousands of an inch of 
hundredths of a millimeter or fractions of degrees or radians or anything else 
you could cook up.  This flexibility had it selling into light industry for 
operating turntables, indexers, etc.

There were and are lots of solutions for the very computer literate but this 
was for the illiterate.

The flavour of the FROG reminds me of forth somewhat.  Small, fast, flexible 
and a bit obscure.   A niche in other words.

The project would need a new hardware platform - probably an open source hobby 
platform with PCB’s and assembled gizmos with options and cases which would be 
fun but the key functionality would be the firmware.  Developing a small, 
simple, elegant UI I suspect would be very controversial and difficult.  There 
is something of a template in the original but much could be done to introduce 
more elegance.  

I’d try to fund the project so it would be a paying proposition for major 
participants.  

If you are interested in the FROG CNC it is hard to find now.  This might get 
you started.  http://www.cartertools.com/frog.html 



> On Jun 28, 2020, at 10:29 AM, Erich Wälde  wrote:
> 
> - Whacky Ideas
> 
>  - git? -- With all the cool kids using git repositories, should
>I attempt do convert the existing repositories, webpage, etc?
>does sourceforge.net  provide git repositories? 
> Can the
>existing svn repository be converted on the server side?
> 

sourceforge is OK but github seems easier to use to me.  The thing that occurs 
to me is perhaps in github we would get more exposure?  I’m so out of date.

>  - Should we use a ticket system rather than mailing list?
> 
>  - Who of you is using which target controller? Would it be
>feasible to drop msp430, arm, risc-v in order to simplify the
>whole thing? yes/no?

I’m Atmel but have not touched embedded really for a few years.  Life needs to 
settle down. Will it?  I don’t know. 

> 
>  - Can we get rid of the Atmel/Microchip Avrasm Assembler?
> 
>One big difference between the avr8 and the risc-v tree is
>the assembly language. avr8 is using Atmels assembly. Which
>is good, because it is thoroughly documented. And which is
>bad, because there is no working free/libre alternative to
>avrasm.exe. Yes there is the "avra" project but it has been
>abandoned long ago. I have been able to assemble AmForth with
>avra way back in releases 4.2 up until maybe 4.9. I have even
>contributed a small patch to make atmega644p working.
>https://sourceforge.net/projects/avra/ 
> 
> 
>Matthias has contemplated the idea to port AmForth/avr8 to
>use gnu assembly. He might even have produced a working
>branch, I don't know.
> 
>For risc-v there is no avr assembly, naturally. That's where
>all the .s assembly files come into the game.
> 
>I personally would love to have a free/libre assembler for
>avr assembly. AVRASM.EXE is the only thing that forces me to
>install wine on my system.

I agree.  I’m sorry that the opensource AVR assembler has gone by the wayside 
also.  I used it in early days.  I’m somewhat surprised that it is in suspended 
animation given the popularity of Arduino as a platform.   I find 
virtualization a big complicated hammer to solve problems that could be 
resolved in a more elegant manner.  Simplification isn’t very popular today 
though.


Ian - lurking on.   



___
Amforth-devel mailing list for http://amforth.sf.net/
Amforth-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amforth-devel

Re: [Amforth] Reference Card page now missing

[Erich Wälde]

Hello Mark,

short answer: I'm impressed!

I'm still fairly fluent in perl, so I expect this can be fixed
to produce a refcard again. And yes I also expect to fix up the
source .asm files. Regarding those headers: IFF the files were
indeed generated from forth code, then I would like to include
said forth code in the comment section, and add a line, how it
was generated (I think this is a must for generated files
anyhow).

So, if you would please be so kind as to send your changed
script along the mailing list, that would be grand!

Thank you very much for looking into this.
And then my secret plan also worked out: give the folks
something to chew on :-)))

Cheers,
Erich

Mark Roth writes:

> Hopefully this will get sent to the right place...
>
> I have been poking around the Perl script that creates the Refcard and
> managed to get it to spit out a new (well, not quite but...) refcard.rst
> which can then be build into the webpage when using make to create the
> htdocs. It works if I change the asm source directory to the avr8 one but
> not with the common/words. The problem is that the Perl regexes are so
> tightly bound with the file having 4 line in exactly the right place. It
> looks for the "VE_" + name + ":" line then uses the three prior lines to
> fill the name, stack effect and description. The files in the common/words
> don't have the same layout since they contain the ".if cpu_" etc lines to
> handle the msp430 and avr8 flavors.
>
> Now for the good news. It seems that every file in the common/words has a
> variant for both the msp430 and avr8. Any of those could just be
> tagged with a new hash called %TYPEOF or something to that effect. Then the
> avr8/words could be parsed since they have the same top 3 lines and be
> tagged for the avr8. As I mentioned, those already work just fine. I've
> created a new refcard.html for those. Now, the msp430 is a whole other bag
> of worms. The information that is being parsed is all mashed up in the
> first line and doesn't contain a section name.
>
> So, to start out with. The msp430 asm files could be fixed by hand to have
> the proper information in the top 3 lines, but only if they are not
> generated somehow that will undo the work. Perhaps whatever spits out those
> asm files could be convinced to put that info there? I have no idea. I know
> that with g4 I could easily convince it to place those 3 lines as needed
> just by putting them as comments in the file. It works quite nicely. So,
> since I don't know anything about the 430 stuff I'll leave that for now.
>
> The bigger issue is the way the asm files are searched to create the
> refcard. That could be fixed without too much of a problem I think. Well,
> apart from having to figure out the right place to add in for the new type
> thing. Perl is Perl. I didn't have any idea about how to read it before
> this and at least can now follow the internal logic. It reminds me of a
> project to find solutions from every permutation of some packing puzzles I
> made. I wrote one bash script that fed off another and another etc. It
> works, but I don't hardly dare breathe too hard on it. ;)
> This has the same feel. It works but only because it works. I can keep
> playing with it this week and try to make some progress on it. Perl is easy
> enough to pick up enough. However, if I spoke Python I'd probably just
> rewrite the whole mess and fix the comments in the common/words to conform
> with the rest of them. Most of it is just scut work with the addition of
> tagging the crossover words with their proper flavor.
>
> So, if anyone is still this far along. I can use the scripts to create the
> stuff to make the htdocs for the site and the epub file. I tried to do the
> pdf but after adding package after package after package of the texlive
> stuff I gave up and just wrote a new rule to create the epub but not the
> pdf. It shouldn't matter except I can't test the pdf output. I will gladly
> continue to pursue this as I do find Perl interesting enough to muck with.
> It's pretty amazing apart from the fact that it is like reading a book that
> is printed entirely in stereogram.
>
> That's that for now. It seems that AmForth Weekend 1 is still bearing fruit
> (such as it is).
>
> Mark
>
> ___
> Amforth-devel mailing list for http://amforth.sf.net/
> Amforth-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/amforth-devel


-- 
May the Forth be with you ...


___
Amforth-devel mailing list for http://amforth.sf.net/
Amforth-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amforth-devel

Re: [Amforth] Reference Card page now missing

[Mark Roth]

Hopefully this will get sent to the right place...

I have been poking around the Perl script that creates the Refcard and
managed to get it to spit out a new (well, not quite but...) refcard.rst
which can then be build into the webpage when using make to create the
htdocs. It works if I change the asm source directory to the avr8 one but
not with the common/words. The problem is that the Perl regexes are so
tightly bound with the file having 4 line in exactly the right place. It
looks for the "VE_" + name + ":" line then uses the three prior lines to
fill the name, stack effect and description. The files in the common/words
don't have the same layout since they contain the ".if cpu_" etc lines to
handle the msp430 and avr8 flavors.

Now for the good news. It seems that every file in the common/words has a
variant for both the msp430 and avr8. Any of those could just be
tagged with a new hash called %TYPEOF or something to that effect. Then the
avr8/words could be parsed since they have the same top 3 lines and be
tagged for the avr8. As I mentioned, those already work just fine. I've
created a new refcard.html for those. Now, the msp430 is a whole other bag
of worms. The information that is being parsed is all mashed up in the
first line and doesn't contain a section name.

So, to start out with. The msp430 asm files could be fixed by hand to have
the proper information in the top 3 lines, but only if they are not
generated somehow that will undo the work. Perhaps whatever spits out those
asm files could be convinced to put that info there? I have no idea. I know
that with g4 I could easily convince it to place those 3 lines as needed
just by putting them as comments in the file. It works quite nicely. So,
since I don't know anything about the 430 stuff I'll leave that for now.

The bigger issue is the way the asm files are searched to create the
refcard. That could be fixed without too much of a problem I think. Well,
apart from having to figure out the right place to add in for the new type
thing. Perl is Perl. I didn't have any idea about how to read it before
this and at least can now follow the internal logic. It reminds me of a
project to find solutions from every permutation of some packing puzzles I
made. I wrote one bash script that fed off another and another etc. It
works, but I don't hardly dare breathe too hard on it. ;)
This has the same feel. It works but only because it works. I can keep
playing with it this week and try to make some progress on it. Perl is easy
enough to pick up enough. However, if I spoke Python I'd probably just
rewrite the whole mess and fix the comments in the common/words to conform
with the rest of them. Most of it is just scut work with the addition of
tagging the crossover words with their proper flavor.

So, if anyone is still this far along. I can use the scripts to create the
stuff to make the htdocs for the site and the epub file. I tried to do the
pdf but after adding package after package after package of the texlive
stuff I gave up and just wrote a new rule to create the epub but not the
pdf. It shouldn't matter except I can't test the pdf output. I will gladly
continue to pursue this as I do find Perl interesting enough to muck with.
It's pretty amazing apart from the fact that it is like reading a book that
is printed entirely in stereogram.

That's that for now. It seems that AmForth Weekend 1 is still bearing fruit
(such as it is).

Mark

___
Amforth-devel mailing list for http://amforth.sf.net/
Amforth-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amforth-devel