Re: [Tinycc-devel] trying to make absolute jump

[grischka via Tinycc-devel]

On 22.04.2024 06:16, Paul Moore wrote:

Weird processor. It only supports absolute jumps. So this is something that 
needs to be fixed up during link time. I do not know the elf details well 
enough to be able to work out how to do it

Seems like I need the jmp instruction to be a have reloc that refers to a 
symbol that matches the jump destination, say “dest1”. So I need a reference to 
Symbol “dest1” at the jump site and a definition generated later (by 
gsym_addr). Can I even do that , have a reference to a symbol that I export but 
want resolved at link time and not have ELF go “well I know that symbol, its 
right here”


Hi,

certainly it's easily possible, however assuming this 16-bit 'hack'
computer/processor doesn't really have an operating system capable
to load ELF files and hence assuming that you probably want to go
with the -Wl,-oformat=binary option, you can get away without relocation
at all.

You'd simply replace the relative patch in gsym_addr()
write32le(ptr, a - t - 4);
by an absolute one, for example
write16le(ptr, a + LOAD_ADDRESS);

where LOAD_ADDRESS obviously is where the program is supposed to be
loaded.

By the way, if you prefer to start hacking with something more
simple, maybe you want to have a look at tinycc's initial revision

https://repo.or.cz/tinycc.git/commitdiff/initial

which already can compile itself, almost at least. (IIRC parsing
for 0x... hex numbers needs to be added and the #ifdefs need to
be removed.  It can handle "#define SYM VAL" though.  Also the
vac and/or vat table should be initialized with '0', IIRC)

-- gr



I thought that maybe computed goto offered a way to prise the lid off this, but 
no , that doesn’t not work the same way.




___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] TinyCC on Windows & some suggestions

[grischka via Tinycc-devel]

On 11.03.2024 20:38, Robert Schlicht wrote:
> Yes, it is. But it’s really primitive, basically just a text editor that has 
the compiler integrated, and calling it an IDE may be an exaggeration. It’s the 
thing that is intended to make writing a “Hello, World!” program (and slightly 
more interesting stuff) as painless as possible for beginners.

But still really sleek, how it's "self-hosted", in a surprising way...

I pushed some changes based on your suggestions:
https://repo.or.cz/tinycc.git/commitdiff/2b0a663df9236763c3967575e2c0fb89a9e58566

-- gr

>

Robert


Jake Anderson (2024-03-11 15:10):

Is the IDE open source? An IDE that is packaged separately and uses the TCC
compiler could be useful.

On Sun, Mar 10, 2024 at 1:01 PM Robert Schlicht  wrote:


At our university we offer a course where we program simple spatial
simulations in various programming languages, one of them being C, for
illustrating close-to-the-machine programming concepts. We here need a C
implementation that is small (since it’s accessed over a network), works
out of the box on Windows computers (since our students are beginners) and
runs fast (so compiler errors are available instantaneously). We do not
need advanced developer tools, and code running three times slower is
acceptable because that is still faster than scripting languages.

TCC is obviously a good option here, and for our course starting in April
of this year, I put together a package https://rschlicht.eu/tc-ide.zip
that includes a minimalist IDE running TCC and a very basic form of a C
standard library, all contained in a standalone executable tc-ide.exe. The
library is just headers that directly access the Windows API (no runtime
needed) and should satisfy the requirements of a conforming freestanding
implementation, while also including common memory, file, math and the
printf family of functions. (If anyone finds this useful, I’ll gladly
contribute it to the TCC project.)

The executable is compiled by itself, although this currently requires a
few hacks and workarounds to get it working as desired. I list these here
as suggestions for improving TCC:

(1) For using TCC as a library, it would be nice if it did a more thorough
cleanup:
– In a few places exit() is called in case of failure, but terminating the
program is not very user-friendly; cleanly propagating failure or even some
longjmp hacks might provide a better solution. [tc-ide does the latter,
while patching function calls to keep track of memory and open files.]
– Another problem I encountered is that TCC does not always properly
restore the state of the global variables; compiling the following code
fragment the first time produces an error message (as it should), but the
second time it causes an exception (which I assume is a bug):
 void nothing(void) {for ( ; ; ) break;}  void garbage(void) {switch
[The workaround in tc-ide is ugly but straightforward: Make a copy of the
memory block containing all global variables, and restore this block after
TCC returns.]

(2) I really appreciate that TCC can directly link to functions in Windows
DLLs with no auxiliary .lib file and that it even supports directives like
#pragma comment(lib,"kernel32"). The current implementation of the DLL
lookup with a huge number of lseek & read calls (via read_mem() in tccpe.c)
may be inefficient on some file systems. [tc-ide avoids this issue by
creating file mappings in memory and redirecting lseek and read to those
memory buffers, which it has to deal with anyway to access the embedded
headers.]

(3) The C23 preprocessor directive #embed would be of help for embedding
headers and other files as byte arrays in the program. [tc-ide currently
does this by providing a non-standard feature with a custom notation like
#include "stdlib.h#".]

(4) TCC uses fixed buffer sizes for file paths in certain places. For
example, libtcc.c has 260(=MAX_PATH) in config_tccdir_w32() and
_fullpath(), 1000 in tcc_add_systemdir() and 1024 in
tcc_add_library_internal(), while tccelf.c has 1024 in getcwd(). Windows
has been supporting long file paths for quite a while now, so it might be
better to allocate those buffers dynamically:
https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation

(5) Some rarely used C library functions could perhaps be replaced to make
the code less dependent on such features. Examples are the single use of
alloca() in libtcc.c to set up a buffer and the use of scanf() in tccpp.c
to convert the TCC version string into a number. [tc-ide here provides
stubs.]

(6) It would be useful to allow the user to set the entry point symbol
(either the one called by the OS or the one called by the startup code),
like other compilers do. [tc-ide provides its own version of _start(),
which simply calls main().]

(7) Additional observations:
– In tcc_new() (tcclib.c), checking the return value of tcc_mallocz is
probably redundant.
– In tcc_close() (tcclib.c), I do not understand why the test is ">0"

Re: [Tinycc-devel] Minimizing libtcc memory use

[grischka via Tinycc-devel]

On 08.03.2024 07:30, Eric Raible wrote:

I guess that I just want the numbers to add up.
Using your example:

1) -DMEM_DEBUG -DCONFIG_RUNMEM_RO=0
2) your test.c
3) but I added an early return to tcc_delete() to no-op it

Running: valgrind tcc -nostdlib -vv -bench -run test.c
produced:

tcc version 0.9.28rc 2024-03-03 mob@9d2068c6* (AArch64 Linux)
-> test.c
-
0: .text0x4ccb000  len 00020  align 1000
1: .data.ro 0x4ccb020  len 4  align 0008
2: .data0x4ccb028  len 4  align 0008
2: .bss 0x4ccb030  len 4  align 0008
-
protect rwx 0x4ccb000  len 01000
-
# 3030 idents, 4 lines, 92 bytes
# 0.463 s, 8 lines/s, 0.0 MB/s
# text 32, data.rw  4, data.ro  4, bss 4 bytes
# memory usage: 8192 to run, 649 symbols, 2901 other, 1639290 max (bytes)
mem_cur_size=11742 (bytes)



So tcc_print_stats() says 11742, but then displays values totaling 11786.


What 11786 ?

8192 + 649 + 2901 = 11742

> ==2188==
> ==2188== HEAP SUMMARY:
> ==2188== in use at exit: 14,814 bytes in 32 blocks
>
> And valgrind reports 14814.  I have never seen valgrind wrong about this,
> and especially so b.c. my (luckily-correct) allocator reported 14814 as well.

There is nothing wrong here:  11742 + 32 blocks * 96 = 14814
Where 96 is the size of tcc's mem_debug extra header.

If you want to see the 11742 from valgrind then you just need to
run the same example with a normal tcc compiled without MEM_DEBUG.

Which makes sense I would think.

But when showing the example with MEM_DEBUG and -bench -vv  I
did not expect you to doubt the numbers in the first place.

Rather I just was trying to show how you could get some numbers
for your own real case instead.  Which as you suspect could be
minimized from 29kB down to 1-2 kB.  Most likely impossible but
if we had some numbers we could tell also why.

-- gr


___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Minimizing libtcc memory use

[grischka via Tinycc-devel]

On 03.03.2024 21:26, Eric Raible wrote:

 > isn't there a garbage collecting done at the end to remove all the unused 
stuff
 > to produce a binary that contains only the necessary parts ?

That very well might be the case, but given that tcc_get_symbol()
can be used at any time between tcc_relocate() and tcc_delete(),
it follows that _at least_ symbols are resident in the TCCState.
What I'm wondering about is the feasibility of keeping just code and
data, and flushing everything else.  This would require a new API -
something like tcc_finalize(TCCState *) or perhaps
tcc_finalize(TCCState *, flags), where flags specify what to flush.


Me thinks this discussion is going around in circles.

Your tcc_finalize() subject of obsession was what we already had
all the time since 0.9.25, that is the option to allocate the
executable code separately from the state and therefor the option
to delete (finalize) the state and still keep the code.

It just happened that at some point people found that tcc not only
should allow them to have the cake for their own, but that they
also want tcc to provision them with some extra convenience to
cleanup with it (including unprotect and release of the unwind
table on win64).

At which point I found that maintaining two options for running
code might not be to the best for both sides, neither for tcc wrt.
maintainability nor for users wrt. avoidance of inconvenience.

Of course now that we have that simpler API, it was clear that
soon someone would come up suspecting some benefit if it were not
only simple but complicated too, optionally of course.

Where simple plus complicated is more complicated than just
complicated btw.

And while it's completely unclear what such benefit could be,
under what scenario.


I don't know enough about the internals, but if I'm willing to run with
CONFIG_RUNMEM_RO, it seems like the per TCCState memory use in my case
could be decreased from something like 29K to 1K or 2K.

I should mention that the memory usage in my case is 29K regardless
of whether CONFIG_RUNMEM_RO is 0 or 1.


How do you know this?

Some output from tcc -nostlib -vv -bench -run test.c
(tcc compiled with -DMEM_DEBUG -DCONFIG_RUNMEM_RO=0)

  int x = 1; //.data
  int y; // .bss
  const int z; // .rdata
  int _start() { return 0; } //.text

tcc version 0.9.28rc 2024-03-04 mob@d4f7fcbb* (i386 Windows)
-> test.c
-
memory  007B6390  len 02000
-
0: .text007B7000  len 00018  align 1000
1: .rdata   007B7040  len 4  align 0040
2: .data007B7044  len 4  align 0004
2: .bss 007B7048  len 4  align 0004
-
protect rwx 007B7000  len 01000
-
# 1074 idents, 6980 lines, 137488 bytes
# 0.001 s, 698 lines/s, 137.5 MB/s
# text 17, data.rw 4, data.ro 4, bss 4 bytes
# memory usage: 10250 bytes (8192 to run, 499 for 14 symbols, 1559 for state 
etc)

Which is 8kB to run and 2kB extra.

Same for -run tcc.c
-
memory  00E159D8  len 63000
-
0: .text00E16000  len 43418  align 1000
1: .rdata   00E59440  len 08c00  align 0040
2: .data00E62040  len 00311  align 0004
2: .bss 00E62358  len 15938  align 0008
-
protect rwx 00E16000  len 44000
-
# 29262 idents, 99132 lines, 3377368 bytes
# 0.202 s, 490752 lines/s, 16.7 MB/s
# text 270677, data.rw 4, data.ro 35789, bss 88112 bytes
# memory usage: 413204 bytes (405504 to run, 6030 for 158 symbols, 1670 for 
state etc)

Where the 4 bytes initialized data.rw btw. is Eric Raible's
static void *(*reallocator)(void*, unsigned long) = default_reallocator;

-- gr



Thanks - Eric



___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] TCC compiling TCC

[grischka via Tinycc-devel]

On 02.03.2024 20:15, Jake Anderson wrote:

When compiling TCC using TCC, there is an error about missing _strtoui64 in 
msvcrt.dll on Windows 2000 and older. This does not occur when using old 
versions of GCC to compile TCC.


Maybe you want to replace
in tcc.h
#  define strtoull _strtoui64
by
#  define strtoull strtoul

You just cannot use 64-bit numbers in the assembler then.

-- gr

___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] tcc assembler emits wrong pc-relative symbol difference reloctions in immediates

[grischka via Tinycc-devel]

On 01.03.2024 17:32, Rich Felker wrote:

The relocation emitted is the pc-relative address based on the
beginning of the add opcode, but it needs to be based on the beginning
of the immediate operand in the add opcode.


I just moved the addition of (ind + 4)

https://repo.or.cz/tinycc.git/commitdiff/ca061f3a96216f85f6fe72868083208aa0a740b9#patch2
to where the operand is put.

https://repo.or.cz/tinycc.git/commitdiff/ca061f3a96216f85f6fe72868083208aa0a740b9#patch1
thanks, -- gr



diff --git a/i386-asm.c b/i386-asm.c
index e134d804..63cfbf6b 100644
--- a/i386-asm.c
+++ b/i386-asm.c
@@ -738,6 +738,8 @@ ST_FUNC void asm_opcode(TCCState *s1, int opcode)

  s = 0; /* avoid warning */

+int ind0 = ind;
+
  again:
  /* optimize matching by using a lookup table (no hashing is needed
 !) */
@@ -1153,6 +1155,7 @@ again:
} else if (pa->op_type[i] == OPT_DISP || pa->op_type[i] == 
OPT_DISP8) {
  gen_disp32([i].e);
  } else {
+if (ops[i].e.pcrel) ops[i].e.v += ind-ind0;
  gen_expr32([i].e);
  }
  }

Please CC me on replies, as I'm not subscribed.

Rich

___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel




___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] problem on win64 with latest commit

[grischka via Tinycc-devel]

On 01.03.2024 07:49, Herman ten Brugge via Tinycc-devel wrote:

On 2/29/24 23:17, grischka wrote:

On 29.02.2024 07:26, Herman ten Brugge via Tinycc-devel wrote:

Setting CONFIG_RUNMEM_RO=0 looks incorrect to me because it sets write in 
executables.
Apple has implemented W^X (Writes can not occur in executables) for security 
reasons.
This may also be implemented in in future linux/bsd releases.


Using CONFIG_RUNMEM_RO=1 may be the right thing to do in
future, however there was a severe off-bounds problem with
the un-mprotect call which I just fixed.  Maybe that was
the reason?


This did not work. We still use 'PROT_READ | PROT_WRITE | PROT_EXEC'.
Apple does not support that for security reasons.


Last time you mentioned "Apple W^X", which according to
   
https://developer.apple.com/documentation/apple-silicon/porting-just-in-time-compilers-to-apple-silicon
would require mmap(), MAP_JIT, and some pthread_jit_write_protect_np()
to work around.

Since that is not what tcc has I was concluding that something else
must be at work.


Why do you want CONFIG_RUNMEM_RO=0? It was allways set to 1 before
and that worked fine on all targets I can test (about 20).
You changed it in "tccrun: resign from "advanced" system calls 
(memaligh/gettid)" on feb 25.
Why?


Some things have been simplified lately, the second argument to
tcc_relocate() was removed, memalign was removed, etc. So in the
course of going back to more simplicity, I did change that in
order to see whether it still would work.

Now it seems that it would work in most cases,  except that it doesn't
on "Apple Silicon M1 arm64". If that is what you're saying.

Btw. I've seem some Apple arm64 related patches in
   https://github.com/frida/tinycc/commits/main/
such as
   
https://github.com/frida/tinycc/commit/263232e8cf53991f195d7f7c028488cbd6f6b117

Anyway, I have no problem setting CONFIG_RUNMEM_RO=1 at all, we just
need to be aware that it increases run-memory size by additional two
pages.  Since we also dropped memalign,  the minimum run-memory size
with CONFIG_RUNMEM_RO=1 now is 3 pages plus one for alignment, that is
minimum 16 kB (4 x 4096 bytes).

-- grischka



 Herman

___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel



___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] problem on win64 with latest commit

[grischka via Tinycc-devel]

On 29.02.2024 07:26, Herman ten Brugge via Tinycc-devel wrote:

Setting CONFIG_RUNMEM_RO=0 looks incorrect to me because it sets write in 
executables.
Apple has implemented W^X (Writes can not occur in executables) for security 
reasons.
This may also be implemented in in future linux/bsd releases.


Using CONFIG_RUNMEM_RO=1 may be the right thing to do in
future, however there was a severe off-bounds problem with
the un-mprotect call which I just fixed.  Maybe that was
the reason?

-- grischka



Can I revert the change and set CONFIG_RUNMEM_RO=1 for all targets as before?

 Herman


___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel




___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] problem on win64 with latest commit

[grischka via Tinycc-devel]

On 26.02.2024 19:55, Herman ten Brugge via Tinycc-devel wrote:

I commited a change to lib/runmain.c to silence a warning from gcc.
After that testcase 112 did not work any more on 64 bits windows (tested with 
wine).

I can fix this with fix1 (see below) or fix2 (see below).
I do not know enough about windows to make the correct fix.
Both fix1/fix2 look not correct to me.


Hi Herman,

well, first we'd need to know what problem we want to fix.  Maybe
the problem is not in tinycc at all.

Also, the obvious to avoid that gcc warning IMO would be using:

__attribute__((noreturn)) void __rt_exit(rt_frame *, int);

Also, while at it:

# ifdef _WIN32
#   define CONFIG_RUNMEM_RO 0
# else
#   define CONFIG_RUNMEM_RO 1
# endif

would suggest a special case needed for windows.  However that is not
the case.  Whereas according to your comment, the exception is APPLE
actually.  So we'd better say #ifdef __APPLE__ or just use '1' for all
and leave a comment that it's needed for APPLE.

Although tests on github actions did not show such problem, at least
not on macos-11.  Note that CONFIG_RUNMEM_RO was first introduced with
.rodata sections in commit 72f1dea5372ed551d203311e4f2ab769a54de9bd
from 2021-02,  and it seems that tcc already did work on MacOS even
before that commit.  If those previous commits still work then the
problem now might be elsewhere.

Anyway, we cannot disable the runtime-function-unwind table because
without it longjmp() would crash on (a real) _WIN64.

Also obviously, if __rt_exit doesn't return, then any code after it
cannot have any effect:


  __rt_exit(, code);
+abort (); // avoid noreturn warning


I'd assume something weird going on with set/longjmp of WINE and/or
the mingw-gcc that you use.  In any case it is important that
setjmp/longjmp/jmp_buf implementations do match each other.

Which is why I made it so that tcc_setjmp records the longjmp
function pointer too because otherwise it could end up using the
longjmp from libtcc.dll/so and the setjmp from the application
that links to it, with possibly different concepts each. (Not
sure if/where it really can happen though).

-- grischka



 Herman

fix1:
diff --git a/lib/runmain.c b/lib/runmain.c
index 1cbf6dd..307bf45 100644
--- a/lib/runmain.c
+++ b/lib/runmain.c
@@ -60,6 +60,7 @@ typedef struct rt_frame {
  } rt_frame;

  void __rt_exit(rt_frame *, int);
+void abort(void);

  void exit(int code)
  {
@@ -69,7 +70,7 @@ void exit(int code)
  f.fp = 0;
  f.ip = exit;
  __rt_exit(, code);
-for (;;); // avoid noreturn warning
+abort (); // avoid noreturn warning



  }

  #ifndef _WIN32

fix2:
diff --git a/tccrun.c b/tccrun.c
index b27ccc5..5ea271e 100644
--- a/tccrun.c
+++ b/tccrun.c
@@ -456,7 +456,7 @@ static int protect_pages(void *ptr, unsigned long length, 
int mode)
  static void *win64_add_function_table(TCCState *s1)
  {
  void *p = NULL;
-if (s1->uw_pdata) {
+if (0&>uw_pdata) {
  p = (void*)s1->uw_pdata->sh_addr;
  RtlAddFunctionTable(
  (RUNTIME_FUNCTION*)p,


___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel



___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Newest TCC builds broken on legacy Windows versions on GCC 4.5.2

[grischka via Tinycc-devel]

On 25.02.2024 11:02, Eric Raible wrote:

If build with:
-DCONFIG_RUNMEM_ALIGNED=0
I'm hoping you'd be good to go.


Don't mind, I reverted the usage of those "more recent" system
features (as in memalign() & gettid()).  Simpler code, less
problems ...

-- grischka



On linux one would:
./configure --extra-cflags=-DCONFIG_RUNMEM_ALIGNED=0

On windows I'm not sure, but perhaps augmenting the lines in
win32/build-tcc.bat to look like this would do the same:

set D32=-DTCC_TARGET_PE -DTCC_TARGET_I386 -DCONFIG_RUNMEM_ALIGNED=0
set D64=-DTCC_TARGET_PE -DTCC_TARGET_X86_64 -DCONFIG_RUNMEM_ALIGNED=0

- Eric


On Sat, Feb 24, 2024 at 7:29 AM Dog Dog mailto:andreywilliamsjeff...@gmail.com>> wrote:

I get an error about missing entry point _aligned_free in msvcrt.dll. Since 
the newest builds are labeled as TCC version 0.9.28 RC one could wait until 
after an official TCC version 0.9.28 to break compatibility with legacy Windows 
versions.

The newest builds of TCC are broken on Windows 2000, Windows NT 4.0, 
Windows NT 3.51, Windows Me, Windows 98 and Windows 95. I realize that old 
versions of Windows can't be supported forever and that is an amazing level of 
compatibility for builds prior to recent builds.

The problem started with this build:

https://repo.or.cz/tinycc.git/commit/7b9f19eaab7e568a7c7a42725da812377a588f50
___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org 
https://lists.nongnu.org/mailman/listinfo/tinycc-devel



___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel




___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] A couple of questions

[grischka via Tinycc-devel]

On 12.02.2024 15:35, George Sedov wrote:

Hi all,

First of all, I noticed that throughout the codebase there is a mix of spaces 
and tabs. This makes it very difficult to produce out-of-tree patches, since 
most of the editors will replace tabs with spaces. Is there a reason behind it?


Obviously then the reason must be that most people here did not
use "most of the editors" ... ;)



The second question is about the bounds check. In the current implementation it 
is common to first access the provided pointer, and only after that check if it 
is valid or not (e.g. the __bound_str* functions). Doesn't it defy the whole 
purpose of the functions? What is the intended usecase of this feature?


I guess the intention is to warn you about potential problems,
rather than about problems that you already have (crashes or
sume such).

-- gr



BW,
George



___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Question about in-memory compilation, target

[grischka via Tinycc-devel]

On 11.02.2024 11:08, draco via Tinycc-devel wrote:

You're right, I didn't verify closely, but the ABI is still intact. Sorry...


To be nice I've made tcc_relocate() abort with a notice
when it's called with the former two-step method ;)

-- gr



Am 10.02.24 um 22:12 schrieb Eric Raible:

> This means, that not only the public API changes, but also the
> libtcc.dll/so ABI, making all programs using libtcc crash without warning.
>
> Is this intended?
>
> Michael
I just tried it.  It looks like only programs that _don't_ use
TCC_RELOCATE_AUTO would be affected (at least on my x86_64 debian box).
Looks like the breakage would only be in a program uses manual memory 
management,
that uses the system tcc (instead of a private  version), and where the system 
tcc is
updated w/out rebuilding the application.  Anyone out there who would be in 
that situation?
- Eric



___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel




___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Question about in-memory compilation target

[grischka via Tinycc-devel]

On 09.02.2024 01:44, Eric Raible wrote:

 > Then, instead of adding a new API to support the
 >  "run without state"
 > option better (as you suggest), we could just as well remove that
 > option entirely, and have a simpler and more "lovely" API then ...
 >
 > What do you think?
 >
I think removing that option entirely would be fine.  Others might disagree.
But if we _keep_ that option, then we should have tcc_unprotect().
With respect to "tweak the state a bit", that sounds risky if we're really
trying to stabilize for a release.


Ok, I like it to remove stuff:
https://repo.or.cz/tinycc.git/blob/b671fc0594625eb5ac147ec83be6d0c1fc1a6ad5:/libtcc.h


On an unrelated note, there was a commit a while back that proposed putting 
more into
TCCState, such that no global state at all exists.  I am in favor of that 
proposal, which
kind of implies an even bigger TCCState.


Yeah, answer is always the same:
- eyes would hurt from seeing "s1->" all over the place
- finger would hurt for those who'd like to contribute
- not to mention what if you'd try to rebase some patch that you already have
See also
https://repo.or.cz/tinycc.git/commitdiff/af686a796bda94dc92fc3ad140ef438dafa08950

--- grischka

___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Question about in-memory compilation target

[grischka via Tinycc-devel]

On 07.02.2024 09:38, Eric Raible wrote:
> The alternative is having to know about messy system-dependent details,
> which seems very much against the spirit of the (lovely) libtcc API.

Well, if it's "lovely" then maybe because it's still small and
fairly easily to read.

In any case, there is always a conflict between simplicity
and any new addition, even if they may come very convenient
in some case.

Maybe you (or other people) could tell a bit more why now that
tcc_set_realloc()
can be useful enough that tcc should have it, more exactly?

And if so, why then do we need
tcc_get_realloc()
too?  Is that just an exercise about "setters" and "getters" or
is it really useful?

While at it, why do we need these ones
tcc_get_error_func()
tcc_get_error_opaque()
seen that people should be able to remember what they have set,
in their own code too?

As to "tcc_relocate()" and "tcc_unprotect(mem)", that would
probably good to have.  On the other hand it's already built-in
into tcc_delete().

Maybe it is not worth to keep up that option to compile into
memory provided by the user at all.  If we'd tweak the state
data a bit to keep only the minimum required instead,

for example with "tcc -run tcc.c -v"

code size:
- 421047 bytes

state data sizes:
- 1088 bytes : struct TCCState itself (on i386)
- 5649 bytes : public symbols (= 160 public symbols ~= 35 bytes each)
-  546 bytes : file names, include paths, defines, etc.
-
  7283 bytes total

Which means that in this case the state data is less than 2% of
the total memory usage.

Then, instead of adding a new API to support the
 "run without state"
option better (as you suggest), we could just as well remove that
option entirely, and have a simpler and more "lovely" API then ...

What do you think?

-- grischka


When using

void *mem = malloc(tcc_relocate(s, NULL));
tcc_relocate(s, mem);

wouldn't it make sense for tcc to export a function to safely
restore the pages to R/W in a platform independent way?

I haven't been able to untangle tcc_relocate() -> tcc_relocate_ex() ->
set_pages_executable(), but it seems to me that since tcc sizes and then
populates the 'mem' from above, it could certainly add metadata to allow
a helper could undo the page protection and any other platform-specific
actions that are required before free()-ing the memory.

Perhaps something like tcc_unprotect(mem), which would only be allowed
when one is done with the memory passed to tcc_relocate().

The alternative is having to know about messy system-dependent details,
which seems very much against the spirit of the (lovely) libtcc API.

Comments?




___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Variable declaration as first statement in a 'case' block gives error.

[grischka via Tinycc-devel]

On 19.01.2024 07:18, Herman ten Brugge via Tinycc-devel wrote:

I wanted to avoid an extra check for ';' because I found code like:

 case xxx: ; }

Instead of writing 'break;' they used ';'.
Without checking for ';' a warning was printed:


Always same problem: we want something and do something else.

Anyway, what you told tcc is to accept declarations instead of
an expression, alternatively:

-} else {
+} else if (!decl(VT_JMP)) {
 gexpr();
 vpop();


Which is what happens.  Current tcc accepts:

if (x)
int y;
else
int z;


So how about the next patch:
C23: Implement declaration after label (see attached patch)


block_after_label:
...
if (tok != ':')
decl(VT_LOCAL);
if (tok != '}')
goto again;
/* we accept this, but it is a mistake */
tcc_warning_c(warn_all)("deprecated use of label...

Yes, I'd read this as "accept declarations after a label", too.
So it might be what you want, but is it C23?

Because then, to have a declaration where it was not possible
previously, we'd now just need to put a label in front:

Not accepted
  if (x)
  int n;
  for (n = 0; n < x; ++n)
   ;
  else ...

Accepted:
  if (x)
  decl_please: int n;
  for (n = 0; n < x; ++n)
   ;
   else ...

I'd think that feels a bit strange, because the label would not
serve any purpose other than to support the declaration.

And of course, if that is C23, then people then will start to
write this.

-- gr

___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] looking for documentation for porting

[grischka via Tinycc-devel]

On 16.01.2024 00:38, Michael Larson wrote:

Hello,

I have been tinkering around with TCC porting to a RISC cpu of my own
design, I have my own C simulator and verilog targeted for a FPGA.. I
have a lot of tests for functionality, but I need more so TCC looked
like a good option to generate code for.

I have been following some of the code samples given like the C67 and
MIPS source, I have about half of it done but documentation on how
this works seems to be really missing.

I did a normal port, I copied all the functions out of the header
files required and built a C file for them. This worked well... but..
there are a lot of questions.

Like

int gjmp_cond(int op, int t)

I look around the source and see a bunch of different implementations
of this, but there is no documentation on how it's supposed to work? I
am sure I could dig into it a bit more.. but it's time to ask where
the documentation is, maybe I am missing something.. is there
documentation out there?


No, because it's really so simple ;)

It's simply supposed to put the conditional jump opcode
as given in 'op', fill its address field with 't', and
return the offset of that field (ind - 4 after it was put).

There is also

 void gsym_addr(int t, int a)

which then is supposed to replace the field at offset 't'
with address 'a', and continue to do so with the former
contents of that field unless it's zero.

As you can see to write this little documentation already
was much more effort than it ever was or would be just to
write or read those functions. ;)

Other than that, the maybe as simple as possible single file
implementation of that stuff may be found in the first revision
of the repo here

   https://repo.or.cz/tinycc.git/blob/27f6e16b:/tcc.c

You'll find that for example the 'if' statement code in block()
is still almost identical to how it is now.

   https://repo.or.cz/tinycc.git/blob/27f6e16b:/tcc.c#l780

--- gr


I only have a few functions left so I am not going to give up at this point.

Thanks ahead of time,

Mike



ST_FUNC int gjmp_append(int n, int t)
{
printf("%s:%s unimplemented\n", __FUNCTION__, __FILE__);
return 0;
}

ST_FUNC void gen_opi(int op)
{
printf("%s:%s unimplemented\n", __FUNCTION__, __FILE__);
}

ST_FUNC void gen_opf(int op)
{
printf("%s:%s unimplemented\n", __FUNCTION__, __FILE__);
}

ST_FUNC void gen_cvt_ftoi(int t)
{
printf("%s:%s unimplemented\n", __FUNCTION__, __FILE__);
}

ST_FUNC void gen_cvt_itof(int t)
{
printf("%s:%s unimplemented\n", __FUNCTION__, __FILE__);
}

ST_FUNC void gen_cvt_ftof(int t)
{
printf("%s:%s unimplemented\n", __FUNCTION__, __FILE__);
}

ST_FUNC void ggoto(void)
{
printf("%s:%s unimplemented\n", __FUNCTION__, __FILE__);
}

ST_FUNC void gen_vla_sp_save(int addr)
{
printf("%s:%s unimplemented\n", __FUNCTION__, __FILE__);
}

ST_FUNC void gen_vla_sp_restore(int addr)
{
printf("%s:%s unimplemented\n", __FUNCTION__, __FILE__);
}

ST_FUNC void gen_vla_alloc(CType *type, int align)
{
printf("%s:%s unimplemented\n", __FUNCTION__, __FILE__);
}

___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel




___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Question about in-memory compilation target

[grischka via Tinycc-devel]

On 15.01.2024 00:51, Brad Robinson via Tinycc-devel wrote:

Questions:

1. Is it possible to keep, use and then release the compiled code after the 
initial compiler instance has been deleted


FYI below some version of libtcc_test.c to run without state.

Note the LoadDll is needed only when the exe itself was not
linked with msvcrt, i.e. to prevent it from being unloaded
in tccelf_delete.

-- gr

void *mem; int size; HANDLE dll; DWORD xxx;

if ((size = tcc_relocate(s, NULL)) < 0)
return 1;
tcc_relocate(s, mem = malloc(size));
if (!(func = tcc_get_symbol(s, "foo")))
return 1;
dll = LoadLibrary("msvcrt.dll");
tcc_delete(s);

func(32);

FreeLibrary(dll);
VirtualProtect(mem, size, PAGE_READWRITE, );
#if _WIN64
RtlDeleteFunctionTable(*(RUNTIME_FUNCTION**)mem);
#endif
free(mem);
return 0;

___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Variable declaration as first statement in a 'case' block gives error.

[grischka via Tinycc-devel]

On 18.01.2024 17:38, Herman ten Brugge via Tinycc-devel wrote:

That depends on the version of gcc / clang.
Gcc 13.2.1 prints no error.
Clang 17.0.6 prints "error: expected expression"
Clang 18.0.0(not released yet) prints "warning: label followed by a declaration is a 
C23 extension [-Wc23-extensions]"

I probably should have mentioned it when committing the fix.


Maybe yes.

Plus mention that it's not a fix, but an experiment
with undocumented non-standard syntax.

Plus mention the benefit and the price too, please.

Plus while at it, spend some thoughts first:

-} else {
+} else if (!decl(VT_JMP)) {
 gexpr();
 vpop();


Hm... just one line changed.  But wait, does not tcc look
for decl before expression already?  Sure it does because
it supports C99.  Why now look twice ?  And why skip parsing
the expression if where was a declaration?  What's the point?
Who wrote that?

Let's maybe try to use the tcov for tcc (which as we know was
you who wrote it and is really not bad btw):

$ tcc -ftest-coverage ../tcc.c -o tcc1.exe && tcc1.exe ../tcc.c -o tcc2.exe

tcc1.exe.tcov:
   -: 0:Function:decl 68.67%
Before:
  10701: 8417: int v, has_init, r, oldint;
After:
  18408: 8417: int v, has_init, r, oldint;

We note 7707 additional (+72%) calls for decl(),

  183: 7247:block_after_label:
 1006: 7248:  {

and that however only 1006 labels/cases/defaults were seen
at all.

Also we note no "label: decl" in tcc anywhere and also in
noone elses code we've seen so far (the OP's doesn't count
because it's generated C IIUC)

Now back to the "benefit vs. price" question ...

-- gr



 Herman



___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel



___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Variable declaration as first statement in a 'case' block gives error.

[grischka via Tinycc-devel]

On 15.01.2024 00:34, Brad Robinson via Tinycc-devel wrote:

Hey All,

First post here.  Firstly, thank you all for your work on this project.  I 
discovered tcc just a month or so ago and really enjoying using it as a 
back-end code generator for a custom scripting language I'm working on.

Anyway, I noticed this small bug:

A variable declaration as the first statement in a switch case block fails when 
it shouldn't:

 case 1:
 int z = 123;// This fails with "identifier expected"
 break;


Hi,

well, yes, no.  As a fact, a variable declaration is not a
statement.


There's a pretty easy workaround, just insert an empty statement before.

 case 1:
 ;   // This fixes it
 int z = 123;
 break;



Yes, label before statement is allowed, also label before empty
statement, just not label before no statement.

As for example "label: }" is not allowed either (in the theory).

See also what gcc has to say:

test.c: In function 'main':
test.c:7:23: error: a label can only be part of a statement and a declaration 
is not a statement
switch(a) case 1: int z;
  ^~~

Which of course if a lot better than just "identifier expected"
I would agree that far.

-- gr


(this was build of the mob repo on github - not sure if that affects things).

Brad


___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel

Re: [Tinycc-devel] Outdated tcc: Major bug in the pow function resulting in: 0.993013 = 0.860892

[grischka via Tinycc-devel]

On 06.01.2024 01:10, Detlef Riekenberg via Tinycc-devel wrote:


Of course, there might be more issues in 0.9.28rc
As example, "i386-tcc" on a x86_64 linux machine
is missing additional search dirs:


Can we see the output of "i386-tcc -vv" ?


include: "/usr/i686-linux-gnu/include"
lib: "/usr/i686-linux-gnu/lib"
crt: "/usr/i686-linux-gnu/lib"

(there are no "crt*.o" files, "libc*" files and similar
in "/usr/lib/i386-linux-gnu")





Please do a release.
Please 
Please .



___
Tinycc-devel mailing list
Tinycc-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/tinycc-devel