Re: Pass compiler options to file compiled with the compile pragma
It is simpler than my code, but there is a problem in your code when you want
to link multiple C files. If 2 C files have same name static variables outside
of function, it would generate redefinition error. In C++, redefinition error
will be also generated when 2 C++ files have same name variables or functions
in anonymous namespace. You can avoid such error by creating 1 nim file per 1
C/C++ files, but I don't think that is not good idea when you want to use many
C/C++ files.
And when you got compile error from C/C++, line number in the error doesn't
match line number in C/C++ files. I think you can fix it by emitting #line N
before C code.
My code might looks like complicated but linkModule template can be placed in
other module and be used in elsewhere without copy it.
For example: test1.c
static int x = 1;
int getX1() {
return x;
}
Run
test2.c
static int x = 2;
int getX2() {
return x;
}
Run
testc.nim
const
Ccode1 = staticRead"test1.c"
Ccode2 = staticRead"test2.c"
{.emit: Ccode1.}
{.emit: Ccode2.}
Run
sample.nim
import testc
proc getX1(): cint {.importc.}
proc getX2(): cint {.importc.}
echo getX1(), getX2()
Run
Re: Why does wrapping the code in a top level procedure make it faster?
In my experience batteries are almost never fully charged and it's hard to get feedback if you only release a perfect cathedral. With no feedback, it's kind of unlikely you will build something popular. To take a topical example, even after 30 years of tuning, the core language `dict` in Python still seems to have no easy way to "pre-size" an instance. There is no power of 2, no `rightSize`, nuthin'. So, one of your rough edges literally cannot arise due to an inflexibility/API design flaw (IMO). Yes, there must be 3rd party replacements or ways to pre-size in some slow subclass or whatever, but you could also just write a `proc initTab` that always calls `rightSize` for your own code. What's at issue here is "out of the box" (and personally I think the Nim workaround is less onerous than workarounds in the Python world). Do you have to learn your way around? Sure. A sales pitch of "just like Python but compiled/with native types" is definitely not quite right. That's Cython/similar. Analogies and oversimplifications help some people while others want the real story, and you strike me as the latter. Nim gives programmers more power and flexibility, but responsibilities also come with that. Cue Spider Man's Uncle Ben. ;-) It is yet a comparatively tiny community. Bugs, workarounds, and rough edges come with that. Unused patterns like your `&` are just unused, unoptimized things waiting to be discovered/fixed. There was a time when `C` had no proper namespacing of struct members and that is why some core Unix types have prefixes like the `st_*` in `struct stat` (run "man 2 stat" on any unix machine). No one using Nim _wants_ it to be hard, but there may also be good reasons some things are the way they are (often flexibility, performance, or safety, but yes sometimes neglect). I'm really sorry to hear your entry has been tough, but the flip side of that is you could probably make a HUGE impact to future similar-but-not-quite-you's, and I again encourage you to try! Even just documenting everything somewhere would probably help at least a few other Python Refugees. :-) Cheers and best wishes whatever you decide.
Generic function resolution
Can someone explain this
type A* = object of RootObj
p: int
type
B* = object of A
q: int
type
C* = object of A
r: string
proc g[T](d: string) =
when T is B:
let l = (first: -1, last: 0)
echo(d.substr(l.first, l.last))
proc g[T]( k: string, d: string) =
let l = (first: -1, last: 0)
echo(d.substr(l.first, l.last))
proc f[T](d: string): seq[T] =
result = newSeq[T]()
g[T](d)
echo "g[T]( d)"
proc f[T](k: string, d: string): seq[T] =
result = newSeq[T]()
g[T](k, d)
echo "g[T](k, d)"
let d = ""
let t = f[B](d)
let sk = f[C]("SM", d)
Run
Compiling gives
d:\temp\test2.nim(34, 11) template/generic instantiation of `f` from here
d:\temp\test2.nim(24, 7) template/generic instantiation of `g` from here
d:\temp\test2.nim(16, 17) Error: undeclared identifier: 'l'
Run
The second call to f resolves to the first definition of f with arity 1 instead
of the second one.
Changing the first definition of g to
proc g[T](d: string) =
echo(d)
Run
makes it compile
Re: Why does wrapping the code in a top level procedure make it faster?
Another lesson: when batteries are included, make sure they are fully charged.
Re: Why does wrapping the code in a top level procedure make it faster?
Whenever you leave the C code that Python happens to use like actually looping over an array it's slow. Python is in fact an _abstraction inversion_ , primitive operations are slow and high level operations are fast (because they can avoid Python's slow VM) and so you need to use premade libraries for everything. In Nim it's the opposite, primitive operations are fast and custom code wins over premade libraries quite often because custom code is specialized for the task at hand. **That 's the fundamental lesson to learn when coming from Python.** Surely we can and will improve the rough edges you encountered but it won't change the fundamental lesson.
Re: Why does wrapping the code in a top level procedure make it faster?
As the person who added `rightSize`, I agree with those points actually. The only slight gotcha is that doing as you suggest would result in old code that calls it with (the correct power of two to avoid a resize, or maybe already a `rightSize`) would allocate the _next higher_ power, wasting space. That might be a price worth paying ease-of-use wise. We _could_ only call `rightSize` if the argument is not a power of two, but it's probably simpler to just always use `rightSize` and just tell people. Wasting space isn't quite a "failure". Anyway, you should feel free to file a github issue and/or work on a pull request. Maybe I should have pushed harder at the time to change the parameter semantics. There is definitely a learning curve coming from other languages. There are definitely more examples of the stdlib not providing the "best in class" for various functionalities. There will likely be even more with time. Nim core only has so much manpower to write/maintain/support such things. We discussed doing a "distribution" over at [https://github.com/nim-lang/RFCs/issues/173](https://github.com/nim-lang/RFCs/issues/173) and elsewhere. This "fusion" repository (see end of that thread) seems to have been the output, but not much has been happening over there. Anyway, I think if you persevere there are good chances you could be a happy Nimmer once you've learned your way around. You seem pretty capable of finding these issues and you could probably help round some of the sharp edges, if you can sustain the patience. There is probably a Porting to Nim From Python guidebook or maybe you could help do one!
Re: Why does wrapping the code in a top level procedure make it faster?
My experience so far with Nim is that there are a lot of "sharp and unexpected
edges". Some examples:
* 100M loops of s = s & 'x' is 8100x slower than s.add('x'). Response is
"nobody writes this way". If something is 8100x slower, IMO it shouldn't be
allowed at all.
* 100M loops of s.add('x') uses 325MB of RAM
* Nim stdlib code is not compiled in production mode unless my code is also
in production mode, so even simple timing results like hashing data in a loop
make no sense
* tables are documented to require a power of 2 size _in the current
implementation_. The documentation for initTable says to use nextPowerOfTwo or
rightSize to specify the default size. What if the next implementation of
tables requires the table size to be a prime number? The only correct, portable
way to specify a table size is with rightSize, and IMO, that should be done
inside initTable rather than expecting users to know implementation details.
Users only know how many things they want to put in a table.
* if you use a table size not a power of 2, it asserts - fine. But if you
compile with -d:danger, it loops forever. Another reason initTable should
always use rightSize on the initialSize arg.
* everything having value semantics "for consistency" is a little on the
strange side IMO. My guess is that when someone uses arrays, tables, and
sequences, it is because they have a largish collection of data. If that gets
passed around, it is being copied every time. I don't think most people would
know or expect this, and since the program still works, their conclusion might
be "Nim is slow".
* a related problem: I wrote an array sort test with a small 100-element
array. Very simple, worked great. Then I changed the array size to 10M and it
fails with "Illegal instruction: 4", I'm guessing because of a stack overflow.
Changed it to a ref array, added a new stmt, then the high function no longer
worked (changed that to a constant), then the sort function no longer works
(have to add a cmp proc), etc. It feels like a bit much just because an array
size was increased.
* the standard db_sqlite module is 30% slower than Python on a small test
script because it doesn't cache prepared SQL. Another guy (tiny_sqlite) added
that recently and his lib runs 3x faster than Python for the same test. Users
of a language need to feel like the tools the language provides are "best in
class" rather than providing minimal required facilities.
* the SomeInteger type is compatible with all integers. But an int64 can't be
passed to an int type, even on a 64-bit system.
* requiring 10'i64 or 8'i32 on integer constants is unwieldy and ugly, and
it's in Nim system code too.
I'll admit people (including myself) are not great at writing or understanding
the results of benchmarks. If the language environment has rough edges like
this, it turns every small test into a head-scratching exercise. And after a
few weeks of this, it leaves one with the feeling "If I can't even get simple
10-20 line programs to do what I want, does it make sense to use this for a
200K line program?" It makes me feel a bit like "to drive this car, I have to
become a mechanic and know how to rebuild it first or I'll crash".
When Nim works, is fast, and uses reasonable amounts of memory, it's fantastic
and I love it. But getting there seems more difficult than it should be.
Re: Why does wrapping the code in a top level procedure make it faster?
Ah. Sorry that I misunderstood the sarcasm. Maybe my content/xref was not worthless anyway. ;-)
Re: Why does wrapping the code in a top level procedure make it faster?
> The idea of defaulting to an optimizing mode has come up before. Sorry, maybe I made my statement not clear enough: Of course it is fine that default compiler mode is to use runtime checks and do not use -d:danger. @didlybom asked to generate an automatic main function so that newcomers making a benchmark would get good results. But generally these newcomers also forget to compile with -d:release, and also these newcomers often use too many ref objects, too many string allocations and other slow stuff. So the recommendation is: Before making benchmarks or videos read a tutorial.
Re: Connection-Pooling Compile-Time ORM
Can't help myself: const data: SqlQuery = sqls: SELECT Pet.name FROM Pet INNER JOIN House ON House.id = Pet.houseId WHERE House.country = "CA" Run That would be really clean.
Re: Connection-Pooling Compile-Time ORM
To followup on @Variount 's question. Is the ORM goal to be something like:
type
Pet = object {. tableName: "pets" .}
name: string
age: int {. fieldName: "age_years" .}
const data: SqlQuery = sqls:
select Pet.*
`from` Pet
where Peg.age = 4
Run
Re: Connection-Pooling Compile-Time ORM
As to case, in this particular application, one could have a version that supports the typical styling guidelines for SQL statements. SELECT blah FROM xyz WHERE zin=4 Run could be DSL'd as: const data: SqlQuery = sqls: SELECT "blah" FROM "users" WHERE "zin = 4" Run It would avoid the use of backticks and have a certain visual appeal to SQL coders. But then you would be violating nim's case guidelines. Sometimes one can't win. :-)
Re: Why does wrapping the code in a top level procedure make it faster?
@Stefan_Salewski \- The idea of defaulting to an optimizing mode has come up before. It seems contrary to what almost any user coming from a "compiled language" world would expect. For various reasons (debugability, compile-time performance, etc.), almost all compilers default to a non-optimized (or very weakly optimized) output and allow various knobs to crank up optimization, as does the current `nim` setup. There is even a whole dark art of "best optimization flags" which can be taken [to severe extremes](https://github.com/Acovea/libacovea). More simply/efficiently/some might say intelligently, you can often use PGO [https://forum.nim-lang.org/t/6295](https://forum.nim-lang.org/t/6295) to get 1.25..2.0x boosts on object code generated from nim-generated C. Some flags like `-ffast-math` can even change semantics in subtle ways that can impact correctness or not depending on use cases. I don't know what to do about people publicizing misleading benchmarks. That seems an ineliminable hazard, not only for Nim, but actually **_everywhere and all the time_** , and often not on purpose (though most "marketing" wants to do strawman comparisons on purpose). Besides compiling wrong, they could also use a bad algorithm, misrepresentative inputs, weird machines, benchmarks naive relative to intended measurements, and probably several other mistake categories. :-/ The best strategy may be trying to be supportive where & when we can, educating as we go, though I agree/admit that is a neverending battle.
Re: Pass compiler options to file compiled with the compile pragma
You can probably do something like this
# ccode_wrapper.nim
const Ccode = staticRead"ccode.c"
{.localPassC:"-DMY_CUSTOM_FLAG".}
{.emit: Ccode.}
Run
i.e. replace the direct C file compilation by staticRead + emit + localPassC.
Re: Why does wrapping the code in a top level procedure make it faster?
We also should ask people to use the main proc even when they do when isMainModule, because that doesn't introduce any new scopes, so we should advise to do something like proc main = echo "hi" when isMainModule: main() Run
Re: Why does wrapping the code in a top level procedure make it faster?
> whenever somebody does a benchmark So should we make -d:danger the default just for the case a fool makes a benchmark? Global code is generally used only for short scripts where nobody really cares for performance. And the benefit of a main proc is explained in most tutorials, it is in section [http://ssalewski.de/nimprogramming.html#_scoping_visibility_and_locality](http://ssalewski.de/nimprogramming.html#_scoping_visibility_and_locality) in my book.
Re: Why does wrapping the code in a top level procedure make it faster?
OK, it sense to focus on the higher impact optimizations. That being said, this is one of those things that are mentioned whenever somebody does a benchmark, so it might be good to optimize it at some point.
Pass compiler options to file compiled with the compile pragma
There is still no way to pass C/C++ compiler flags to only c/c++ files added by
compile pragma. Related RFC:
[https://github.com/nim-lang/RFCs/issues/137](https://github.com/nim-lang/RFCs/issues/137)
I found workaround that create static link library from the nim file that only
has passC and compile pragma so that passC affect to only c/c++ files specified
in that nim file.
Because of this bug, this code works only in devel Nim.
[https://github.com/nim-lang/Nim/issues/12745](https://github.com/nim-lang/Nim/issues/12745)
For example:
test.h
int getX();
Run
test.c
int getX() {
return MY_X;
}
Run
testc.nim
# Pass C compiler options only to test.c
{.passC: "-D MY_X=123".}
{.compile: "test.c".}
Run
testmod.nim
import os, strformat, std/compilesettings
proc getX*(): int {.header: "test.h".}
template linkModule(modulePath, nimCmd = "c", nimOptions: static[string] =
"") =
static:
doAssert nimCmd == "c" or nimCmd == "cpp"
block:
const
modName = splitPath(modulePath).tail
libpath {.inject.} = querySetting(SingleValueSetting.nimcacheDir) /
modName & ".staticlib"
modulePath2 {.inject.} = modulePath.quoteShell
nimCmd2 {.inject.} = nimCmd
nimOptions2 {.inject.} = nimOptions.string
{.passL: libpath.}
# `--os:any --gc:arc -d:useMalloc` is used to avoid compiling and
linking stdlib_io.nim for faster build.
# It doesn't affect this module.
const cmd = fmt"{getCurrentCompilerExe()} {nimCmd2} --app:staticlib
--os:any --gc:arc -d:useMalloc --out:{libpath.quoteShell} {nimOptions2}
{modulePath2}"
static: echo cmd
const ret = gorgeEx(cmd)
static: echo ret.output
when ret.exitCode != 0:
{.error: "Abort due to compile error".}
linkModule "testc.nim"
Run
sample.nim
import testmod
echo getX()
Run
