Re: Is it possible for a macro to know the context it is used?
Ah, you probably want `declared` instead of `declaredInScope`. The latter only
works for the immediately enclosing scope and not the ones above it. The
documentation may not be very clear on that
[https://nim-lang.org/docs/system.html#declared%2Cuntyped](https://nim-lang.org/docs/system.html#declared%2Cuntyped).
import macros
template foo(body: untyped) =
block:
let inFoo {. inject, used .} = true
body
template bar(body: untyped) =
block:
let inBar {. inject, used .} = true
body
proc callApiFromFoo =
echo "Calling API with foo scope"
proc callApiFromBar =
echo "Calling API with bar scope"
template callApi =
when declared(inFoo):
callApiFromFoo()
elif declared(inBar):
callApiFromBar()
else:
echo "You can call API only in foo or bar scopes!"
proc foobar =
foo:
block:
callApi()
bar:
block:
callApi()
foobar()
Run
Re: Is it possible for a macro to know the context it is used?
Could you explain how/why it works?
I have reproduced that pattern in my project and it works **sometimes but not
always** , and I don't understand why.
When it does not work, I have changed the `callApi` template to be like:
template callApi =
when declaredInScope(inFoo):
callApiFromFoo()
elif declaredInScope(inBar):
callApiFromBar()
else:
echo "inFoo scope? = ", declaredInScope(inFoo)
echo "inBar scope? = ", decalredInScope(inBar)
error("You can call API only in foo or bar scopes!")
Run
And in the `foobar` proc (a macro in my case), I dump the whole AST tree. I can
see that `inFoo` or `inBar` are correctly defined and deeper in the tree I find
LetSection
IdentDefs
PragmaExpr
Ident "inFoo"
Pragma
Ident "inject"
Ident "used"
Sym "Bool"
Call
...
Command
Ident "echo"
Call
Ident "callApi"
Run
(in that specific case, `callApi` returns a `string`)
But the call `callApi` goes in the `else:` branch and I see
echo ["inFoo scope? = ", "false"]
echo ["inBar scope? = ", "false"]
error("You can call API only in foo or bar scopes!",
nil)
Run
The case where it fails is more complex than this example. It is like
`declaredInScope` does not return the correct result in that particular
situation. I've tried to look at the code for `declaredInScope` but that's a
[magic
function](https://github.com/nim-lang/Nim/blob/version-1-2/lib/system.nim#L163)
and I can't find it to understand how it works.
In case it could be useful to find the source of the problem, compilation fails
with the error message (which is true because `callApi` expects to return a
`string` but that's the error case...)
Error: expression has no type:
echo ["inFoo scope? = ", "false"]
echo ["inBar scope? = ", "false"]
error("You can call API only in foo or bar scopes!",
nil)
Run
Re: Is it possible for a macro to know the context it is used?
You could do it by wrapping in a template
import macros
template foo(body: untyped) =
block:
let inFoo {. inject, used .} = true
body
template bar(body: untyped) =
block:
let inBar {. inject, used .} = true
body
proc callApiFromFoo =
echo "Calling API with foo scope"
proc callApiFromBar =
echo "Calling API with bar scope"
template callApi =
when declaredInScope(inFoo):
callApiFromFoo()
elif declaredInScope(inBar):
callApiFromBar()
else:
echo "You can call API only in foo or bar scopes!"
proc foobar =
foo:
callApi()
bar:
callApi()
foobar()
Run
Re: Is it possible for a macro to know the context it is used?
In other words, is it a part of the job that I can delegate to the Nim compiler semantic parsing phase or it it something I must do before it, when I'm managing the DSL syntax?
Re: Is it possible for a macro to know the context it is used?
I reopen this thread as I'm facing a similar problem, but now when calling an
API.
Is it possible to write a proc whose behaviour is based on existence or not of
macro injected variables? I've tried to reproduce a sample case below:
import macros
template foo(body: untyped) =
block:
let inFoo {. inject, used .} = true
body
template bar(body: untyped) =
block:
let inBar {. inject, used .} = true
body
proc callApi =
if declaredInScope(inFoo):
echo "Calling API with foo scope"
elif declaredInScope(inBar):
echo "Calling API with bar scope"
else:
echo "You can call API only in foo or bar scopes!"
proc foobar =
foo:
callApi()
bar:
callApi()
foobar()
Run
This code **does not give the expected result!**
What I'm trying to do and that does not work: the `callApi` proc takes no
parameters and depend on the scope it is used to call different API entry
points. Its behaviour is really given in the scope it is used. In order to
determine in which scope it is used, I've tried to inject 2 different variables
`inFoo` and `inBar` and base `callApi` behaviour on the presence or absence of
these variables in the scope.
Having changed the way my DSL works, I'm now able to know in which
scope/context the AST is parsed. But in that case, `callApi` is a Nim function
that can be called by the user and I don't want for these types of functions to
have a complex syntax: the user knows if she is using it in the `for` or `bar`
contexts and that's the reason it doesn't have arguments in my example.
Behind the scene, I have two functions `callApiFromFoo` and `calApiFromBar`
with all the parameters required. I want to be able to find which function to
call...
Is the only way to get the expected result to parse the whole AST tree and when
finding a `callApi` call NimNode, replace it with `callApiFromFoo` (resp.
`callApiFromBar`) when in the `foo` (resp. `bar`) scope?
Or is there another way to solve this problem?
Re: Is it possible for a macro to know the context it is used?
> Instead of having multiple small macros that don't play well together and > have only a local context, I'll apply a visitor-like pattern and I can > control the scopes in the DSL. Yes! Now you're cooking.
Re: Is it possible for a macro to know the context it is used?
Night has been enlightening. I understand my error. Instead of having multiple macros `algo1`, `evaluate`, etc. that are called when Nim compiler evaluates the syntax tree from the `dsl```call, I must see ``dsl` as a compiler itself. Now, I have only one macro, `dsl`, that is used to convert its argument to an AST, and then this AST is parsed/translated by procs in a new AST that will be used by the Nim compiler. Instead of having multiple small macros that don't play well together and have only a local context, I'll apply a visitor-like pattern and I can control the scopes in the DSL. Thanks @dawkot for forcing me to think out of the box with your example.
Re: Is it possible for a macro to know the context it is used?
If there's a pattern of doing init(...), evaluate: ..., let ... = getValue(), then you can convert it to something like this: dsl: algo1 as x: echo x algo2(x) as y: echo y Run
Re: Is it possible for a macro to know the context it is used?
Well, the DSL is more a language than a fixed structure. I gave a small sample code to feel it but you could have code like: dsl: algo1: evaluate: # Do something with algo1 result let x = getValue() algo2: # Using x to calculate algo2 init(x) evaluate: # Getting a new value let y = getValue() etc. Run That's the reason why `dsl`, `algo1` and `algo2`, `evaluate` are macros in that example that can be combined by the user. `dsl` is used by the user to start using the DSL syntax. `algo1` and `algo2` are two keywords of the DSL to do some operations, and `evaluate` is another keyword to get access to result from these operations. Like I said, eventually the solution would be to change the DSL syntax, either by using dedicated `evaluate1` and `evaluate2`, but in the real DSL I could have more that 2 `algoN` and having a single keyword for a common action is more orthogonal. Or do you have a better idea?
Re: Is it possible for a macro to know the context it is used?
This is how I would do it import macros macro dsl(nodes) = result = newStmtList() for n in nodes: var value: NimNode if eqIdent(n[0], "algo1"): value = newLit"Foo" elif eqIdent(n[0], "algo2"): value = newLit(123) else: error "Unexpected ident: ", n[0] for kid in n[1]: if eqIdent(kid[0], "evaluate"): let body = kid[1] result.add quote do: template value: untyped = `value` `body` else: error "Unexpected ident: ", kid[0] dsl: algo1: evaluate: echo "[algo1] Evaluation is " & $value algo2: evaluate: echo "[algo2] Evaluation is " & $value Run
Is it possible for a macro to know the context it is used?
For instance, I have a small dsl like: dsl: algo1: evaluate: echo "Evaluation is " & $value algo2: evaluate: echo "Evaluation is " & $value Run The role of `evaluate` is to retrieve the result of calculation using `algo1` or `algo2` that use different data structures in an external API. So when `evaluate` is called in the context of `algo1`, it must call a different API than when used in the context of `algo2`. The simplest solution would be to use different names for `evaluate` in context 1 vs 2, like `evaluate1` and `evaluate2`. Another solution, in `evaluate`, would be being able to detect when walking up the AST if it contains NimNode calls with "algo1" or "algo2". But I don't think that you can walk up the AST to the root like you can do down to the children with [findChild](https://nim-lang.org/docs/macros.html#findChild.t%2CNimNode%2Cuntyped) for instance. Another idea is for `algo1` and `algo2` to set the algorithm used in a variable and let `evaluate` retrieve the value of that variable before calling the API. But if the macro `evaluate` is executed in an `untyped` context from `algo1` / `algo2`, it can't access that variable content because that context hasn't been semantically checked yet, if I understand correctly. What would be an approach to solve that problem? Please note that I've simplified my problem and that in the real DSL, there could be multiple layers of block calls between `algoX` and `evaluate`.
Re: Is it possible to share a Hashmap between threads?
Someday destructors may be defined/automatically called, but today there will be no clean-up without the `close` call. You may not need to/want to clean-up anyway. need to: `memfiles.open()` creates a memory mapping, but does not (by default) keep the file descriptor open. The only thing the `mf.close` above does is call `munmap`. Memory maps are a less scarce resource than file handles due more to default OS limits than anything else. Being much less scarce, it's not quite as bad to leak it. want to: You might even want to leak it on purpose if you wanted to keep the string data alive beyond the lifetime of the procedure invocation. E.g., you may want to return an `MemSlice` or a `Table` with them as keys or values. They are just pointers to that memory mapped region (hence zero copy) which means if you `mf.close/munmap` the pointers become invalid. Of course, if you are leaking millions of memory mappings then even small leaks can grow to be burdensome on the OS -- if you don't need continued access to the data. So, as with all resource management there is a balance of concerns and trade offs.
Re: Is it possible to share a Hashmap between threads?
Very sorry to be annoying but with --gc:arc, is it necessary to close the memfile?
Re: Is it possible to share a Hashmap between threads?
Well, the data "on disk" is not NUL-terminated. So, though the type I am using is called "cstring" the string data is not terminated. If anything, the data is more "newline terminated" in your file format. Those newlines are mapped out by the `memSlices` iterator, though with the result stored in `.size`. So, I just use that to bound the loop. I used that type because Nim defines convenient `[]` style access to the bytes of a `cstring` giving back `char` s. It isn't strictly necessary, but that was my reason. (You would have to do your own pointer arithmetic and dereferencing otherwise).
Re: Is it possible to share a Hashmap between threads?
Just a small question regarding your parseUint, you are taking an argument called size. If I convert something to a cstring, wouldn't I get a null terminated string? In which case I can just check if the char is 0 or not to end the loop.
Re: Is it possible to share a Hashmap between threads?
One vexing thing about modern CPUs is that a single CPU core cannot saturate RAM bandwidth. So, while throughput in L1 can be 100..200 GB/sec, and RAM bandwidth might be 50..150 GB/s, the amount just one L1 cache system can read from L2/L3/DIMMs might be only 5-30 GB/sec. This puts (often much simpler) single-threaded code that could otherwise saturate RAM throughput (thereby running as fast as possible anyway) at a major throughput disadvantage. But I totally agree with @mratsim here that staying with ST-code is the best idea. The cynic inside me guesses the throughput disparity incentivizes people to write more parallel code, thusly selling more CPUs. So, these days CPU manufacturers are sadly incentivized to complicate lives of programmers. At the dawn of the industry, it was the other way around. Maybe there is some less cynical "scaling" problem, but usually with throughput/bandwidth you can just add more lanes of traffic.
Re: Is it possible to share a Hashmap between threads?
Note that with a shared table, accesses will need to be serialized to ensure soundness and data consistency: * This means that threads will compete for cache accesses and might lead to cache thrashing/cache ping-pong/false sharing/cache invalidation. If your task per thread is short (less than micro-second ~ 1 cycles) and your table is small, you will likely experience a huge slowdown anywhere from 2x to 10x as threads compete for cache. * Nim shared tables are really really bad and should be removed from the standard library. It's only useful for proof-of-concepts but underneath it's just a shared heap allocated table + lock. Unfortunately writing fast and correct concurrent data structures is an extremely **hard** problem. * processing strings in parallel brings memory-management woes as strings are heap allocated and reclaiming memory that was allocated from another thread is about 5x slower at best than reclaiming memory from your own thread. * Furthermore if all the created strings ends up owned by the shared table thread you are in a producer/consumer scenario with many threads that depletes their extra memory cache and 1 thread that ends up hoarding lots of extra memory when it collect the strings, so both of their cache ends up useless. Unfortunately few memory allocators deal with that effectively (Microsoft's Mimalloc and Snmalloc), and it's the worst case behaviour for the default allocators in Linux, MacOS and Windows. Lastly this part is suspect in terms of extra allocation `paramStr(1).open().readAll().splitLines().ytz()` as mentioned by @leorize So I suggest you use @cblake code single-threaded.
Re: Is it possible to share a Hashmap between threads?
Oh, and if you did want to use a zero copy IO interface like `memfiles` on
this, you can and indeed it is over 3X faster, but you have to write your own
integer parser because `parseUint` and all its analogues take a Nim `string`
not `MemSlice`. You can use the dollar operator `$` to convert, but that is no
faster than just using `lines` in the first place. Parsing an integer in ASCII
is actually pretty easy, though. E.g.,
import tables, os, memfiles
proc parseUInt(data: cstring, size: int): uint {.inline.} =
let zero = ord('0')
let nine = ord('9')
for i in 0 ..< size:
let code = ord(data[i])
if code >= zero and code <= nine:
result *= 10
result += uint(code - zero)
else:
return 0 #non-digit; parse error
proc ytz(path: string): uint =
var
table = initTable[uint, uint]()
mf = memfiles.open(path)
if mf.mem == nil:
quit("Cannot open " & path)
for s in memSlices(mf):
let n = parseUInt(cast[cstring](s.data), s.size)
if n == 0:
quit("Parse error: " & $s)
table.mgetOrPut(n, 0) += n
for key, val in table:
result = max(result, val)
mf.close
proc main() =
echo ytz(paramStr(1))
main()
Run
Not a big deal here, but if the file were really large that factor of 3 might
be more appreciated.
Re: Is it possible to share a Hashmap between threads?
This topic can be confusing to beginners because the word "read" is overloaded.
There is a "system call" named "read", but also many higher level APIs that are
"buffered" which are also called "read", and people often refer to the general
activity as just "reading". The buffering means those higher level APIs are
doing "big reads" into a buffer and then copy small parts of that out for you
for each line. If data is copied out shortly after it was read into that big
buffer with a system-call read then the copy is cheap because it was from CPU
caches (L1/L2/L3).
There are also "zero copy" APIs to do IO like "mmap" on Unix, or in the Nim
world "memfiles". These let the OS itself do the "buffering" of what is in RAM
vs "on disk" (or other persistent device like "flash memory" these days).
Your linked yahtzee file is only 900 KB with average line length of about 9
bytes, but only 790 unique entries (so a small Table). With such a short line
length you might profit from not creating new strings at all but re-using the
same one over and over in the loop. That is because string creation and
anything you might do with only 9 bytes take comparable time. The Nim stdlib
also has a `split` iterator which lets you iterate over the split fields
without creating a new `seq[string]`. Looking at your input file format there
is no need to even `split` at all, though. So, there are simpler things than
`memfiles` you can do to speed it up.
Also, because you only have 790 unique lines, it should be much faster to get
the max value by looping over the `Table` after you are done creating it (both
no 2nd lookup, and 790 <<< 100,000).
So, this is a faster version of your program:
import tables, os, parseutils
proc ytz(path: string): uint =
var
table = initTable[uint, uint]()
n: uint
for numStr in lines(path):
if parseUInt(numStr, n) == 0:
quit("Parse error: " & numStr)
table.mgetOrPut(n, 0) += n # here
for key, val in table:
result = max(result, val)
proc main() =
echo ytz(paramStr(1))
main()
Run
Incidentally, when posting code to this forum, if you put `Nim` after the
triple backquote it will be rendered with color syntax highlighting.
Re: Is it possible to share a Hashmap between threads?
> [https://nim-lang.org/docs/sharedtables.html](https://nim-lang.org/docs/sharedtables.html) Thank you, I never noticed it. I love Nim's stdlib! > What would be faster is to read from the file line-by-line. I always thought calling read() over and over again is fairly expensive. That said I wouldn't know better, I'm just a beginner programmer. In my test case, for this input: [https://gist.githubusercontent.com/cosmologicon/beadf49c9fe50a5c2a07ab8d68093bd0/raw/fb5af1a744faf79d64e2a3bb10973e642dc6f7b0/yahtzee-upper-1.txt](https://gist.githubusercontent.com/cosmologicon/beadf49c9fe50a5c2a07ab8d68093bd0/raw/fb5af1a744faf79d64e2a3bb10973e642dc6f7b0/yahtzee-upper-1.txt) which contains a billion lines of texts, reading the entire file seems marginally faster.
Re: Is it possible to share a Hashmap between threads?
> Is there any way I can leverage multiple cores of the target system to > maximize efficiency? Yes, but efficiency can't be guaranteed. The workload must be sufficiently large for the cores to most all of their time processing it, or else the bottleneck will fall into the creation and destruction of threads, which will make it slower than the single threaded version. See [https://forum.nim-lang.org/t/5909](https://forum.nim-lang.org/t/5909) for an example of inefficiency caused by threading. But back to the main question: > Is it possible to share a Hashmap between threads? Yes, you can with this: [https://nim-lang.org/docs/sharedtables.html](https://nim-lang.org/docs/sharedtables.html), note that as the access to the table have to be placed behind a lock, this will actually decrease ability to do multiple core efficiently as they will spend most of the time waiting to have access to the table. Your main bottleneck would be this snippet: paramStr(1).open().readAll().splitLines().ytz() Run What would happen is that you `open` a file, then a cycle of string allocation & read syscalls until the file is read, then all of the resulting string is parsed for new lines, where each line will have their own strings allocated for the resulting seq then eventually passed to `ytz`. And all of that is really really slow. What would be faster is to read from the file line-by-line. This can be done via the `lines` iterator, which will skip the huge overhead from reading then parse from the start to split by line, as the iterator returns each line as it's read from the disk.
Is it possible to share a Hashmap between threads?
In pursuit of optimizing the following code, I realized that if somehow the
loop is paralleled (i.e. each thread operates on a certain chunk of the array),
the code may execute faster:
import tables, os, parseutils, strutils
proc ytz(args: openArray[string]): uint =
var
table = initTable[uint, uint]()
n: uint
for numStr in args:
if parseUInt(numStr, n) == 0:
quit("Parse error: " & numStr)
table.mgetOrPut(n, 0) += n # here
if table[n] > result: # here
result = table[n] # and here
proc main() =
echo(paramStr(1).open().readAll().splitLines().ytz())
main()
Run
However I need to access the table (both mutably and immutably) in the places I
pointed out. Is there any way I can leverage multiple cores of the target
system to maximize efficiency? As far as I know Java has concurrent hashmap but
I never used it myself.
is possible to defer a macro call?
Hi everyone, i'm trying to create a enum type dinamically (not at runtime obviously) using macros. the problem is that the module where is called the macro for generate the enum is loaded before calling the macro for add enum fields and this doesn't add those fields, here is an example group Color group Window group Layer generateGroupEnum(MyEnum) # Macro that i want defer # Generates: # type MyEnum = enum #grColor #grWindow #grLayer group View group Toolbar # Expected Generate if it can defer: # type MyEnum = enum #grColor #grWindow #grLayer #grView -> Added #grToolbar -> Added when isMainModule: echo grLayer # 2 echo grView # Error, not declared Run
Re: Is this possible?
Ooh! That seems perfect...I'll test that out. Thanks.
Re: Is this possible?
you can also consider creating shared object files and load them. Depending on your use case this might be "dynamic" enough?
Re: Is this possible?
See [https://github.com/nim-lang/Nim/blob/devel/tests/compilerapi/tcompilerapi.nim](https://github.com/nim-lang/Nim/blob/devel/tests/compilerapi/tcompilerapi.nim) and [https://github.com/nim-lang/Nim/tree/devel/tests/compilerapi](https://github.com/nim-lang/Nim/tree/devel/tests/compilerapi) for how to embed the compiler into your own projects.
Re: Is this possible?
Alrightyy! Thanks all for the answer.
Re: Is this possible?
@yglukhov, I am keen to understand the embedding of nim compiler. Could you give a example for the same?
Re: Is this possible?
juan's snippet still resolves import at compiletime. nim is not python.
Re: Is this possible?
That's just another method of doing it at compile time. Nim only supports compile time importing. Because it has to compile modules. Compiling is done once for the whole program.
Re: Is this possible?
Thanks that helped me understand it better :) What do you have to say about @juancarlospaco 's answer?
Re: Is this possible?
I think there's some misunderstanding here. The short answer to "is this possible?" is no, this is not possible. Since nim is a compiled language, all imported modules have to be compiled and embedded in the resulting binary. It is possible in dynamic languages that offer eval of some sort, but not in statically compiled languages. Now in theory it is possible to make a lib that would allow eval and dynamic imports in some form, it would embed nim compiler in it, and maybe utilize hot code reloading, but i assure you, that's not something you want to consider at this point.
Re: Is this possible?
const some_module {.strdefine.} = "some_module"
template makeAnImport(module: static[string]): untyped = import module
makeAnImport(some_module)
Run
:)
Re: Is this possible?
I have no idea.
Re: Is this possible?
Alright thanks anyway for the answer :) That atleast got me halfway...
Re: Is this possible?
How can I return the value that is returned in a proc inside the timeit_temp.nim?
Re: Is this possible?
I mean if you want to import module dynaically, yon can write it directly into
file or use when flag. for example,
import streams, strformat
let
module = readline(stdin)
tempFile = getTempDir() & "timeit_temp.nim"
var temp = newFileStream(tempFile, fmWrite)
temp.writeLine(fmt"import {module}")
temp.write(othercode)
dicscard execCmdEx(fmt"nim c -r -f --d:{option} --verbosity=0 --hints=off
--hint[source]=on " & tempFile)
Run
or
when defined(this):
import this
elif defined(that):
import that
Run
Re: Is this possible?
> please write Nim code like this Maybe I missed a lot, but I have no idea what that part of your answer may mean and how it may help for the initial question.
Re: Is this possible?
Thanks! I'll try that
Re: Is this possible?
IMO, Nim can't import module at run time. You must specify the name of module
directly.
import "jester"
Run
If you want to import module at run time, please write Nim code like this:).
var temp = newFileStream(tempFile, fmWrite)
if not isNil(temp):
temp.writeLine("import timeit")
if def == "":
temp.writeLine("""var m = monit("test-whole")""")
temp.writeLine("m.start()")
Run
Is this possible?
var module = "jester" import module Run If not, how do I implement something like this?
Re: Is it possible to see sql executed?
If you use Norm, you can use logging module to log all generated queries, just add import logging; addHandler newConsoleLogger()
Re: Is it possible to see sql executed?
You should be able to just echo string(theQuery) Run
Is it possible to see sql executed?
I'm using Jester to create a web-application. It connects to sqlite using db_sqlite. For debugging purposes is it possible to see sqls executed during development?
Re: Is it possible to browse the nimble.directory?
Search is great if you know what you're looking for. But it is useless for getting an overview of what's available. And an overview of the third-party libraries is what I want when starting with a new language.
Re: Is it possible to browse the nimble.directory?
The current tags are far from descriptive, while categories organize packages only along one arbitrary dimension. A much better taxonomy in used in Debian by Debtags: [https://salsa.debian.org/debtags-team/debtags-vocabulary/blob/master/debian-packages](https://salsa.debian.org/debtags-team/debtags-vocabulary/blob/master/debian-packages) because it implements [https://en.wikipedia.org/wiki/Faceted_classification](https://en.wikipedia.org/wiki/Faceted_classification) For example, nmap is tagged: admin::monitoring, implemented-in::c, interface::commandline, network::scanner, role::program, scope::utility, use::scanning
Re: Is it possible to browse the nimble.directory?
That's one of the many reasons why I listed all the packages available in Nimble this summer, and documented them in a [Google Sheet](https://docs.google.com/spreadsheets/d/1HWy2YumMMcgEDHk34ACauuWR5TYDJTRUVQ6B-LuRXCs/edit?usp=sharing) where you can use the **Categories** columns to regroup them. I had suggested adding this information to Nimble and even proposed a [pull request](https://github.com/nim-lang/packages/pull/1165). Maintaining this information in a file, JSON or Google Sheet is not ideal though. A central repository like [https://nimble.directory](https://nimble.directory) with a database would be the best to implement browsing by categories or displaying tag clouds. Meanwhile I created the [Curated Packages](https://github.com/nim-lang/Nim/wiki/Curated-Packages) page to provide hierarchical categories. As a wiki page, it is more user oriented than tool oriented, and it does not show other information like descriptions or licenses...
Re: Is it possible to browse the nimble.directory?
> lists each tag Unfortunately not all packages have an optimal set of tags. For tagging, once my nimble publish was rejected due to too many tags, then I may had a nimble publish fail maybe because I typed the tags too slowly, then I tried gain with only one fast typed tag and nimble publish was successfull. So rtree package has only library tag I guess. It would be fine if we could select from available tags when we publish a package, and maybe add a new one. And it would be nice if we could improve tags for our package. Maybe we can add tags to the .nimble file which each package has, so we have update tags with a git push? And while we are talking about nimble -- the website entry seems not to work too nice. My rtree first got no website, while there is [http://ssalewski.de/tmp/rtree.html](http://ssalewski.de/tmp/rtree.html). Many packages have website entry identical to url as your $ nimble search aporia aporia: url: https://github.com/nim-lang/Aporia (git) tags:app, binary, ide, gtk description: A Nim IDE. license: GPLv2 website: https://github.com/nim-lang/Aporia Run which is useless noise. Does at least nimble search description takes content from .nimble file? Seems not really to work for me. And for nimble search, would be nice if we get latest version number available and date of last update.
Re: Is it possible to browse the nimble.directory?
You can introduce a page which lists each tag, sorted by how many packages are in that tag. This would already give a nice way to explore our packages.
Re: Is it possible to browse the nimble.directory?
nimble search
Re: Is it possible to browse the nimble.directory?
Unfortunately packages.json is not categorized and the tags are not based on structured classifiers, see [https://github.com/nim-lang/packages/pull/339](https://github.com/nim-lang/packages/pull/339) I'd be happy to add browsing pages if we had categories or if the Curated-Packages had a machine-parsable format
Re: Is it possible to browse the nimble.directory?
Thanks for your helpful replies: and to the others who've replied to my other questions.
Re: Is it possible to browse the nimble.directory?
See also [https://github.com/nim-lang/Nim/wiki/Curated-Packages](https://github.com/nim-lang/Nim/wiki/Curated-Packages) [https://github.com/VPashkov/awesome-nim](https://github.com/VPashkov/awesome-nim)
Re: Is it possible to browse the nimble.directory?
You can look here, is just a JSON [https://github.com/nim-lang/packages/blob/master/packages.json](https://github.com/nim-lang/packages/blob/master/packages.json)
Is it possible to browse the nimble.directory?
The [https://nimble.directory](https://nimble.directory)/ has a good search function, but in reaquainting myself with nim I'd like to be able to see categories and browse. Is this possible?
Is it possible to use MbedTLS for HTTPS instead of OpenSSL?
I need to write HTTPS client application, but I'd like to use MbedTSL (statically linked) for SSL/TLS support instead of OpenSSL. How can I hook it up so that Nim's HTTP protocol implementation would use MbedTLS? Thanks!
Re: Possible to use identifier construction to call proc by name (string)?
I'm pretty sure this a regression, [https://github.com/nim-lang/Nim/issues/12516](https://github.com/nim-lang/Nim/issues/12516)
Re: Possible to use identifier construction to call proc by name (string)?
This is my solution to call functions by name: [https://github.com/krux02/opengl-sandbox/blob/master/examples/console.nim](https://github.com/krux02/opengl-sandbox/blob/master/examples/console.nim) This example in this project does not have any dependentcies of that project itself, it should just work stand alone.
Re: Possible to use identifier construction to call proc by name (string)?
I don't think nameToProc can be a compileTime variable because an address of procedure cannot be determined at compile time.
Re: Possible to use identifier construction to call proc by name (string)?
Ok, I tried to remove compileTime pragma and it runs successfully, hope somebody can give me explanation.
Re: Possible to use identifier construction to call proc by name (string)?
I tried to write the code (by following the manual):
import macros
import tables
var nameToProc {.compileTime.} = newTable[string, proc(): string
{.nimcall.}]()
macro registerProc(p: untyped): untyped =
result = newTree(nnkStmtList, p)
echo repr(p[0])
let procName = p[0]
let procNameAsStr = $p[0]
result.add quote do:
nameToProc.add(`procNameAsStr`, `procName`)
proc foo: string {.registerProc.} = "foo"
proc bar: string {.registerProc.} = "bar"
proc baz: string {.registerProc.} = "baz"
doAssert nameToProc["foo"]() == "foo"
Run
But I get the following error:
Error: unhandled exception: key not found: foo [KeyError]
Run
Re: Possible to use identifier construction to call proc by name (string)?
To expand on what mratsim said, you need to use an explicit string->proc lookup table. This table can be filled via a macro.
Re: Possible to use identifier construction to call proc by name (string)?
proc names must be known at compile time. Use closures instead.
Possible to use identifier construction to call proc by name (string)?
Just like in python/js, we have the following proc: proc newAuthMessage() = ... proc newSubscribeMessage() = ... Run and I want to call these procedures dynamically, so we have string "Auth" , "Subscribe", and now I need concatenate the string and call them. I tried with template but no luck.
Re: strformat: using a ':' in a string literal possible?
Excellent! Thank you.
Re: strformat: using a ':' in a string literal possible?
Ah, I see. I'm curious though why you want to put the raw string in the curly
braces. You could do:
echo fmt("This is a string{colon:<30}Another String")
Run
There is also `alignLeft()` in the strutils module, but it's less attractive:
import strutils
echo "This is a string:".alignLeft(30) & "Another String"
Run
Re: strformat: using a ':' in a string literal possible?
Also, using a simple template, you could do: import strutils template `<`(str: string, padding: int): string = str.alignLeft(padding) echo "This is a string:"<30, "Another String" Run
Re: strformat: using a ':' in a string literal possible?
To follow myself up, this also "works", and requires only one variable
definition for multiple fmt specs:
let colon = ":"
echo fmt("{\"This is a string\"}{colon:<30}{\"Another String\"}")
Run
This is a string: Another String
Run
Re: strformat: using a ':' in a string literal possible?
Hi @jyapayne \- Thanks for the quick response! Yes, that's the "workaround" I've taken to using, but it adds visual clutter if you're formatting a bunch of lines (which I am). Not a biggie, but I wondered if I was overlooking something.
Re: strformat: using a ':' in a string literal possible?
Seems like a bug to me. This works:
let str = "This is a string:"
echo fmt("{str:<30}")
Run
strformat: using a ':' in a string literal possible?
Another newbie question: is it possible to embed a ':' in a string literal in a
fmt specification, as in:
echo fmt("{\"This is a string:\":<30})
Run
yields
/usercode/in.nim(3, 10) Error: could not parse `"This is a string`.
/nim/lib/core/macros.nim(540, 1) Error: closing " expected
Run
Attempting to "escape" the colon:
echo fmt("{\"This is a string\:\":<30})
Run
yields
/usercode/in.nim(3, 31) Error: invalid character constant
Run
How can I sneak a colon in there short of assigning the string to a variable?
Thanks!
Re: Possible minor bug in unused import checker [1.0]
Done. Sorry I'm kind of new to using Github issues. Link for convenience: [https://github.com/nim-lang/Nim/issues/12259](https://github.com/nim-lang/Nim/issues/12259)
Re: Possible minor bug in unused import checker [1.0]
Thank you. Please report this issue on github.
Possible minor bug in unused import checker [1.0]
Hi, signed up to post this. Congrats on achieving 1.0! My two current major projects already built just fine and in fact thanks to the unused imports message I was able to cull a bunch. I did however find a small problem. In one of them I have some modules which I export functionality from. The issue I found is that when one exports an imported module with a different name, i.e. import X as Y export Y Run The compiler still warns about _X_ being an unused import, even if this is used in a module which imports this one. Here's the minimal sample I confirmed this with: [https://anonfile.com/b3ueEe6an1/minimal_sample_zip](https://anonfile.com/b3ueEe6an1/minimal_sample_zip)
Re: Is it possible to detect context in relation to a threadvar?
I'm not sure, but maybe this helps a little: let mainThread = getThreadId() proc isMainThread(): bool = getThreadId() == mainThread Run
Is it possible to detect context in relation to a threadvar?
I'm developing a library that maintains a global pool of resources that can be
seen in both thread and non-threaded contexts. Rather than jump into great
detail, let show this with a silly example:
import strutils
import jester
var someGlobal: string = "value at the moment"
var someThread {.threadvar.}: string
proc getSomeMain(): string =
result = "copy of $1".format(someGlobal)
proc getSomeFromThread(): string {.gcsafe.} =
someThread = "copy of $1".format(someGlobal)
return someThread
someGlobal = "another change"
var x = getSomeMain()
echo x
routes:
get "/":
var y = getSomeFromThread()
resp y
Run
Everything works great.
But, I ask, would it be possible to write a generic equivalent of "getSome".
Something like:
proc getSome(): string =
if MAGIC_DETECT_METHOD:
return "copy of $1".format(someGlobal)
else:
someThread = "copy of $1".format(someGlobal)
return someThread
Run
Or, if not at runtime, some kind of compile-time thing that detects whether
"getSome" is being called from a thread, such as a Macro or Template.
I apologize if the answer to this is well documented somewhere. I've not been
able to find an answer yet.
Re: possible in Nim
Hardly. You can do something like:
template `!foobar`(a) = echo a
`!foobar`("abc")
Run
But I fail to see the point.
possible in Nim
Hello everyone, Is it possible in Nim to have a macro/template that starts with a custom character? For example, a macro like: #Declaration
Re: possible in Nim to have a macro/template
@mratsim: > with backticks... That's really neat! I knew about backticks, but never thought of using them this way. Thanks.
Re: possible in Nim to have a macro/template
@ScottTorres: a Julia fan, eh? > is it possible in Nim to have a macro/template (name - GBG) that starts with > a custom character? No, I don't think it's possible to start a macro name with anything other than a letter character as it would seem to be contrary [to the general naming of (any) identifiers in the manual](https://nim-lang.org/docs/manual.html#lexical-analysis-identifiers-amp-keywords) that they need to start with a letter
Re: possible in Nim to have a macro/template
with backticks macro `@myMacro`(): untyped = discard Run
possible in Nim to have a macro/template
Hello everyone, Is it possible in Nim to have a macro/template that starts with a custom character? For example, a macro like: > #Declaration macro @myMacro() : ... ...
Re: Is it possible to tune C output to be truly human readable?
I think the main issue would be the use of gotos :P
Re: Is it possible to tune C output to be truly human readable?
Have you seen what the generated C/C++ code looks like? Nim generated C code will use nim's seq and string implementation. I don't think you well get that passed C/C++ reviewers...
Re: Is it possible to tune C output to be truly human readable?
Well it's still C code, you can submit it for review and see what happens. ;-)
Is it possible to tune C output to be truly human readable?
As I understand, Nim produces C code just to be used as an interface with backend compiler. Is it theoretically possible to tune code generation to make the generated code be accepted at code review in teams allows only C/C++ as a programming language for development? I mean, that some artifacts can be present, but code should be enough readable and editable if someone wants to do work with this code as a C source manually.
Re: Regex substitutions with Perl's syntax... possible with macros?
Yeah, it would be possible via macros, but I'm not sure what it should do? Replace and return true if there was a replacement?
Regex substitutions with Perl's syntax... possible with macros?
I know Nim already can do this (from documentation):
line =~ re"\s*(\w+)\s*\=\s*(\w+)":
# matches a key=value pair:
echo("Key: ", matches[0])
echo("Value: ", matches[1])
elif line =~ re"\s*(\#.*)":
# matches a comment
# note that the implicit ``matches`` array is different from the
# ``matches`` array of the first branch
echo("comment: ", matches[0])
else:
echo("syntax error")
Run
What about the following syntax for regex substitution? Can it be emulated with
macros ou added to the core language?
string =~ s/regex/replacement/g;
Run
It would be great to have regex syntax sugar in the core language. That would
make Nim even more practical, while still remaining readable.
Re: Is something like EnTT possible with Nim?
@yglukhov You're comment encouraged me to seek out a better ECS tutorial than the one I read prior to my first ECS(-ish). Thank you for that. The tutorial, for anyone interested, is at: [http://t-machine.org/index.php/2007/09/03/entity-systems-are-the-future-of-mmog-development-part-1](http://t-machine.org/index.php/2007/09/03/entity-systems-are-the-future-of-mmog-development-part-1)/
Re: Is something like EnTT possible with Nim?
Thank you for the suggestion, @yglukhov. I'm familiar with ECS and have implemented one before in Go. I wanted to try the same in Nim to help me gain familiarity with the language.
Re: Is something like EnTT possible with Nim?
@stephenwithav it's important to understand the problem ECS solves. ECS is notable for its memory utilization efficiency, gained from cpu-cache-friendly data layout. I'd suggest you read about that before implementing your ECS. But, small spoiler, neither polymorphic objects (interfaces), nor references are the options. At this point, i'd suggest not to consider variant objects either. :)
Re: Is something like EnTT possible with Nim?
Ideally, I'm hoping for something comparable to Golang interfaces. I avoided C since the early '90s due to early compiler incompatibilities. Only recently, in the past few months, have I looked at C/C++ seriously. Thank you for pointing to `variant`. I'm going to look at entt's source more closely and try to find a good equivalent in Nim. (I couldn't seem to make anything useful out of sequtils; Araq, I'm curious as to what you had in mind.)
Re: Is something like EnTT possible with Nim?
> Then I learned this morning that seq[Component] fails. That really should be obvious. A seq in Nim is similar to vectors in C++, that is basically a plain C array that automatically resizes when necessary, where resize is new allocation and copying elements. So all elements must have same type and same size as in C arrays -- as all refs and pointers have the same size, we can have different ref types in a seq and can test for exact type with "of" operator at runtime. Note that we have sum types (objects variants) in Nim.
Re: Is something like EnTT possible with Nim?
I thought I had something good last night: type Position = object x: int y: int type Velocity = object dx: int dy: int dz: int type Component = Position | Velocity proc showProps(p: Position) = echo p.x, ", ", p.y proc showProps(v: Velocity) = echo v.dx, ", ", v.dy, ", ", v.dz let p = Position(x: 9, y: 9) let v = Velocity(dx: 1, dy: 1, dz: 3) showProps(p) showProps(v) Run While tedious, it was good. Then I learned this morning that seq[Component] fails. _sigh_ type Position = object x: int y: int type Velocity = object dx: int dy: int dz: int type Component = Position | Velocity proc showProps(p: Position) = echo p.x, ", ", p.y proc showProps(v: Velocity) = echo v.dx, ", ", v.dy, ", ", v.dz let p = Position(x: 9, y: 9) let v = Velocity(dx: 1, dy: 1, dz: 3) var s: seq[Component] s.add(p) s.add(v) echo s Run
Re: Is something like EnTT possible with Nim?
Thank you, Araq and doofenstein. I'm going to try to develop a VERY simple ECS today in Nim. Somehow, I had totally missed (or forgotten) varargs and set. Those two should help make this first attempt successful.
Re: Is something like EnTT possible with Nim?
Another ECS attempt: [https://github.com/yglukhov/ecs](https://github.com/yglukhov/ecs)
Re: Is something like EnTT possible with Nim?
ha, you might be interested in this ECS implementation, from one of my failed Nim game engine attempts: [https://gist.github.com/RSDuck/6c8caf82aeb88991d440b228b3f32f06](https://gist.github.com/RSDuck/6c8caf82aeb88991d440b228b3f32f06) Note that this is mostly untested. For examples look at the bottom of the file.
Re: Is something like EnTT possible with Nim?
What's hard about `registry.view`? Please look at `sequtils.nim`.
Re: Is something like EnTT possible with Nim?
assign shouldn't be too difficult... just add a subclass of Component to the seq. But filtering for registry.view? I can't figure that one out yet. Templates, maybe?
Is something like EnTT possible with Nim?
Specifically, I'm curious if tuples could take the place of structs, and equivalents for registry.assign and registry.view. I can imagine registry as a seq[int], with each entity being represented by a unique integer, but I don't know how to assign tuples to an int. I really like the ability to filter entities to only those with the specific components of interest. Is there a way to achieve the same in Nim? [EnTT examples](https://github.com/skypjack/entt/blob/master/README.md#code-example)
