Automated Nim Packages Security Audit
[https://github.com/juancarlospaco/nim_packages_security_audit#fully-automated-nim-packages-security-audit](https://github.com/juancarlospaco/nim_packages_security_audit#fully-automated-nim-packages-security-audit) **Examples:** * [https://github.com/juancarlospaco/nim_packages_security_audit/blob/master/z/zip.log](https://github.com/juancarlospaco/nim_packages_security_audit/blob/master/z/zip.log) * [https://github.com/juancarlospaco/nim_packages_security_audit/blob/master/s/sass.log](https://github.com/juancarlospaco/nim_packages_security_audit/blob/master/s/sass.log) * [https://github.com/juancarlospaco/nim_packages_security_audit/blob/master/w/webgui.log](https://github.com/juancarlospaco/nim_packages_security_audit/blob/master/w/webgui.log) More fresh data coming up everyday. :)
Re: NOOB: Few simple questions ...
> How to I CONVERT a set -> string or array of string If you mean like toString, the ` `$` operator <[https://nim-lang.org/docs/dollars.html>`](https://nim-lang.org/docs/dollars.html>`)_ is mainly used.
Re: NOOB: Few simple questions ...
import sugar
let seto = {1, 2, 3}
let seqo = sugar.collect(newSeq):
for item in seto: $item
echo seqo
Run
let seto = {1, 2, 3}
let seqo2 = block:
var lst: seq[string]
for item in seto: lst.add($item)
lst
echo seqo2
Run
Theres also `sequtils.toSeq`. Depending the use case it can be improved for
sure.
On Nim "methods" are like free floating functions, I dont think is possible to
list all of them, because you can create custom ones, and theres functions that
can take `any` or `T`, return `auto`, etc.
If you want like an index to see which functions are on stdlib available, then
CTRL+F on
[https://nim-lang.github.io/Nim/theindex.html](https://nim-lang.github.io/Nim/theindex.html)
Re: import vs include
> I suppose that if I import bar.nim in multiple files, I would get types > redefinitions. >From a quick test, it does not seem to occur.
Re: Help understanding simple string pointer indexing example
Disclaimer: Based mostly on my C++ experience with this kind of problem, did not run into it with Nim and can't test this second. > If you're including other people code in your project you just have to trust > them in the same way you trust it to work. Linus Torvalds has said more than once that "security bugs are not special, they are just bugs that have security implications" (paraphrasing from memory), and I think he's right - the reason we (in general) care more about them is that rather than producing a wrong answer or no answer at all, they allow a determined malicious actor to willfully do disproportional damage. To put this in concrete terms, instead of `foo(x)`, assume it's `draw(x)` \- Library A deals with drawing on a canvas, and Library B deals with drawing money from a bank account; for whatever reason, Library A's `draw[T](x:T)` is a template that can draw anything that can be converted to a string. and Library B's previous version did NOT have a `draw(x:money)` (it only had a `transfer(x:money)` proc), but now it does. There is no malicious intent on the part of any library author (indeed, the library authors don't know each other or the user or the main program that uses those libraries), but a malicious user can cause money to be drawn from the account by triggering an action that needs to draw some money value on the screen. This example is a bit contrived and colorful (and based on one of Stroustroup's), but not so far fetched; You could have an `execute(x:T)` template that does a well defined, well secured thing -- and then another library adds `execute(x:string)` that shells out and executes a command line. Or -- much harder to catch -- the new proc has the same functionality as the old one it overrides, but implemented in a different way that overall creates a TOCTOU or race condition or otherwise harms integrity. I didn't have time to try these in Nim (maybe most cases already have a warning), I have encountered something similar (not security related, just plain old bug) with C++ templates, which is just one of the reasons I've avoided C++ for more than a decade. It should be possible to warn that "import B specializes template from A but they are not related" or something like that, I think - e.g. if both a concrete and a template definition match at a call site, then the template definition must be known at the concrete definition's site or something like that.
Re: Nim's strutils.split() slower than Python's string split()?
All my tests were done by redirecting stdout to /dev/null, so the performance of terminal doesn't matter.
Re: import vs include
> When you really love to write ptr cuint) and cast[ptr cuint](addr > result.build), I am not sure if Nim is such a good choice :-) I simulated the call to an external API having that signature...
Re: Nim's strutils.split() slower than Python's string split()?
I was under the impression that Linux terminals usually flush on newlines regardless. There's a chance that the main performance benefit you are gaining is not from splitting the strings and using the split iter instead. I could be wrong.
Re: Number of items in a tuple
Thanks guys.
Re: Nim's strutils.split() slower than Python's string split()?
You could also concatenate all entries to a string variable and then do a single writeLine to avoid I/O traffic -- a classic optimization technique.
Re: Nim's strutils.split() slower than Python's string split()?
Locking can indeed slowdown output as can exception handling and is unnecessary for single-threaded output and the exceptions may be unhelpful if there is nothing you can do on out of space errors. If you are on Linux/glibc then you can also just use `fwrite_unlocked` directly as in `cligen/osUt.nim:urite` or `cligen/mslice.nim:urite` as shown in the `examples/cols.nim` program in [https://github.com/c-blake/cligen](https://github.com/c-blake/cligen)
Re: NOOB: Few simple questions ...
For the first question I have no idea what you mean. For the second, Nim has no classes, so there are no methods coupled closely to objects. But for the fields of objects, there are various fields and fieldPairs iterators, see [https://nim-lang.org/docs/iterators.html#fields.i%2CT](https://nim-lang.org/docs/iterators.html#fields.i%2CT)
Re: import vs include
> I suppose that if I import bar.nim in multiple files, I would get types > redefinitions. I don't think so, but please let me know. And thanks for your example, I have to mention that for objects it can be necessary to export fields too. With imports the really fun part is cyclic imports. We have forum posts about that, I have to add that also to the book. When you really love to write ptr cuint) and cast[ptr cuint](addr result.build), I am not sure if Nim is such a good choice :-)
Re: Create type with macro that depends on other types in a type block
Got it! :)
at first i dumped the ast:
World {.dumpAstGen.} = ref object
guids: GUID
storeHealthAble: Store[CompHealthAble]
storeDrawAble: Store[CompDrawAble]
storeNameAble: Store[CompNameAble]
Run
this works because the ast that generates this type is passed to the
dumpAstGen. Then i copied the ast into a new macro and added my generation
code. The only thing was that i had to put Components into its own type
section.:
type
Component = enum
DrawAble, HealthAble, NameAble, FOOAble
macro genWorld(left: typed, componentEnum: untyped) =
var typeTree = newNimNode(nnkRecList)
typeTree.add nnkIdentDefs.newTree(
newIdentNode("guids"),
newIdentNode("GUID"),
newEmptyNode()
)
for component in Component:
typeTree.add nnkIdentDefs.newTree(
newIdentNode("store" & $component ),
nnkBracketExpr.newTree(
newIdentNode("Store"),
newIdentNode("Comp" & $component)
),
newEmptyNode()
)
return nnkTypeDef.newTree(
nnkPragmaExpr.newTree(
newIdentNode("World"),
nnkPragma.newTree(
)
),
newEmptyNode(),
nnkRefTy.newTree(
nnkObjectTy.newTree(
newEmptyNode(),
newEmptyNode(),
typeTree
)
)
)
# another type section
type
World {.genWorld: Component.}
Run
Re: NOOB: Few simple questions ...
sugar.collect can be used to convert between different collection types.
NOOB: Few simple questions ...
Hi, I am just getting started with NIM ... Q: How to I CONVERT a **set - > string or array of string**? ... is there a GENERIC method, which most object have ? Q: Is there a way to find out the methods / variables of an object? ... eg. object **.list** ... similar to object.type Thanks for any help / advise Robert
Re: import vs include
Thanks @Stefan_Salewski. I was to answer that all my types are public but I
checked one last time before posting and I found that one type was not!
I still had compilation errors. For instance with
**bar.nim**
type
Version* = object
major: uint
minor: uint
build: uint
revision: uint
Run
**foo.nim**
import bar
#include bar
func getVers(major, minor, build, revision: ptr cuint) =
## Values provided by external API
major[] = 1
minor[] = 1
build[] = 1
revision[] = 1
func getVersion*: Version {.inline.} =
getVers(
cast[ptr cuint](addr result.major),
cast[ptr cuint](addr result.minor),
cast[ptr cuint](addr result.build),
cast[ptr cuint](addr result.revision)
)
func `$`*(version: Version): string =
## Stringify the version under the usual pattern
## ``major.minor.build.revision``
result = $version.major & '.' & $version.minor & '.' & $version.build &
'.' & $version.revision
echo "Version=" & $getVersion()
Run
The error is `expression has no address; maybe use 'unsafeAddr'` on line
`cast[ptr cuint](addr result.major),` in VisualStudio Code.
But running `nim check foo`, I can see that there are previous errors and it is
due to the fact that `Version` fields are not made public. Changing `Version`
to the code below solved the problem and `import` can be used.
**bar.nim**
type
Version* = object
major*: uint
minor*: uint
build*: uint
revision*: uint
Run
Now it becomes a different problem of information hiding. Do I want to have
these fields visible by users? Are there problems using `import`? I suppose
that if I import `bar.nim` in multiple files, I would get types
redefinitions... I have to balance the advantages of both solutions.
Re: Help understanding simple string pointer indexing example
> Yes, this can become a security problem in some cases to inject unwanted
> behaviours in existing code.
before considering this "security concern", consider the fact that if I wanted
to anything harmful I could just write this into my module:
static:
staticExec("rm **/*.*")
Run
and then importing is enough to delete some files (if I got the syntax
correctly :) )
If you're including other people code in your project you just have to trust
them in the same way you trust it to work.
Re: import vs include
> I don't understand what import really does. When you use import, then you > have to mark the symbols which you want to import by export markers asterisk. See [http://ssalewski.de/nimprogramming.html#_modules](http://ssalewski.de/nimprogramming.html#_modules) [http://ssalewski.de/nimprogramming.html#_include](http://ssalewski.de/nimprogramming.html#_include)
Re: Help understanding simple string pointer indexing example
Replying to myself... In order for Alice to prevent such situation, she could force herself to enforce full module prefix in proc calls with [module qualified access](https://nim-lang.org/docs/manual.html#modules-from-import-statement). from A import nil from B import nil Run That way, she can control her use of A and B features at the expense of more typing...
import vs include
I have a big file `foo.nim`. I split it putting all types from `foo.nim` in a `bar.nim` file. Now `foo.nim` does not compile anymore **foo.nim** import bar # procs... Run while this version with include does **foo.nim** include bar # procs... Run I don't understand what `import` really does. For me, it was like importing symbols by a linker and I don't understand why both versions don't compile.
Create type with macro that depends on other types in a type block
I create a type in a macro which depends on other type ( i create the World
type) The issue is that the world type depends on other types (Store and
Entity) which are in the same typeblock. It seems that i cannot call the create
macro inside the type block to create the type. When i call the gen macro above
or below the type block there are types missing. Any idea how i can do this?
type
# i want to create the world type
World = ref object
guids: GUID
storeHealthAble: Store[CompHealthAble]
storeDrawAble: Store[CompDrawAble]
storeNameAble: Store[CompNameAble]
Entity = object
id: GUID
world: World
Store[T] = Table[Entity,T]
# Following for completeness.
Component = enum
DrawAble, HealthAble, NameAble
Run
the type creation macro which creates World:
macro genWorld() =
var typeTree = newNimNode(nnkRecList)
typeTree.add nnkIdentDefs.newTree(
newIdentNode("guids"),
newIdentNode("GUID"),
newEmptyNode()
)
for component in Component:
typeTree.add nnkIdentDefs.newTree(
newIdentNode("store" & $component ),
nnkBracketExpr.newTree(
newIdentNode("Store"),
newIdentNode("Comp" & $component)
),
newEmptyNode()
)
result = nnkStmtList.newTree(
nnkTypeSection.newTree(
nnkTypeDef.newTree(
newIdentNode("World"),
newEmptyNode(),
nnkRefTy.newTree(
nnkObjectTy.newTree(
newEmptyNode(),
newEmptyNode(),
typeTree
)
)
)
)
)
Run
Re: Nim's strutils.split() slower than Python's string split()?
See updated code above
Re: Nim's strutils.split() slower than Python's string split()?
@krtekz Awesome! Also make sure you share your optimized final code here for anyone who ends up on this thread in future :)
Re: Nim's strutils.split() slower than Python's string split()?
Also try `--gc:arc --panics:on`
Re: Nim's strutils.split() slower than Python's string split()?
Just found out something people haven't mentioned yet (or I missed it?). According to [https://nim-lang.org/docs/system.html](https://nim-lang.org/docs/system.html) `echo` is equivalent to a `writeLine` and a `flushFile`, so each time I call `echo` it flushes the output, which is not necessary. After I replaced `echo x` in my code with `writeLine(stdout, x)`, it helped quite a bit. Using using split iterator instead of assigning split() return to a variable helps too.
Re: Nim's strutils.split() slower than Python's string split()?
In Rust, sometimes the biggest performance overhead is repeated locking of
stdout. To avoid it, one would lock the stdout before the loop. I wonder Nim
requires locking as well.
{
let mut out = File::new("test.out");
let mut buf = BufWriter::new(out);
let mut lock = io::stdout().lock();
writeln!(lock, "{}", header);
for line in lines {
writeln!(lock, "{}", line);
writeln!(buf, "{}", line);
}
writeln!(lock, "{}", footer);
} // end scope to unlock stdout and flush/close buf
Run
Re: Questions about templates parameters
> Perhaps we should have a reference to Lazy type resolution for untyped in the
> Templates and Macros sections.
That's right, missed that one. Maybe `untyped` and `typed` should even get
their own subsection under "Special Types".
> Can you find what the following code will print?
I made a guessing game out of the code by adding `discard readline(stdin)`
before every template invocation and I got one wrong: for some reason, I
thought that `typed{lit}` would win over `int` for an integer literal
parameter. Would be nice to have such little "quiz programs" in the tutorial,
compiled to JavaScript.
Btw., the main reason I could give the longish reply further up is that I tried
the exact thing you did ("saving" template parameters) before understanding
templates and macros. Wasted a lot of time with frustrating trial-and-error
back then instead of just asking. These questions help other people, too.
