Re: recvmsg on AsyncSocket
File descriptors (in unix) are only unique to the process, they are not meaningful when passed between processes. They are however meaningful when passed to a child process created by fork(). Please see [https://www.ibm.com/support/knowledgecenter/en/ssw_aix_72/generalprogramming/using_file_descriptors.html](https://www.ibm.com/support/knowledgecenter/en/ssw_aix_72/generalprogramming/using_file_descriptors.html)
Re: Cryptic error message relating to procedure type mismatch
Done, and solved, thank you.
Re: FOSDEM CfP deadline is 26/11/19
Submitted 3 talks: * Nim - A walk-through of a new emerging language * Async await in Nim - A demonstration of the flexibility metaprogramming can bring to a language * Karax - SPAs in a compiled systems programming language I'm quite proud of my abstract for the first one, the others I threw in as a backup. Hoping the first one is picked :)
Re: "out of memory" when creating new Thread inside sharedTable lock
Thanks for testing it!
I tried different combinations for writing thread instance to sharedTable.
Seems that creating temporary variable may help, but other cases simply don't
work:
import sharedtables
import strformat
import os
import typetraits
type ThreadInfo = object
thread: Thread[void]
var threads: SharedTable[string, ThreadInfo]
threads.init()
proc threadEntrypoint() {.thread.} =
while true:
echo "OK"
when isMainModule:
let key = "key"
# 1) works (well, if it didn't work nim would worth nothing)
var value = ThreadInfo()
value.thread.createThread(tp=threadEntrypoint)
# 2) illegal storage access
threads.withKey(key) do (key: string, value: var ThreadInfo, pairExists:
var bool):
if pairExists:
raise newException(ValueError, "Thread with this name was already
registered")
value = ThreadInfo()
value.thread.createThread(tp=threadEntrypoint)
pairExists = true
# 3) works
threads.withValue(key, value) do:
echo "Value already exists"
do:
var info = ThreadInfo()
info.thread.createThread(tp=threadEntrypoint)
threads[key] = info
# 4) hangs forever
threads.withValue(key, _) do:
echo "Value already exists"
do:
threads[key] = ThreadInfo()
threads.mget(key).thread.createThread(tp=threadEntrypoint)
sleep(1000)
Run
Strange Crash
Traceback (most recent call last) /home/stefan/pet2/pet.nim(717) pet /home/stefan/pet2/pet.nim(714) test /home/stefan/pet2/pet.nim(683) newDisplay /home/stefan/.nimble/pkgs/gintro-#head/gintro/gio.nim(24220) run /home/stefan/Nim/lib/core/macros.nim(570) connect_for_signal_cdecl_scroll_event4 /home/stefan/pet2/pet.nim(584) scrollEvent /home/stefan/pet2/pet.nim(518) updateAdjustmentsAndPaint /home/stefan/pet2/pet.nim(479) paint /home/stefan/pet2/pet.nim(434) draw /home/stefan/pet2/pet.nim(334) drawSelected /home/stefan/.nimble/pkgs/gintro-#head/gintro/cairoimpl.nim(243) popGroup /home/stefan/Nim/lib/system/gc.nim(440) newObj /home/stefan/Nim/lib/system/alloc.nim(777) rawAlloc SIGSEGV: Illegal storage access. (Attempt to read from nil?) Run I get this in a drawing app rarely. I assume that it is not really GC related but a bug in my own code, maybe the GTK/Cairo bindings. My code may damage some data structures of GC.
Re: "out of memory" when creating new Thread inside sharedTable lock
your example works for me (on win10, Nim Compiler Version 1.1.0 [Windows: amd64])
Re: asyncftpclient troubles
You have to specify a password for anonymous login.
This works for me:
import asyncdispatch, asyncftpclient
proc main() {.async.} =
echo "trying to connect..."
var ftp = new_async_ftp_client("speedtest.tele2.net", user =
"anonymous",
pass = "in ancients times i was an email :)")
await ftp.connect()
echo "connected"
var x = await ftp.send("pwd")
echo x
waitFor(main())
Run
"out of memory" when creating new Thread inside sharedTable lock
Hello guys! I need your help, cannot figure out wtf is going on here.
I made a minimal example which produces an "out of memory" error. The idea is
simple: create a "spawn" proc which
1. accepts a name (just a string)
2. runs some dumb thread
3. adds mapping to shared table: name -> thread
Since I use shared table, each thread can (safely, due to lock) access other
thread information using thread's name.
Here is the code:
import sharedtables
import strformat
import os
import typetraits
type
ThreadName = string
ThreadInfo = object
thread: Thread[ThreadName]
# ... and some other info (channel etc)
var threads: SharedTable[ThreadName, ThreadInfo]
threads.init()
proc threadEntrypoint(name: ThreadName) {.thread.} =
echo &"Inside thread: name={name}"
proc spawn(name: ThreadName) =
echo &"Spawning: name={name}"
threads.withKey(name) do (key: ThreadName, value: var ThreadInfo,
pairExists: var bool):
if pairExists:
raise newException(ValueError, "Thread with this name was already
registered")
echo &"withKey: name={key}"
value = ThreadInfo()
echo &"Creating thread: name={key}"
value.thread.createThread(param=key, tp=threadEntrypoint)
echo &"Created thread: name={key}"
pairExists = true
when isMainModule:
spawn(name="a")
sleep(1000)
Run
The output is really surprising, I cant google something similar and it's even
not an exception:
Spawning: name=a
withKey: name=a
Creating thread: name=a
Created thread: name=a
out of memory
Run
It's probably something related to shared table lock and spawning a thread
during this lock (but im not sure). I tried different ways to fill that shared
table with spawned threads but didn't succeed.
Re: onSignal closure memory violation
Try this: import os from posix import onSignal, SIGINT type App = object template cleanup(app: App) = discard proc start*(app: App) = onSignal(SIGINT): echo "Exiting: ", sig app.cleanup while true: os.sleep(1) var app = App() Run
Re: How to parallelize a for-loop
> love to know how to harness the multicore capability inside a for loop in Nim. Have you already played with [https://nim-lang.org/docs/manual_experimental.html#parallel-amp-spawn](https://nim-lang.org/docs/manual_experimental.html#parallel-amp-spawn) But I think there may coming other ways eventually, I think using openMP is possible too, and PMunch and mratsim may be writing about this topic.
Re: Which is the fastest way to use a timer
The fact that the time of times module is not monotonic is a bit surprising for me -- well maybe when the user sets the wall time of his box? The fact should be noted in times module. But for all benchmarks done with times.getTime() I have never seen a negative delta yet.
Re: recvmsg on AsyncSocket
Sorry to bump this but anyone here ever succeeded to pass a file descriptor through an ipc socket in nim async ? It seems to me that it's a common practice when writing servers and I'm surprised nim would not cover it.
Re: Which is the fastest way to use a timer
AFAIK, The "-" operator returns a Duration type when we use "x = a-b", where both a and b are monotimes. Then Can i use "x.seconds" or "x.nanosecond" ?
Re: Which is the fastest way to use a timer
> I cant find any timers on nim manual. But i found times module. Why would you expect timers in the Nim manual? Nim is not a kind of virtual basic which may integrate all the user stuff into the language. And when you have already times module, you may inspect its functions: [https://nim-lang.org/docs/times.html#getTime](https://nim-lang.org/docs/times.html#getTime) It is specified with ns resolution, and is often used for benchmark tests. Of course you use it with deltas, like t0 = getTime(), ... , echo "Elapsed:", getTime - t0.
Re: Which is the fastest way to use a timer
<<< echo "Elapsed:", getTime - t0. >>> Hi, thanks for the reply. Instead printing, result, i would like to store the result in a variable. So what is the return data type of "-" operator ? a Time type ? If so can i use let x = getTime - t0 echo x.nanosecond Run
Re: Which is the fastest way to use a timer
Well, this code return error [https://play.nim-lang.org](https://play.nim-lang.org)/
Re: Which is the fastest way to use a timer
@pmags, Thanks for the reply. Let me check. :)
Re: Which is the fastest way to use a timer
Perhaps `std/monotimes` ([https://nim-lang.org/docs/monotimes.html](https://nim-lang.org/docs/monotimes.html)) might work for you.
Re: Advent of Nim 2019 megathread
I should participate again I guess. I fear I'll have even less time than last year though. We'll see!
Which is the fastest way to use a timer
Hi all, I need to measure the time difference of mouse down event and mouse up event. So I want to start a timer in the mouse down event and then in the mouse up event, i want to stop the timer and get the time difference. And i want the time difference in milliseconds or micro seconds. I cant find any timers on nim manual. But i found times module. Some on IRC chat suggest me to use "now()" on both events and get the difference. But what is the datatype of "now1() - now2()" ? Is that fast enough ?
Re: Problem using sometable.keys() in this case
keys is an iterator so either iterate over it with a for loop or use toSeq from the sequtils module
Re: How to parallelize a for-loop
I agree, however it would perform better if there were too many arguments
(which is an edge case). Nonetheless I would love to know how to harness the
multicore capability inside a for loop in Nim.
For reference here's the program in D:
import std.stdio : writeln;
import std.algorithm : splitter;
import std.range : chain;
import std.path : dirSeparator, pathSeparator;
import std.process : environment;
import std.file : exists;
pragma(inline, true) public void which(in ref string bin) {
foreach (path; environment.get("PATH", "").splitter(pathSeparator)) {
auto probe = chain(path, dirSeparator, bin);
if (probe.exists()) {
writeln(probe);
break;
}
}
}
void main(in string[] args) {
if (args.length == 2) {
which(args[1]);
}
else {
version (par) {
import std.parallelism : parallel;
foreach (arg; args.parallel())
which(arg);
}
version (sng) {
foreach (arg; args)
which(arg);
}
}
}
Run
$ gdc source/app.d -fversion=par -O3
$ time ./a.out $(ls /usr/bin)
// ...
0.040s
$ gdc source/app.d -fversion=sng -O3
// ...
0.084s
$ time ./a.out $(ls /usr/bin)
// ...
Run
Advent of Nim 2019 megathread
This Sunday, December 1st, will mark the start of the fifth edition of [Advent of Code](https://adventofcode.com/). Advent of Code is, to quote its author, _“a series of small programming puzzles for a variety of skill sets and skill levels in any programming language you like”_. The rules of AoC are quite simple. Starting from December 1st until Christmas, every day at 5 a.m. UTC a new task is released. The tasks consist of two parts, where second part is revealed after you solve the first part, and it is a continuation and/or variation of the first part. You don’t submit your code, just the result of your calculation. The participation in AoC is free (although, if you like it, consider [donating](https://adventofcode.com/2018/support)), all you need to do is log in with your Github, Google, Twitter, or Reddit account. If you have never participated in AoC before or you want to prepare yourself for the start of the competition by solving some tasks, take a look at [the previous events](https://adventofcode.com/2019/events). # Nim leaderboard We are organizing the official Nim [private leaderboard](https://adventofcode.com/2018/leaderboard/private), where you can compete against other Nim users. To join the Nim leaderboard, all you have to do is use `40415-c732e66e` code in the previous link. If you are one of 120 (!) people who have joined this leaderboard last year, there's no need to do it again — you're already in. # Misc You can use this thread to post links to your AoC repositories, share your solutions, ask for help, discuss the tasks, etc. but have in mind these things: 1\. **SPOILERS** : Try to keep this thread as spoiler-free as possible: [r/adventofcode subreddit](https://old.reddit.com/r/adventofcode/) will have megathreads for each day and you can post there as soon as the global leaderboard is populated (and that is veeery quickly) — we would like to encourage you to share your Nim solutions there and showcase the beauty of Nim. 2\. **ASKING FOR HELP** : Feel free to ask for help either here or on Nim IRC/Gitter/Discord channel if you have some Nim-related problem, but have in mind that your snippets might contain spoilers for other who haven't solved the task yet — not everybody will be able to solve the tasks at 5 a.m. UTC. Consider waiting at least couple of hours before asking for help (in that time, try it some more to see if you can solve it by yourself :)). 3\. **HAVE FUN** : This is mandatory! # Our AoC 2019 repositories (this will be populated and continuously updated as people share links to their repos)
Re: Seq's and string with --gc:none
Use --gc:destructors
Re: Seq's and string with --gc:none
I've just tried it, but the compiler emits a Error: system module needs: nimGCvisitSeq error
Seq's and string with --gc:none
Is there any way I can use sequences and strings _without_ a GC? (basically, that's exactly what I'm doing - Nim doesn't complain at all - but I guess I'm leaking too much memory) How do I manually allocate/deallocate a seq/string or whichever object for that matter?
Re: FOSDEM CfP deadline is 26/11/19
Submitted 2 talks: * "Making multi-dimensional arrays first-class in Nim" * "Designing an ultra-low overhead multithreading runtime for Nim"
Re: Async loops & multi-processing
You're mistaken, there is no OpenMP on OSX, Apple wants to force everyone to
use Grand Central Dispatch (which as someone writing a multithreading runtime,
must say that it's a fine library but not portable on Windows at the moment).
So if you're benchmarking on OSX you should get GCC or Clang from Homebrew.
To wait for all you can add `sync()` at the end.
To wait for some, unfortunately you must return a dummy value at the moment due
to
[https://github.com/nim-lang/Nim/issues/8040](https://github.com/nim-lang/Nim/issues/8040).
So without further ado
import cpuinfo, times, math, threadpool
# Bench script
#
template benchmark(benchName: string, body: untyped) =
# When multithreading, make sure to measure wall-clock time
# If you use CPU time you might measure the cumulated CPU time on each
processor.
let start = epochTime()
body
let stop = epochTime()
let elapsed = stop-start
echo "Wall time for ", benchName, ": ", round(elapsed, 3), " s"
# OpenMP
#
# Add OpenMP to compilation flags
{.passC:"-fopenmp".}
{.passL:"-fopenmp".}
# Nim native threading
#
template parallelChunks(start, stop: int, chunkOffset, chunkSize, threadID:
untyped{ident}, body: untyped): untyped =
## In-place declare and define "chunkOffset" and "chunkSize"
## That corresponds to a slice of the start..stop range
## that will be processed on the same core
let
numIters = (stop - start)
numChunks = countProcessors()
baseChunkSize = numIters div numChunks
remainder = numIters mod numChunks
# The following simple chunking scheme can lead to severe load imbalance
#
# `chunkOffset`{.inject.} = chunkSize * threadId
# `chunkSize`{.inject.} = if threadId < nb_chunks - 1: chunkSize
# else: numIters - chunkOffset # remainder if
division isn't exact
#
# For example dividing 40 items on 12 threads will lead to
# a base_chunk_size of 40/12 = 3 so work on the first 11 threads
# will be 3 * 11 = 33, and the remainder 7 on the last thread.
# Instead of dividing 40 work items on 12 cores into:
# 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 7 = 3*11 + 7 = 40
# the following scheme will divide into
# 4, 4, 4, 4, 3, 3, 3, 3, 3, 3, 3, 3 = 4*4 + 3*8 = 40
var chunkOffset {.inject.}, chunkSize {.inject.}: Natural
for threadID{.inject.} in 0 ..< numChunks:
if threadID < remainder:
chunkOffset = start + (baseChunkSize + 1) * threadID
chunkSize = baseChunkSize + 1
else:
chunkOffset = start + baseChunkSize * threadID + remainder
chunkSize = baseChunkSize
block: body
# Benches
#
let maxLim = 1_000_000
proc doSth(i: int) =
let k = i*2
benchmark "normal loop":
for i in 1..maxLim:
doSth(i)
# Don't interleave stacktraces, they require to allocate strings
# and OpenMP doesn't create a GC or would require "setupForeignThreadGc()"
{.push stackTrace:off.}
benchmark "parallelized OpenMP":
for i in 1||maxLim:
doSth(i)
{.pop.}
type Dummy = bool
## Allow waiting on void spawns:
https://github.com/nim-lang/Nim/issues/8040
proc chunkedDoSth(chunkOffset, chunkSize: Natural): Dummy =
## A chunk is processed on the same core
## It returns a dummy calue so that we can wait on it
for i in chunkOffset ..< chunkSize:
doSth(i)
# Let's not use heap allocation for this, but we need to wrap in a proc
when defined(windows):
proc alloca(size: csize): pointer {.header: "".}
else:
proc alloca(size: csize): pointer {.header: "".}
proc parNimSpawn() =
var tasks: ptr UncheckedArray[FlowVar[Dummy]]
tasks = cast[type tasks](alloca(countProcessors() *
sizeof(FlowVar[Dummy])))
# Transforming this into a nice "parallel_for" is left as an exercise to
the reader
parallelChunks(1, maxLim, chunkOffset, chunkSize, threadID):
# Spawn a task for each chunk
tasks[threadID] = spawn chunkedDoSth(chunkOffset, chunkSize)
# Wait all
for i in 0 ..< countProcessors():
let dummy = ^tasks[i]
benchmark "parallelized Nim spawn":
parNimSpawn()
Run
Re: How to parallelize a for-loop
You should edit your post and add the code directly inside. Here you are trying to parallelize command line arguments which seems to be directory paths to check if a file exist. This is IO-bound and your bottleneck will be your disk. Parallelizing this loop will probably slow your program down due to parallel overhead and also competition over the kernels limited file descriptors.
Re: env needs to be copied for closure
To add it into `system.nim` is not a good idea, `macros.nim` looks like a better place. Here is the PR: [https://github.com/nim-lang/Nim/pull/12712](https://github.com/nim-lang/Nim/pull/12712).
Re: Cryptic error message relating to procedure type mismatch
the project has been updated recently, please pull the latest (which is version 1.1.3)
