Re: Nim's strutils.split() slower than Python's string split()?

2020-04-26 Thread krtekz
You are right. That's why outputs are sent to /dev/null in my tests.

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-26 Thread cblake
If you are not redirecting to a file but letting it display to the terminal then a big variable is what GUI terminal emulator you are using - XTerm, rxvt-unicode, st, etc. I think that could create a lot of varying numbers..why it may even depend on if you use TrueType fonts or cheaper to

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-26 Thread adnan
But OP is not going to divert the output into `/dev/null`. In a Linux terminal you don't really gain much benefits from avoiding flushing because it will flush on a newline regardless. $ time cat bigfile.txt | ./mine ; cat -n mine.nim # ... real0m3.707s user

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-25 Thread krtekz
All my tests were done by redirecting stdout to /dev/null, so the performance of terminal doesn't matter.

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-25 Thread adnan
I was under the impression that Linux terminals usually flush on newlines regardless. There's a chance that the main performance benefit you are gaining is not from splitting the strings and using the split iter instead. I could be wrong.

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-25 Thread ksandvik
You could also concatenate all entries to a string variable and then do a single writeLine to avoid I/O traffic -- a classic optimization technique.

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-25 Thread cblake
Locking can indeed slowdown output as can exception handling and is unnecessary for single-threaded output and the exceptions may be unhelpful if there is nothing you can do on out of space errors. If you are on Linux/glibc then you can also just use `fwrite_unlocked` directly as in

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-25 Thread krtekz
See updated code above

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-25 Thread kaushalmodi
@krtekz Awesome! Also make sure you share your optimized final code here for anyone who ends up on this thread in future :)

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-25 Thread Araq
Also try `--gc:arc --panics:on`

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-25 Thread krtekz
Just found out something people haven't mentioned yet (or I missed it?). According to [https://nim-lang.org/docs/system.html](https://nim-lang.org/docs/system.html) `echo` is equivalent to a `writeLine` and a `flushFile`, so each time I call `echo` it flushes the output, which is not

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-25 Thread adnan
In Rust, sometimes the biggest performance overhead is repeated locking of stdout. To avoid it, one would lock the stdout before the loop. I wonder Nim requires locking as well. { let mut out = File::new("test.out"); let mut buf = BufWriter::new(out); let

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-24 Thread cblake
This comes up a lot. Here are a couple recent ones: [https://forum.nim-lang.org/t/4738](https://forum.nim-lang.org/t/4738) and [https://forum.nim-lang.org/t/5103](https://forum.nim-lang.org/t/5103).

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-24 Thread krtekz
Thanks a lot! Total newbie here

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-24 Thread mratsim
Your code is slow for several reasons: 1\. You should wrap your code in a proc, otherwise all variables are globals and globals are much harder to optimize, in particular they have the same lifetime as your program so you can't reuse their memory 2\. `split` in each loop, you are allocating 1

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-24 Thread krtekz
With `-d:danger`, it improves a bit but still much slower than the Python version. Nim: 2.65s, Python: 1.26s

Re: Nim's strutils.split() slower than Python's string split()?

2020-04-24 Thread juancarlospaco
Compile with `-d:danger`.