Re: Sort a table by value
The examples I saw for sorting OrderedTable were for the key, not sorting by the value, so I'm not sure if what I need can be done with an OrderedTable.
Re: Sort a table by value
Thanks, I'll probably just a seq of tuples.
Re: Sort a table by value
A Table has no ordering, so you’d have to first copy it to some form that does, like a seq of (int, string) tuples, and then sort that (e.g. using a comparator proc that returns the result of comparing the two strings.)
Re: Sort a table by value
FYI there's an OrderedTable and it has a .sort proc. So you can convert your table to an OrderedTable and then sort it :) Or just use OrderedTable right from the start
Sort a table by value
I have a table: > my_table: Table[int, string] How do I sort by the table by the value field?
Re: Python transpiler
I have a 200K-line Python app that uses 5-10 C extensions plus a few Cython modules I wrote for content-defined file chunking and a dedup hash table. Performance in general is good, but Python has its limits in several ways: * threading allows sharing data but the GIL limits performance * multiprocessing allows message passing without the GIL, but there's overhead to code, exchange, and decode messages * large data structures become impossibly large because of 24/28 byte integers * operating on large data structures is slow because of Python's VM I love Python, and as one of the most popular programming languages in the world, if not _the_ most popular, other people apparently love it too. But I have hit its limits. If it was possible to get reasonably maintainable Nim code from Python source, even if it required extensive editing, I think that would be huge. It's not like we're talking about converting some niche language like OCaml, Scala, Haskell, Erlang, Elixir, etc. to Nim. That would be a waste of time. But when you're talking about converting the most popular programming language on the planet, I think that would justify a lot of effort. While it is possible to use Nim like a Cython replacement to speed up certain slow parts of a Python program, and that may be the best way forward for an individual or small team, it has limitations similar to multiprocessing: you still may have to take a lot of Python data, convert it to native Nim structures, do something with them, then convert it all back to Python, and that won't be fast. I haven't tried it, so can't say whether it's worthwhile or not. One way to see how reasonable it is to do a real Python to Nim transpiler would be to start manually converting small Python programs, like at [https://rosettacode.org/wiki/Category:Python](https://rosettacode.org/wiki/Category:Python). The advantage of starting with real programs vs a grammar is that you'd be putting effort into converting things that people use all the time rather than beating your head against a wall trying to figure out how to convert some esoteric Python OO construction that maybe only a few people actually use. If a transpiler converted 75% of the code and left the other 25% of "weird" stuff to be done by hand, that would be huge. Someone there wrote the weird Python, someone there can convert it by hand. This would also be a great tool for Python programmers to learn Nim. If a person is already familiar with their Python program, seeing a decent Nim translation would be a great way for them to map the Python concepts they already understand, to Nim. And it isn't just about performance. Have you ever tried to do a static build of a Python program that uses C extensions (not the ones that come with Python), Cython modules, and needs an updated OpenSSL, for 3 platforms? Try it. It ain't fun. Or if you have a C library you'd like to use, like the new Blake3 hash function, that doesn't have a Python binding yet. Ever seen those C-to-Python modules, with refcounts all over the place? What if you forget a few and your program starts crashing regularly. How do you debug that? Again, not fun. Those are some of my reasons for looking at Nim. I looked at Go, and while it's much more mature and stable because of Google, it doesn't do much for me. Rust is way too fussy (Nim is too sometimes) and Rust is starting to look like APL. Compared to C++, which is a mess I never even wanted to learn, Rust might be wonderful, but they're too much "kitchen, bathroom, and laundry sink" for me.
Re: Experimenting with a FreeRTOS OS Port
> So it's a pain to rebase... Anyone know what's required to get the base PR > merged? Where is your PR?
Re: Understanding Nim compiler
> Is there a wish to build a compiler's internals documentation or is it a > knowledge that lives in the compiler hackers' minds? Well we have the internal documentation and we're always accepting PRs but I generally assume that compiler contributors have read some book about compiler development, there are good ones available like [https://people.inf.ethz.ch/wirth/CompilerConstruction/index.html](https://people.inf.ethz.ch/wirth/CompilerConstruction/index.html) > What are the important internal data structures? It's ASTs all the way down and the compiler's AST is the same AST that is exposed in the macro system. :-)
Re: Experimenting with a FreeRTOS OS Port
I was able to auto-merge your branch with upstream devel with no immediate problems, but only tested a hello world, didn't test all the POSIX stuff. The question of how to separate the different options is fraught... It would be great to be able to specify --os:esp-idf and have it work as smoothly as, e.g. --os:nintendoswitch Or are you thinking that since there's so many variants, better to specify each separately? How do you specify --lwip when compiling? I've got lots more bikeshedding, but I'll put it on your github
Re: Experimenting with a FreeRTOS OS Port
That's a good list! But the LwIP and FreeRTOS <=> POSIX brigde requires support in the Nim standard library to make using Nim sockets/selects/etc libraries possible. The port on my branch with networking supports the LwIP, and some of the VFS via the normal POSIX API's (uart, select, etc). The core part to make Nim usable with ESP-IDF is: * the FreeRTOS <=> POSIX bridges * Nim compiler constants for 'os:freertos' * LwIP wrappers * Modifications to the standard libraries. Once those are in place the VFS, FreeRTOS tasks, etc can be handled readily as libraries, or small PR's. Unfortunately the standard libraries include number of when ... blocks to avoid functions that don't make sense on FreeRTOS slimmed POSIX. Since you need to space the rest of the proc with the when block, any upstream changes in those files cause conflicts. So it's a pain to rebase... Anyone know what's required to get the base PR merged? Then it'd be much easier to start working on testing out the full system and expanding out the features.
Re: Experimenting with a FreeRTOS OS Port
The other option is that I may re-apply my changes to 1.2.4 branch and just develop from there as the "stable" version.
Re: Python transpiler
I think part of the "sales pitch" for Nim out in the world, for good or ill, is "sorta like Python but compiled with static types". It isn't so strange that people show up asking how to automate "sorta like" into "more so". ;-) The vast majority of Python code that I have seen uses precious little of Python's very dynamic nature. So, doing a half-baked py2nim with a **_slew_** of caveats might have some value. It's not like `c2nim` is perfect, either. The questions are more A) if the caveats are beyond the grasp of the typical person porting such code and B) if the rather open-ended nature (ever changing Py3) would leave it half-done forever. @juancarlospaco may be right that a group effort here could yield the best results. It isn't my question to answer, but as to "why?", it is likely "easier good performance". Python unassisted by C/Cython/Pythran/etc. or Pypy is one of the slowest programming languages around. Combined with piles and piles of Python code out there doing xyz useful thing that could be safer/faster/etc. with Nim. Nim just invoking/calling that code is one way to access functionality. If it was too slow to begin with, it won't help. People may have misplaced expectations about how little they have to think in order to improve performance, of course.
Re: Python transpiler
I created the repo because people were asking for **an Example** on how to do that, it is an example it was not meant to be complete. ...but if people are interested on making a real working one, we can team-up or whatever. :)
Re: Python transpiler
If you mean compiling _from_ python, there's [https://github.com/metacraft-labs/py2nim_deprecated](https://github.com/metacraft-labs/py2nim_deprecated)/ which has gained "deprecated" in its name since I last looked at it. It's not very complete though. If you compiling _to_ python, there's [https://github.com/juancarlospaco/nim-new-backend](https://github.com/juancarlospaco/nim-new-backend) which is also incomplete. The biggest question is _why_. If you want to use Python from nim, or the other way around, then nimpy and nimporter might be better choices than trying to fit everything into the same language runtime.
Re: Understanding Nim compiler
I wrote how to debug Nim program and Nim compiler with GDB long time ago. [https://internet-of-tomohiro.netlify.app/nim/gdb.en.html](https://internet-of-tomohiro.netlify.app/nim/gdb.en.html) It might contain outdated content.
Re: Python transpiler
@ShalokShalom can perhaps clarify what he meant, but I took him to be asking for a `c2nim`-like program `python2nim` to help facilitate porting pure Python code to pure Nim code (as opposed to using already written Nim code from within Python or already written Python code from within Nim). Some runtime libraries to bridge API differences might also help along those lines. Some quick web searchers turn up a few partial moves along this direction.
Re: tests in the same files as the code
In my opinion, it really depends on the the size of your file and the complexity of your procedure. For me, if it's one simple procedure, I guess having it in the file works, and I'd do it the way @shirleyquirk does. But if it's multiple procedures or a complex one, I'd probably separate it out. For projects, I'd have all the tests in a separate directory. I personally follow the nimble directory layout: * src/ * bin/ * tests/ Hope this gives you some insight.
Re: Understanding Nim compiler
Thanks. For this particular example, I've started looking at how static generic expressions are handled by procs and templates into `semtypes.nim`. But the question wanted to be more general. How a new comer can understand how the Nim compiler works? What tools/options are available to have a global understanding of the compilation process? Are there ways to see the results of the different compilation phases? I've listed 2 undocumented options to see the scan and parser phases, but after the AST is built there are many phases when the AST is reworked before code generation. What are the important internal data structures? Is there a wish to build a compiler's internals documentation or is it a knowledge that lives in the compiler hackers' minds?
Re: Question about move semantics for objects and seqs
> Case 2. Overloading for optimization, to avoid copy/reuse buffers we can have > 2 versions of a procedure, one with sink and one without, for example That's not supported by Nim and so far I haven't seen convincing examples. If you want to take over ownership, you always want to leaving you with the sink T version only.
Re: Question about move semantics for objects and seqs
Thank you very much @mratsim! :-D This has cleared up a lot of my questions. It's always fascination how smart compilers can be about reading code and optimizing it. A huge amount of work has obviously been put into them. One more question about sink inference: Is it infered once per proc or is it done once for every call to the proc? ie. it creates separate signatures for the proc depending on if the argument is sink-able or not. For example: proc customAdd(s: seq[SomeType], newElement: SomeType) = s.add newElement Run Here `newElement` could be sink-able sometimes, but sometimes not. Does the compiler see that it _could_ be sink-able, and adds the `sink` parameter? Or does it create a new overloaded proc with the `sink` parameter as you showed in your example?
Re: [offtopic]2 cross-platform GUI library
Dear imgui not have any love?
Re: [offtopic]2 cross-platform GUI library
It's nice, but I think it's not the best choice if you actually want to make a user-oriented desktop application. It's very useful for developers (debugging, creating game editors, etc)
Re: Python transpiler
Have you had a look at nimpy? Lets you compile Nim code to a python module, and lets you call python modules from Nim. That's about as good as anything is going to get, prove me wrong! I mean, Haxe transpiles to python, but it's about as readable as Nim-compiled c.
Re: Python transpiler
Do you mean from Nim to Python or from Python to Nim? Which use cases do you have in mind?
Re: tests in the same files as the code
Good point. This comes up from time to time in the forum. It would be really nice if there was some documentation on `testament` (or if there's some, a link to it). Also, I vaguely remember that `testament` also has disadvantages over `unittest`, so the tradeoff when to use one or the other should be explained. So far I've used only `unittest` and like it, but I've written only relatively small programs in Nim. If the cases where `testament` is considered the better option are known, it would help to put this information in the paragraph in the `unittest` documentation. So readers wouldn't be confused about missing out on `testament` if `unittest` is fine for their purposes.
Python transpiler
How is the stance of the community on this? Is it feasible? Planned? Wished? Are there any obstacles? Besides the obvious development effort.
Re: tests in the same files as the code
I think for small simple modules it's ok to have the tests (with doAssert, not assert) in the same file. Apart from that, I prefer a test module using the [unittest](https://nim-lang.org/docs/unittest.html) module for each module to test. In my experience, putting the tests in the same file as the tested code makes maintaining the tests awkward if the tests grow. Consequently, tests aren't as thorough as if put into an extra file. Having the tests in an extra file helps me focus more on writing good tests with good coverage.
Re: "Nim for Python Programmers" wiki page
You can use macros to provide class-like capabilities to object variants (i.e. constructing the object variant from multiple modules) See: [https://github.com/mratsim/trace-of-radiance/blob/99f7d85d/trace_of_radiance/support/emulate_classes_with_ADTs.nim](https://github.com/mratsim/trace-of-radiance/blob/99f7d85d/trace_of_radiance/support/emulate_classes_with_ADTs.nim) The only limitation is that in other languages like C++, you can add a new subtype even when the library is compiled as a DLL while with macros you need compilation from source. But current Nim methods have the same limitation.
Re: Question about move semantics for objects and seqs
### Case 1 `var y = x` would be elided and all further reference of y should use x instead in the generated code. For stack objects this shouldn't really matter because GCC/LLVM should optimize those variable renaming anyway. This is because during code generation, they transform the code in SSA form (Static Single Assignment) meaning no variables are mutable and all changes in values for example `a += 1` are transformed into variable assignment `a1 = a0 + 1` In `consume` the `x` is passed by hidden reference can be mutated, this is useful if you need destructive updates that would be costly if you did a copy first, for example sorting in-place. ### Case 2 It's option 2 > Arc is smart and knows how different fields should be treated. It knows that > it should just copy the pointer of the seq and not the seq itself to avoid > unneccecary copies. ### Extra Arc does sink inference but there are still 2 cases where you might want to use explicit sink. Case 1. To only allow move semantics. Some types can only be moved, in particular all types that represent a resource (memory like GPU memory, open file, socket connection, database handle, lock, message queue). You want to ensure that there is a single owner to those resources throughout the resource lifetime and so you disallow copy but allow moves. Case 2. Overloading for optimization, to avoid copy/reuse buffers we can have 2 versions of a procedure, one with sink and one without, for example proc add1(a: seq[int]): seq[int] = result.newSeq(a.len) # Extra allocation for i in 0 ..< a.len: result[i] = a[i] + 1 proc add1(a: sink seq[int]): seq[int] = # No allocation, we reuse the buffer for i in 0 ..< a.len: a[i] += 1 `=sink`(result, a) Run
Re: tests in the same files as the code
testament lets you separate tests from code but yes, less well documented. [https://nim-lang.org/docs/contributing.html](https://nim-lang.org/docs/contributing.html) has some info
Re: tests in the same files as the code
At the topo of the unittest module documentation it says: > Note: Instead of unittest.nim, please consider to use the testament tool > which offers process isolation for your tests. Also when isMainModule: > doAssert conditionHere is usually a much simpler solution for testing > purposes. Does that mean that unittest is obsolete and should not be used? The docs do not provide a link to the testament module so the advice is not very actionable :-)
Re: tests in the same files as the code
For simple stuff, especially for unit testing small modules when isMainModule: doAssert #condition# Run is fine or for a larger testsuite you can use [https://nim-lang.org/docs/unittest.html](https://nim-lang.org/docs/unittest.html) which also runs in the same file
