Rhombus (a REBOL/Red/Ren parser)

[icedquinn]

Have been working on this for a week or two. Decided to try doing a 
lexer/parser combination instead of the PEGs I've been writing. Not sure if 
that was worth it in the end but it does work. Still need to round off the 
edges (like debug trace statements.)

 (gitea link soon, maybe)

Rebol is a wild format to work with but its both a scripting language and 
flexible data format. It's not exactly Lisp but it does work a bit like one.

For example you can use it as a data format: 


mailbox neat [
  recv-server: imap://some.place.out.there
  send-server: smtp://some.place.out.there
  filters: [
bogofilter [
   server: 127.0.0.1 ; IP4 addresses are a native datatype
   ...
]
  ]
]

version: 1.0


Run

Or you can throw a tree interpreter at the blocks if you want it to be a 
scripting language.

It's a little weird to wrangle and is from an old mostly dead language. But its 
a fun one. 

TLS protocol negociation (TLS-ALPN)

[icedquinn]

Worked out the flow diagram today.

  * Client runs `SSL_CTX_set_alpn_protos` to set the list of protocols its 
willing to use while setting up the handshake.
  * Server runs `SSL_CTX_set_alpn_select_cb` to set a callback which gets the 
list of protocols the server and client are willing to use then returns the 
name of the one to use.
  * Client uses `SSL_get0_alpn_selected` to figure out which one the server 
selected.



It looks like this can be wrapped in to two stdlib functions; one that sets the 
list from a seq[string] and then expose the selected protocol as an 
option[string]. There is a standard way of selection which is offered by 
OpenSSL so it isn't strictly necessary to expose that part.

This would look something like:


proc set_alpn_protocols(context: var SslContext; protocols: 
openarray[string])
proc alpn_protocols(context: SslContext): Option[string]


Run

Question: How to limit concurrent async futures?

[rockcavera]

I once made some pretty naive code to limit the amount of open TCP connections 
using async. The code was very similar to the code below: 


import std/[asyncdispatch, random, strformat]

const limit = 10

var counter = 0

proc testProc(n: int) {.async.} =
  let rwait = rand(1000) + 500
  
  echo fmt"Running {n} async call..."
  
  await sleepAsync(rwait)
  
  echo fmt"{n} async call executed."
  
  dec(counter)

proc main() =
  randomize()
  
  let performNCalls = rand(30) + 10
  
  var x = 1
  
  while true:
if x <= performNCalls and counter < limit:
  inc(counter)
  
  asyncCheck testProc(x)
  
  inc(x)
elif not hasPendingOperations():
  break

if hasPendingOperations():
  poll(50)

main()


Run

I don't know if this is the best approach... try it and see if it solves your 
case.

Macros: why and/or when to use them?

[Sixte]

> But I know that I don't have any problem with modularity in other languages.

You want module extension. In fact, you have a module **A** with type X and a 
module **B** with a type Y and then you extend **A** with **B** that can be 
read as **A[B]** so you "plugged in" the B into the A. Now you can have access 
to both A[B].X and A[B].Y with the same object. You can find the feature 
everywhere, especially in languages that inherit from Java. This is a 
convenient feature and typically some runtime (virtual tables) is involved. The 
programmer is satisfied because of the convenience and thinks that it is 
"modular". However, If you want to do it efficiently, you have to recompile 
both A und B if you change one of them. So there is not much modularity here.

But I think about **separate compilation**. We can automatize this (see post 
above) or we can do it with module descriptions like C++'s upcoming module 
feature. (They are a bit late).

So our focus is a bit different. 

Question: How to limit concurrent async futures?

[kobi]

OK, I will attempt that.

Macros: why and/or when to use them?

[kcvinu]

I know that the people behind Nim doesn't like when I said, "Hey there is a 
problem in modularity". So I stated it in good words. If I say, it's my 
problem, they will say, "Okay then learn how to solve it." But I know that I 
don't have any problem in modularity in other languages. I know they are 
working hard to improve Nim but there are still some problems. And for me, 
there is a language barrier to express my thoughts in this forum. So please do 
not get me wrong. 

Macros: why and/or when to use them?

[Araq]

> But in practice I think it's better to avoid using it and keep things simple.

But what if in order to "keep things simple" you need a macro...

Macros: why and/or when to use them?

[SolitudeSF]

cool thought

Macros: why and/or when to use them?

[Sixte]

There was a typo. UXT and UIT are "constructor helper types". The UIT could 
simply be a value with a parametric type. If you pass it a number, an int, an 
IT with an int will be constructed. With a string, IT will contain a string. 
The helper is not identical with the target IT type. But it could, it depends...

Macros: why and/or when to use them?

[alexeypetrushin]

Macros are nice feature, and it's cool that Nin has it. It's worth to learn and 
play with it, and know its potential. But it's better to avoid using it in 
practice unless you absolutely must have.

It has huge toll, as it's complicated and you need to learn almost another 
language and know AST representation and manipulation. Like, you wrote some 
project with macros half a year ago, and now you need to change something, but 
you forget how it works and also foret macros, may not be easy, better to keep 
things simple and avoid it :).

Macros: why and/or when to use them?

[Araq]

Awesome reply, thanks! Excuse me if more careful reading of your post would 
also answer these questions:

  * What is the `M` in `h(UXT,IXT) → M[h(UXT,IXT)]`? How come `h` "arrows to" 
both `M[h(UXT,IXT)]` and `((XT,_),(XT,IT))`? A syntactic distinction between 
"is of type" and "this is its implementation" might help here, but I probably 
misread it.
  * What is `IXT`? You only defined `IT` and `UIT` afaict.

Question: How to limit concurrent async futures?

[dom96]

For downloading websites I would start with a list of 100 `AsyncHttpClient`s. 
You'll also need to track when the http client is busy, but you can do that 
easily with a future and have the ability to `await` until the http client is 
no longer busy (at which point you can request the next URL). This kind of 
thing should be available as a library, so have a look if anything exists 
already, if not it would be awesome if you could create it :)

Happy to help further if my explanation isn't clear.

Question: How to limit concurrent async futures?

[Clonk]

Look into [std/asyncdispatch](https://nim-lang.org/docs/asyncdispatch.html) and 
[std/asynchttpserver](https://nim-lang.org/docs/asynchttpserver.html).

Notable, the proc 
[shouldAcceptRequest](https://nim-lang.org/docs/asynchttpserver.html#shouldAcceptRequest%2CAsyncHttpServer%2Cint)
 should be close to what you want

Question: How to limit concurrent async futures?

[kobi]

I can track the active connections, seeing how many have finished, then have a 
while active >= limit: sleep 10

I wonder if there is a more structured approach, or perhaps a template already 
exists for that.

Macros: why and/or when to use them?

[Sixte]

>Visibility is not the problem, it's a phase ordering problem.

  * visibility problem <=> phase ordering problem



Well, 100% separate compilation can be achieved if we split the program into a 
general part and a DLL. But we don‘t need to go so far. A C compiler could in 
principle do the split itself, if we could single out a specific type, the 
intrinsic type of the DLL (There might be a couple of them though). It is sad 
that C compilers in general do not support the feature. It is sad because we 
already have a split: „.h“ and „.c“ and this could be extended further.

Let‘s now address it categorically.

If a module/program M1 imports a module M2, than the module M2 has to allow at 
least some access through an exchange type, lets say type XT, at the same time, 
it provides an intrinsic type IT for itself.

We now have to construct the module M2. M2 might do this itself, but it might 
even happen in M1. We do not construct from the pair (XT,IT) directly. We do it 
by „unbound“ or unitialized types UXT and UIT and our constructor is:

h(UXT,IXT) → M[h(UXT,IXT)] where h is a helper function: h(UXT,IXT) → 
((XT,_),(XT,IT))

provided by M2. Our representation type of M2 for M1 is :

M2:((XT,_),(XT,IT))

We‘ll now have two representations of M2: A representation of M2 for M1 as 
given above and a representation of M2 for M2 itself as well:

M2:((XT,IT),(XT,IT))

In M1, the program / the programmer can only see the first of the two pairs, 
that is (XT,_). Any component of XT, let‘s say an XT.t1, can be accessed by M1. 
But an IT.t1 would not be possible. However, the entire representation type is 
still available as an „opaque“ type. The programmer „in M2“ can see both XT and 
IT, because (XT,IT) is available here.

The visibility rules for functions are :

Functions that do only rely on XT can be defined/implemented both in M1 and M2, 
but M2 cannot see functions defined in M1.

Functions relying on IT can only be defined in M2. There is a split in two 
fractions:

Functions that rely on components of IT, like the mentioned IT.t1, can only be 
called from M2.

Functions that rely on IT can be called from M1, as long as no components of IT 
are involved.

What about the size of an IT? The memory layout is part of XT, therefore 
„public“, and if possible, pushed down to the target. The goal here is to keep 
XT as „stable“ as possible.

We now define : a = ((XT,_),(XT,IT)) and b = ((XT,IT),(XT,IT))

and two conversion functions:

> ((XT,IT),(XT,IT)) → ((XT,_),(XT,IT)) („remove“)
> 
> ((XT,_),(XT,IT)) → ((XT,IT),(XT,IT)) („reinstall“)

so we can move the two representations of M2 back and forth.

We now define

„return“ : a → M[a] (see the constructor above) „bind“ : M[a] → (a → M[b]) → 
M[b]

(This is nice for the categorical part and could be useful if we want to go 
further)

...with the help of the two converter functions. (We could even write the 
conversions with functors. „remove“ and „reinstall“ can be used for an fmap).

What about module independency now?

If we change M1 leaving UXT and UIT (resp. XT and IT) untouched, we‘ll not 
recompile M2 because we have already constructed it. If we change M2 leaving XT 
untouched or „stable“, we‘ll not recompile M1, even if the intrinsic type IT 
might change as well.

That‘s pretty much it….

Question: How to limit concurrent async futures?

[kobi]

Hi, How would you go about limiting the amount of concurrent async futures, for 
example, to download a website I don't want to open 100s of connections at the 
same time.

What's the best design for that? I guess, similar to a thread pool.

Advent of Nim 2021

[foldl]

Oh, Jesus. Another year is ending.

What is "Option[system.bool]"?

[xigoi]

Yes, it's similar. The difference is that you can't implicitly convert `T` to 
`Option[T]`.

What is "Option[system.bool]"?

[SoupCookie]

So, almost like an or operator in TypeScript? Like where a function can return 
either a bool or null/undefined etc.

What is "Option[system.bool]"?

[SoupCookie]

Thank you! You are 100% correct, I was trying to use Telebot, and I didn't 
think to even look at the examples!

"you will sometimes see types in error messages in their fully-qualified 
module.type form. so system.int or system.bool are exactly the standard int, 
bool types."

Could you please explain further?

What is "Option[system.bool]"?

[shirleyquirk]

you will sometimes see types in error messages in their fully-qualified 
`module.type` form. so `system.int` or `system.bool` are exactly the standard 
int, bool types.

it looks like you're trying to use `telebot`? if my guess is correct, there's 
an example of how to set chat permissions in the repo 
[here](https://github.com/ba0f3/telebot.nim/blob/master/examples/set_chat_permissions.nim)


import ../telebot, asyncdispatch, logging, options, sam
from strutils import strip

import ../telebot/utils

var L = newConsoleLogger(fmtStr="$levelname, [$time] ")
addHandler(L)

const API_KEY = slurp("secret.key").strip()

proc commandHandler(b: Telebot, c: Command) {.async.} =
  let perms = ChatPermissions(
canSendMessages: some(true),
canSendMediaMessages: some(true),
canSendOtherMessages: some(true),
canAddWebPagePreviews: some(true))
  
  var json  = ""
  marshal(perms, json)
  echo json
  discard await restrictChatMember(b, $c.message.chat.id, 50535480, perms)
  
  discard await getChatMember(b, $c.message.chat.id, 50535480)


when isMainModule:
  let bot = newTeleBot(API_KEY)
  bot.onCommand("perms", commandHandler)
  bot.poll(timeout=300)


Run