Re: [racket-users] Why would writing to a pipe fail?
At Fri, 18 Jun 2021 05:29:14 -0400, George Neuner wrote: > My point was that the docs for write-bytes-avail et al specifically > mention "flush" of data, and in a way that implies (to me) that there is > expected to be something else underlying the buffer to "flush" to, e.g., > storage media, network connection, etc., ... something. But, if I > understand correctly, Racket pipes are a library construct and really > just a named block of memory. Agreed, but that named block of memory isn't the "buffer" in the sense of the port API. The intent for "buffer" in the Racket API is a thing attached to a port where data in the buffer is not visible outside the specific port. Data that's buffered on a file port isn't visible by reading from the file, for example, and data that's buffered for the write end of a pipe (if the pipe had a buffer) is not visible to the read end. I see that I misused the word "buffer" myself in my previous response. Where I wrote "grow the buffer", I should have written "grow the block or memory that holds a pipe's data", or something like that. > So it seems like "flush" is being used in the docs to mean insertion of > data into the buffer ... Into the block of memory, yes. But instead of characterizing a pipe as all buffer and no device, I'd say that pipe is unbuffered and has only a device (for lack of a better word). All data written to a pipe is delivered to the "device" and available to the read end of the pipe. > which is confusing (to me) and at odds with how > the word typically is used in discussions of I/O. FWIW, I really think this is the same notion of buffer as in the C library. A `FILE*` object potentially has a buffer, `fflush` moves data from that buffer to the underlying device, and so on. If you wrap a Unix pipe in a C-library `FILE*`, then `fflush` pushes data from the C library buffer (assuming the `FILE*` is made buffered) to the OS --- even though the OS-level device is just some memory, and that memory tends to be called as "buffer" at the OS level. > So the limit value passed to make-pipe is only a maximum size for the > data buffer, which starts [way too] small and then grows as needed up to > the maximum. Although that does make some sense - I think starting at > 16 bytes is a wee bit restrictive: many (most?) real world uses would > have to grow it very quickly. I agree that 16 bytes is too small. Given that a port object is already on the order of a dozen *words*, an initial size closer to 16 words makes sense. > Growing the buffer is expensive in any case and particularly when > existing data has to be copied. With pre-allocated buffer, the I/O call > will spend only what time is necessary to copy new data. That would > seem to be the fastest implementation possible. There is some cost to growing a buffer, but there's also a cost to allocating a too-large buffer, since that increases GC pressure. For example, one existing use of pipes is in the HTTP library, where a pipe is used to communicate incoming POST data to a decoding thread. The pipe's limit is set to 4096 to promote streaming behavior for large POST data, bu POST data to be much smaller than 4096 bytes in many uses. To get a better handle on relative costs, I tried three implementations for pipes: * the current one, where the the pipe memory always grows by a factor of two; * initializing the pipe memory to 4096 bytes instead of 16 bytes; and * starting with 16 or (more sensibly) 64 bytes for the pipe, but grow the memory based on the size of the number of bytes in a write request instead of always doubling --- at least up to a new size of 4096 bytes, after which the doubling algorithm takes back over. I tried a few benchmarks: * a program like the one Shu-Hung posted that allocates and writes data to a pipe; * a program that alternates writes and reads within a single thread; * a program that writes in one thread and reads in another thread, with and without a limit on the pipe; and * a program that untgzs the Racket v8.0 source repo (because untgz uses a pipe to deliver unzipped data from one thread to untarring in another thread). There was a measurable difference only in that first benchmark. For writes in the range 512 to 32768 bytes, the alternative adaptive implementations outperformed the current one by 30% on the small end of that range and 10% for the upper end. I used 1 million iterations, and the run time for that size range was around 100 ms. (Note that if the data written to a pipe is later read, that would more or less cut the difference in half.) The difference between the two new implementations was in the noise. Overall, although there's not much difference, the revised adaptive implementation seems like a win for a small and easily maintained change to the code, so I expect to push that change. There's likely still room for improvement. Anyone who is interested might start at "racket/src/io/port/pipe.rkt".
Re: [racket-users] Could Racket be used as a "frontend" for a non-Racket language?
Short answer: Yes...ish. I wrote a few code generation programs. It can happen, but the Racket program would be dependent on the target language's toolchain and standard library. Before we get to that, note that not all languages cleanly translate into one another (Some scripting languages actually have features that introduce forward-compatibility problems, see JS and with). The standard library issues become more apparent when you realize that Racket programs have defined value types where other languages don't bother (e.g. complex numbers and path types). If your goal is to generate a few well-defined programs its fine, but if you want a completely correct model for translations with a DSL as a front-end, then that can be a pretty large undertaking. And that's not saying anything about your performance expectations, since Racket's reader/expander probably won't be able to compete with a native compiler. On 6/22/21 10:50 AM, D. Ben Knoble wrote: > No idea if this is what you're looking for, but I have a feeling it wouldn't > be terribly difficult to design a #lang where modules were programs that, > when run, output C# code (kind of like how scribble & pollen can output > HTML). Then you could have a language (w/ or w/o) macros that gets run down > to C# and let the C# compiler and project toolchain take care of the rest? > Essentially it's only a syntax—the semantics is "turn it into this C# code." > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > [https://groups.google.com/d/msgid/racket-users/a70bff98-e143-4c85-bcc9-2d919927764dn%40googlegroups.com](https://groups.google.com/d/msgid/racket-users/a70bff98-e143-4c85-bcc9-2d919927764dn%40googlegroups.com?utm_medium=email&utm_source=footer). -- ~slg -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/81580c35-61e2-cce8-c297-a3e4f61cfca6%40sagegerard.com.
Re: [racket-users] Could Racket be used as a "frontend" for a non-Racket language?
No idea if this is what you're looking for, but I have a feeling it wouldn't be terribly difficult to design a #lang where modules were programs that, when run, output C# code (kind of like how scribble & pollen can output HTML). Then you could have a language (w/ or w/o) macros that gets run down to C# and let the C# compiler and project toolchain take care of the rest? Essentially it's only a syntax—the semantics is "turn it into this C# code." -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/a70bff98-e143-4c85-bcc9-2d919927764dn%40googlegroups.com.
Re: [racket-users] An Apology
Thank you, Siddhartha Kasivajhula, for expressing this. I am an "ordinary" Racket user - I don't hack on Racket itself, I don't even create DSLs (yet). But I care about Racket a lot, I tell people about it, I write software in it, and I even contributed a small feature to a racket package. I was glad to see the Apology email on the list. I think this is a good first step. And I very much understand this is a difficult situation. But I agree there is a crucial next element that is needed: a clear strategy on handling these issues in the future, a commitment to addressing the problem in addition accepting it exists. There are two links I would like to share: The Racket Friendly Environment Policy: https://racket-lang.org/friendly.html It says: > Unacceptable behavior will not be tolerated. (Where unacceptable behavior is defined it a page linked from the Policy.) And a blog post I saw the other day on Software Freedom Conservancy's blog: https://sfconservancy.org/blog/2021/apr/20/how-to-apologize/ I think it has many constructive points. Best, -- Yury Bulka https://mamot.fr/@setthemfree #NotOnFacebook Siddhartha Kasivajhula writes: > I just want to add my words on behalf of the silent majority. I think it is > important to look for ways to empower people like Matthew Butterick who have > contributed significantly to the community and done much to raise the > profile > of the Racket language. While his code contributions are evident, we should > also bear in mind that there is much of value that cannot be parsed by the > reader, and is not easy to evaluate. It takes a special sagacity to do the > kind > of community building that he has done, and I know I've been grateful to > read > some of his - relatively infrequent now - posts on this list in the past. > It's > heartening to see apologies from those involved. We all make mistakes, and > apologies are not easy. I hope that those familiar will also, based on an > honest appraisal of the facts and sincere reflection, take necessary steps > to > ensure that this kind of thing doesn't happen in the future. I hope and > believe > we can do more to help people feel welcome and encouraged, and in > particular, I > am sure I speak for many when I say that I hope that we can make it right > with > Matthew. > > > On Fri, Jun 18, 2021 at 7:07 PM Sam Tobin-Hochstadt > wrote: > >> We were saddened to read [Matthew Butterick's recent post]. Matthias >> has written his own [apology] in response. >> >> For our part, some of us were present for the interaction between >> Matthew Butterick and Matthias Felleisen and some of us learned about >> it later. We did not intervene in the way that we should have, and we >> are sorry. >> >> If you wish to offer your thoughts to the people named below, the >> email address feedb...@racket-lang.org will reach us directly. >> >> Jay McCarthy >> John Clements >> Matthew Flatt >> Robby Findler >> Sam Tobin-Hochstadt >> >> [Matthew Butterick's recent post]: >> >> https://beautifulracket.com/appendix/why-i-no-longer-contribute-to-racket.html >> [his own apology]: https://felleisen.org/matthias/Thoughts/Apology.html >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Racket Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to racket-users+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/racket-users/CAK%3DHD%2BZrg1dw%3Ds1HnO17hBU55x%2B%3D9XkEonLxNjwyi-gu1jrfcw%40mail.gmail.com >> . >> -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/87eecuwdtv.fsf%40privacyrequired.com.
