I'll first finish the current development but then I will see if your
suggestion is usefull.
Ben
Op 27-11-2021 om 20:19 schreef Tomas Kalibera:
On 11/27/21 8:05 PM, Tomas Kalibera wrote:
On 11/27/21 5:24 PM, Ben Engbers wrote:
Op 27-11-2021 om 17:03 schreef Jeff Newmiller:
This is a null-terminated message protocol [1]. It has to be
processed one byte at a time.
[1] https://docs.basex.org/wiki/Server_Protocol
The message may contain embedded 0x00's. To distinguish these
embedded 0x00's (and 0xFF's) from a terminating 0x00, embedded 0x00's
and 0xFFare prefixed with a 0xFF byte. This means that when you
process one byte at a time you have to perform a check on every byte.
This results in totally unacceptable response times (My first version
of this client was based on this approach)
The only alternative solution I can think off is to use C++ to create
a socket and a function that reads from the socket. But since I have
hardly any experience with C++ programming nor using the rcpp
package....
I think you could use non-blocking read. Read what is available in
chunks (e.g. up to 1024 bytes long). And based on what is read
already, figure out whether it is all data or not. Something along the
lines as the demo below. The demo, though, is polling too much
(re-running readBin to only find out no data is available yet). It
should be possible to improve with socketSelect().
This is an extended demo with socketSelect() used to wait on the client
for some data to be available, to avoid consuming too much CPU by
polling. To be pasted into two R sessions running on the same computer.
You would have to replace the function done() with something figuring
out from the data whether it is complete or not, based on the protocol.
Best
Tomas
# the client
con2 <- socketConnection("localhost", port = 6011, open = "rb")
cat("Connected...\n")
total <- 0
done <- function(n) {
n >= 2e8
}
while(!done(total)) {
cat("Waiting for data to become ready...\n")
socketSelect(list(con2))
cat("Reading data...\n")
r <- readBin(con2, "raw", 1024)
total <- total + length(r)
cat("Read", length(r), "bytes (total ", total, ").\n")
}
close(con2)
# the server
n <- 1e8
w <- as.raw(runif(n, 0, 255))
con1 <- socketConnection(port = 6011, blocking = TRUE, server = TRUE,
open="a+b")
cat("Connected...\n")
writeBin(w, con1)
cat("Sent data the first time, sleeping...\n")
Sys.sleep(10)
cat("Sending data the second time...\n")
writeBin(w, con1)
cat("Data sent to client...\n")
close(con1)
Best
Tomas
# the client
con2 <- socketConnection("localhost", port = 6011, open = "rb")
cat("Connected...\n")
total <- 0
done <- function(n) {
n >= 1e8
}
while(!done(total)) {
r <- readBin(con2, "raw", 1024)
total <- total + length(r)
cat("Read", length(r), "bytes (total ", total, ").\n")
}
close(con2)
# the server
n <- 1e8
w <- as.raw(runif(n, 0, 255))
con1 <- socketConnection(port = 6011, blocking = TRUE, server = TRUE,
open="a+b")
cat("Connected...\n")
writeBin(w, con1)
cat("Data sent to client...\n")
close(con1)
Ben
______________________________________________
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
