Is there any code in javascript to access the buffer? Thank you for tour help
On Sun, May 3, 2020, 02:13 Sutou Kouhei <[email protected]> wrote: > Hi, > > >> Is it necessary to create a buffer to send data via socket, is, for > >> example I create the schemas, do I have to create a buffer to store the > >> data and then send he that data?, > > If your C program and JavaScript program run in the same > process, you don't need to do it. You can just pass memory > address. > > If your C program and JavaScript program run in the > difference processes (I think so), you need to do it. > > >> Also is this method more effective compared to writing data to a arrow > >> file and reading from that arrow file in the javascript program? > > If you use memory file system such as tmpfs in Linux for > writing and reading the Apache Arrow file, it may be faster > than sending data via socket. > > >> Finally, what are the option that are available for passing data from > one > >> program language to another using arrow? > > 1. Sending data via socket > (It works with different processes and different hosts) > 2. Writing data to a file and passing the file > (It works with different processes) > 3. Sending memory address (the same process) > (It works with the same process) > > > And also, instead of sending the pointer, can you send the buffer > directly? > > The example I introduced sends the buffer data not the > pointer of the buffer data. > > > Thanks, > -- > kou > > In <cackq_v0t77ciwbbmkdfneakeb+kbyjnjopawbdj_p3lejqz...@mail.gmail.com> > "Re: Sending arrow via socket" on Sat, 2 May 2020 07:46:55 +0200, > swizz one <[email protected]> wrote: > > > And also, instead of sending the pointer, can you send the buffer > directly? > > > > El sáb., 2 may. 2020 a las 7:07, swizz one (<[email protected]>) > escribió: > > > >> Thank you very much, Your answer has been really helpful. Can I ask a > >> question? > >> Is it necessary to create a buffer to send data via socket, is, for > >> example I create the schemas, do I have to create a buffer to store the > >> data and then send he that data?, > >> Also is this method more effective compared to writing data to a arrow > >> file and reading from that arrow file in the javascript program? > >> Finally, what are the option that are available for passing data from > one > >> program language to another using arrow? > >> > >> Sorry, for the questions, I am a bit new to programming. > >> > >> El vie., 1 may. 2020 a las 23:30, Sutou Kouhei (<[email protected]>) > >> escribió: > >> > >>> Hi, > >>> > >>> Do you want to use Apache Arrow C GLib instead of Apache > >>> Arrow C++, right? > >>> > >>> We don't provide a memory pool in Apache Arrow C GLib API. > >>> > >>> You can create a resizable buffer and output serialize data > >>> to it: > >>> > >>> --- > >>> GError *error = NULL; > >>> GArrowResizableBuffer *buffer = garrow_resizable_buffer_new(1024, > &error); > >>> if (!buffer) { > >>> g_print("error: %s\n", error->message); > >>> g_error_free(error); > >>> return; > >>> } > >>> > >>> GArrowBufferOutputStream *output = > >>> garrow_buffer_output_stream_new(buffer); > >>> > >>> GArrowRecordBatchStreamWriter *writer = > >>> garrow_record_batch_stream_writer_new(GARROW_OUTPUT_STREAM(output), > >>> schema, /* You need to create > >>> this */ > >>> &error); > >>> if (!writer) { > >>> g_print("error: %s\n", error->message); > >>> g_error_free(error); > >>> g_object_unref(output); > >>> g_object_unref(buffer); > >>> return; > >>> } > >>> > >>> if (!garrow_record_batch_writer_write_record_batch( > >>> GARROW_RECORD_BATCH_WRITER(writer), > >>> record_batch, /* You need to create this */ > >>> &error)) { > >>> g_print("error: %s\n", error->message); > >>> g_error_free(error); > >>> g_object_unref(writer); > >>> g_object_unref(output); > >>> g_object_unref(buffer); > >>> return; > >>> } > >>> > >>> if (!garrow_record_batch_writer_close( > >>> GARROW_RECORD_BATCH_WRITER(writer), > >>> &error)) { > >>> g_print("error: %s\n", error->message); > >>> g_error_free(error); > >>> g_object_unref(writer); > >>> g_object_unref(output); > >>> g_object_unref(buffer); > >>> return; > >>> } > >>> > >>> GBytes *data = garrow_buffer_get_data(GARROW_BUFFER(buffer)); > >>> gsize data_size; > >>> gconstpointer data_raw = g_bytes_get_data(data, &data_size); > >>> write(websocket_fd, data_raw, data_size); > >>> g_bytes_unref(data); > >>> > >>> g_object_unref(writer); > >>> g_object_unref(output); > >>> g_object_unref(buffer); > >>> --- > >>> > >>> > >>> Thanks, > >>> -- > >>> kou > >>> > >>> In <CACKQ_V3_yeuNn5TzPiF1RS=6wmwjl5jr4vgx39gpq3fvmge...@mail.gmail.com > > > >>> "Sending arrow via socket" on Fri, 1 May 2020 18:12:34 +0200, > >>> swizz one <[email protected]> wrote: > >>> > >>> > Please, I am currently working on a project that require sending data > >>> from > >>> > an c program to perpespective(javascript) via socket from c. Since > >>> > perpespective works with arrow, it was a perfect choice. Is it > possible > >>> to > >>> > send table at x interval from c to javascript via websocket without > >>> > creating a an arrow binary format file? > >>> > How to you create a memory pool primotive with c, without having to > >>> size to > >>> > have a fixed sized on the array like the memory pool in c++. > >>> > > >>> > Your's Faithfully, > >>> > Thank you > >>> > >> >
