[
https://issues.apache.org/jira/browse/ARROW-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445426#comment-16445426
]
Haralampos Gavriilidis commented on ARROW-2457:
-----------------------------------------------
{code:java}
int main() {
gint64 cnt = 100000000;
gint64 *n;
gboolean *valids;
n = malloc(sizeof(gint64) * cnt);
valids = malloc(sizeof(gint64) * cnt);
gint64 i;
for (i = 0; i < cnt; i++) {
n[i] = 5;
valids[i] = TRUE;
}
GArrowArray *array;
GArrowInt64ArrayBuilder *builder;
gboolean success = TRUE;
GError *error = NULL;
builder = garrow_int64_array_builder_new();
if (success) {
success = garrow_int64_array_builder_append_values(builder, n, cnt, valids,
cnt, &error);
}
if (!success) {
g_print("failed to append: %s\n", error->message);
g_error_free(error);
g_object_unref(builder);
return EXIT_FAILURE;
}
array = garrow_array_builder_finish(GARROW_ARRAY_BUILDER(builder), &error);
if (!array) {
g_print("failed to finish: %s\n", error->message);
g_error_free(error);
g_object_unref(builder);
return EXIT_FAILURE;
}
g_object_unref(builder);
g_object_unref(array);
}
{code}
> garrow_array_builder_append_values() won't work for large arrays
> ----------------------------------------------------------------
>
> Key: ARROW-2457
> URL: https://issues.apache.org/jira/browse/ARROW-2457
> Project: Apache Arrow
> Issue Type: Bug
> Components: C, C++, GLib
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Haralampos Gavriilidis
> Assignee: Kouhei Sutou
> Priority: Major
>
> I am using garrow_array_builder_append_values() to transform a native C array
> to an Arrow array, without calling arrow_array_builder_append multiple times.
> When calling garrow_array_builder_append_values() in array-builder.cpp with
> following signature:
> {code:java}
> garrow_array_builder_append_values(GArrowArrayBuilder *builder,
> const VALUE *values,
> gint64 values_length,
> const gboolean *is_valids,
> gint64 is_valids_length,
> GError **error,
> const gchar *context)
> {code}
> it will fail for large arrays. This is probably happening because the
> is_valids array is copied to the valid_bytes array (of different type), for
> which the memory is allocated on the stack, and not on the heap, like shown
> on the snippet below:
> {code:java}
> uint8_t valid_bytes[is_valids_length];
> for (gint64 i = 0; i < is_valids_length; ++i){
> valid_bytes[i] = is_valids[i];
> }
> {code}
> A way to avoid this problem would be to allocate memory for the valid_bytes
> array using malloc() or something similar. Is this behavior intended, maybe
> because no large arrays should be handed over to that function, or it is
> rather a bug?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
