Platob commented on PR #35299:
URL: https://github.com/apache/arrow/pull/35299#issuecomment-1537094036
> Also, it seems there is already ArrowBuffer.BitmapBuilder which is a
buffer builder that is optimized for bits (and #35342 adds an efficient
`AppendRange`). Perhaps you could use that and leave your buffer builder
focused on non-bit types? Either way, it might be good to align on one approach
to maintain.
>
> CC @asmirnov82
Indeed, but for nested structures it needs specify the builders to check how
to merge correctly 2 arrow buffers knowing if they are bit based or byte based
the main reason would be merge 2 array data by just checking their datatypes
since their structures should be the same
```csharp
public virtual IArrayBuilder AppendValues(ArrayData data)
{
// TODO: Make better / recursive fields data type check
if (data.DataType.TypeId != DataType.TypeId)
throw new ArgumentException($"Cannot append data type
{data.DataType} in builder with data type {DataType}");
NullCount += data.NullCount;
Length += data.Length;
Reserve(data.Length);
for (int i = 0; i < Buffers.Length; i++)
{
IValueBufferBuilder current = Buffers[i];
ArrowBuffer other = data.Buffers[i];
if (current.ValueBitSize % 8 == 0)
{
// Full byte encoded
current.AppendBytes(other.Span);
}
else
{
// Safe copy Bytes and remaining bits
int end = (data.Length * current.ValueBitSize) / 8;
current.AppendBytes(other.Span.Slice(0, end));
Span<bool> bits =
BitUtility.BytesToBits(other.Span.Slice(end)).Slice(0, data.Length - end * 8);
current.AppendBits(bits);
}
}
if (Children != null && data.Children != null)
{
for (int i = 0; i < Children.Length; i++)
{
Children[i].AppendValues(data.Children[i]);
}
}
if (Dictionary != null && data.Dictionary != null)
{
Dictionary.AppendValues(data.Dictionary);
}
return this;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
