polestar1988 commented on issue #37976:
URL: https://github.com/apache/arrow/issues/37976#issuecomment-1744691826
@zeroshade Thanks so much for your suggestions I implemented and it worked ,
the only thing is calculating the size of data which I have implemented this
function:
// Function to calculate size based on data type
```
func calculateSize(dataType arrow.DataType, numElements int) int {
// Initialize variables for element size and null bitmap size
elementSize := 0
nullBitmapSize := 0
// Check if the data type is fixed width
if fw, ok := dataType.(arrow.FixedWidthDataType); ok {
// Get the bytes per element for any fixed width data type
elementSize = fw.Bytes()
// Get the size of the null bitmap in bytes
nullBitmapSize = bitutil.CeilByte(numElements)
} else {
switch dataType.(type) {
case *arrow.StringType:
// as string has not fixed size we appriximated as 100
bytes
return 100
case *arrow.BinaryType:
// as binary is for storing larger data and has not
fixed size we appriximated as 1000 bytes
return 1000
}
}
// Return the total size of the column in bytes
return elementSize*numElements + nullBitmapSize
}
```
is there any way to calculate also the size of data for non fixed width data
types like string or binary?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
