vibhatha commented on PR #38396:
URL: https://github.com/apache/arrow/pull/38396#issuecomment-1777097195
@lidavidm I did a few tests, by generating randomized data as follows;
```java
private static void writeRandomIntData(IntVector vector, int valueCount) {
Random random = new Random();
for (int i = 0; i < valueCount; i++) {
vector.setSafe(i, random.nextInt());
}
vector.setValueCount(valueCount);
}
```
Used a function like this, assuming randomized data would be hard to
compress. And I didn't get the expected exception. Also having a very smaller
dataset could also make it hard to compress with (less history, specially with
Zstd), and it didn't work as well.
I might be wrong with my conclusion or something could be erroneous in my
test data, appreciate your feedback here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
