GeneTinderholm commented on issue #279: URL: https://github.com/apache/arrow-go/issues/279#issuecomment-2659982772
Further update, the `runtime.LockOSThread` does not seem to have completely solved the issue, just reduced the occurrence. We're using go 1.23. I found [this issue](https://github.com/golang/go/issues/69085) which seems eerily similar, except for the cgo usage. I don't know how the bp would get nil'd out without it, but the assembly function is using that register, so just in case I updated to go 1.24 (needed an excuse to do that anyways). Just in case the compiler is doing something screwy that nobody was expecting. I'm going to monitor for a bit to see if I can find another example (we're down to ~1 per day for millions of requests at the moment). As much info on the parquet as I can share: - Each file has between ~30 and ~50 columns, ~80% of which are doubles - Encountering files with 1 million+ rows is a fairly common occurrence - The _vast_ majority of the files only have a single row group I'll see what I can do about getting a local copy of the library used. Unfortunately my dev platform is darwin/arm64, so I'm not confident of my ability to reproduce this exact scenario locally. If a deploy/wait cycle becomes too long, I'm just going to spin up something that calls the min/max function in a loop on as many goroutines as possible and leave it running for a while (assuming moving past go 1.23.0 doesn't eliminate this). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
