Perhaps Gandiva does not handle sliced arrays properly? This would be worth investigating
On Mon, Jul 27, 2020 at 7:43 PM Matt Youill <matt.you...@burnian.com> wrote: > > Managed to track down the issue (sort of). > > I removed a call to set_chunksize on TableBatchReader where the chunk > size was less than the number of rows in a table being read. Runs fine > now (tested with 100s threads over mils of rows). > > Strangely, Gandiva fails if I don't call set_chunksize for a subsequent > filter evaluation. It's very strange. > > > On 28/7/20 7:58 am, Wes McKinney wrote: > > Crashing when running from multiple threads doesn't sound right, > > perhaps there are some missing synchronizations in internal data > > structures. Could you open a JIRA issue and show the backtraces of any > > crashes or other clues about how to reproduce the issues? > > > > On Sun, Jul 26, 2020 at 8:12 PM Matt Youill <matt.you...@burnian.com> wrote: > >> Hi, > >> > >> Are there any guidelines for using the Gandiva C++ lib from multiple > >> threads? As I increase the number of threads in an application Gandiva > >> begins to raise memory faults (e.g. double free). Sometimes it is on > >> projector creation, sometimes evaluation, seems to happen in multiple > >> unpredictable places inside Gandiva. Can it be used from multiple > >> threads, and if so which parts need to be guarded, if any? > >> > >> Thanks, Matt > >>
