Re: show files problem
Good catch! Indeed I was using a SNAPSHOT version. Since I have no choice to upgrade now, I will take the opportunity to get the latest version of drill! Thanks Arina, On Mon, 15 Jul 2019 at 15:22, Arina Yelchiyeva wrote: > Hi, > > The issue you are describing was fixed in DRILL-6753 [1] which was > included into 1.15.0 release. > I suspect you are trying to upgrade to some 1.15.0-SNAPSHOT version rather > than on 1.15.0 final version. > > [1] https://issues.apache.org/jira/browse/DRILL-6753 > > Kind regards, > Arina > > > On Jul 15, 2019, at 9:17 PM, François Méthot > wrote: > > > > Hi, > > > > We are aiming to upgrade our Drill cluster to version 1.15. On my test > > environment, we found that show files operation can take close to 1+ hour > > to list content of a directory with only a handful of files. > > > > Based on Jstack, it seems a drillbit is busy collecting permission > > information. > > Also I see the process "ls -ld .../subpath/files.xyz" > > showing up in my ps -ef output. > > > > I have read about the new information_schema, I suspect it is linked to > > that feature, is there a flag to enable the show files behaviour from pre > > 1.15 versions? > > > > The folder and files I am navigating are on an NFS mount. > > > > Any hints or help would be appreciated > > > > Thanks > > Francois > > > > ex: > >> show files from wrk.base.`2019/`; > > > > error... you should enable list_files_recursive > >> alter system set `storage.list_files_recursive`=true > > > >> show files from wrk.base.`2019/`; > > takes hours to get result > > > > > > wrk.base is /data/ > > which contain: > > /2019 > > /07 > > /14/~1000 files > > /13/~1000 files > > /12/~1000 files > > ... > > /2018 > > ... > > /07 > > /14/~1000 files > > ... > >
Re: show files problem
Hi, The issue you are describing was fixed in DRILL-6753 [1] which was included into 1.15.0 release. I suspect you are trying to upgrade to some 1.15.0-SNAPSHOT version rather than on 1.15.0 final version. [1] https://issues.apache.org/jira/browse/DRILL-6753 Kind regards, Arina > On Jul 15, 2019, at 9:17 PM, François Méthot wrote: > > Hi, > > We are aiming to upgrade our Drill cluster to version 1.15. On my test > environment, we found that show files operation can take close to 1+ hour > to list content of a directory with only a handful of files. > > Based on Jstack, it seems a drillbit is busy collecting permission > information. > Also I see the process "ls -ld .../subpath/files.xyz" > showing up in my ps -ef output. > > I have read about the new information_schema, I suspect it is linked to > that feature, is there a flag to enable the show files behaviour from pre > 1.15 versions? > > The folder and files I am navigating are on an NFS mount. > > Any hints or help would be appreciated > > Thanks > Francois > > ex: >> show files from wrk.base.`2019/`; > > error... you should enable list_files_recursive >> alter system set `storage.list_files_recursive`=true > >> show files from wrk.base.`2019/`; > takes hours to get result > > > wrk.base is /data/ > which contain: > /2019 > /07 > /14/~1000 files > /13/~1000 files > /12/~1000 files > ... > /2018 > ... > /07 > /14/~1000 files > ...
show files problem
Hi, We are aiming to upgrade our Drill cluster to version 1.15. On my test environment, we found that show files operation can take close to 1+ hour to list content of a directory with only a handful of files. Based on Jstack, it seems a drillbit is busy collecting permission information. Also I see the process "ls -ld .../subpath/files.xyz" showing up in my ps -ef output. I have read about the new information_schema, I suspect it is linked to that feature, is there a flag to enable the show files behaviour from pre 1.15 versions? The folder and files I am navigating are on an NFS mount. Any hints or help would be appreciated Thanks Francois ex: > show files from wrk.base.`2019/`; error... you should enable list_files_recursive >alter system set `storage.list_files_recursive`=true > show files from wrk.base.`2019/`; takes hours to get result wrk.base is /data/ which contain: /2019 /07 /14/~1000 files /13/~1000 files /12/~1000 files ... /2018 ... /07 /14/~1000 files ...
