[
https://issues.apache.org/jira/browse/DRILL-5811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16197829#comment-16197829
]
ASF GitHub Bot commented on DRILL-5811:
---------------------------------------
Github user vrozov commented on a diff in the pull request:
https://github.com/apache/drill/pull/965#discussion_r143593333
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/schedule/BlockMapBuilder.java
---
@@ -104,12 +104,16 @@ public BlockMapReader(FileStatus status, boolean
blockify) {
@Override
protected List<CompleteFileWork> runInner() throws Exception {
final List<CompleteFileWork> work = Lists.newArrayList();
+
+ final Set<String> noDrillbitHosts = logger.isDebugEnabled() ?
Sets.<String>newHashSet() : null;
--- End diff --
Consider moving `noDillbitHosts` to `BlockMapBuilder` class (use
`Sets.newConcurrentHashSet()` in this case) as it does not seem to belong to
`BlockMapReader`. With such change, other changes are not necessary and likely
this will allow reducing repeated log messages even further. Drop `<Sring>`
from `Sets.<String>newHashSet()`.
> Large number of "Failure finding Drillbit" messages when using MFS
> ------------------------------------------------------------------
>
> Key: DRILL-5811
> URL: https://issues.apache.org/jira/browse/DRILL-5811
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.12.0
> Reporter: Paul Rogers
> Assignee: Timothy Farkas
> Priority: Minor
> Fix For: 1.12.0
>
>
> A query was run on a single node of a three-node cluster running the MapR
> file system (MFS.) When planning the query, zillions of messages of the
> following form appear in the log:
> {code}
> 2017-09-21 16:18:19,401 [263bb743-aaf9-eeb1-2006-32d573e03eba:foreman] INFO
> o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
> numFiles: 1
> 2017-09-21 16:18:19,408 [263bb743-aaf9-eeb1-2006-32d573e03eba:foreman] DEBUG
> o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
> qa-node113.qa.lab. Skipping affinity to that host.
> 2017-09-21 16:18:19,409 [263bb743-aaf9-eeb1-2006-32d573e03eba:foreman] DEBUG
> o.a.d.e.s.schedule.BlockMapBuilder - FileWork group
> (maprfs:///drill/testdata/resource-manager/250wide.tbl,0) max bytes
> 17132639127
> 2017-09-21 16:18:19,409 [263bb743-aaf9-eeb1-2006-32d573e03eba:foreman] DEBUG
> o.a.d.e.s.schedule.BlockMapBuilder - Took 1 ms to set endpoint bytes
> {code}
> Might we put such messages a TRACE level, leaving an aggregate message at
> DEBUG level, something like:
> {code}
> 123 files to be read using remote reads because of missing Drillbit
> processes. Enable trace logging for details.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
