we have a datacenter which is currently used exlusively for spark batch jobs.
in case batch jobs are running against that environment we can see very high peaks in blocked native transport requests (up to 10k / minute). i am concerned because i guess that will slow other queries (in case other applications are going to use that dc as well). i already tried increasing native_transport_max_threads + concurrent_reads without success. during the jobs i cant find any resource limitiations on my hardware (iops, disk usage, cpu, ... is fine). am i missing something? any suggestions how to cope with that? br// roland