[ https://issues.apache.org/jira/browse/IMPALA-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong reassigned IMPALA-9458: ------------------------------------- Assignee: (was: Tim Armstrong) > Improve runtime profile counters for slow IO from remote stores > --------------------------------------------------------------- > > Key: IMPALA-9458 > URL: https://issues.apache.org/jira/browse/IMPALA-9458 > Project: IMPALA > Issue Type: Improvement > Components: Backend > Reporter: Sahil Takiar > Priority: Major > Labels: observability > > Remote storage systems (e.g. cloud stores like S3 and ABFS) often have long > tail latencies. Most I/O finishes relatively quickly, but some calls make > take significantly longer. Even for HDFS, this is an issue (e.g. hedged reads > were developed to help mitigate tail latencies, although no such feature > exists for cloud storage connectors). > Currently, scan nodes just track the total amount of time spent reading data. > It would be good to have a summary stats counter that tracks the min, avg, > and max time spent reading data. This should at least allow us to identify > when calls to remote storage services are taking longer than usual. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org