Yes,it is.
On 01/21/2019 18:27，Lionel Liu<[email protected]> wrote： Ok, I got it. Because the spark applications are submitted by livy in cluster mode, if the application driver is scheduled on a node which doesn't know es host, it could not persist the metric into es. Thanks, Lionel On Fri, Jan 18, 2019 at 10:09 AM 大鹏 <[email protected]> wrote: In other words, I did not set the IP of es server in /etc/hosts on the yarn server, so the metric save request could not reach the es server and the metric data was lost On 01/17/2019 16:52，Lionel Liu<[email protected]> <[email protected]> wrote： I'm not sure about it, maybe I'm not clear of your situation. IMO, if it's the configuration issue, all the metrics should be lost, not like this. Thanks, Lionel On Thu, Jan 17, 2019 at 11:00 AM 大鹏 <[email protected]> wrote: I think I have found the reason. It should be that I did not configure es service IP on yarn server, so there is no metric data in es On 01/15/2019 09:34，Lionel Liu<[email protected]> <[email protected]> wrote： I think you need to check the logs of livy and spark applications. In livy log, you can find how many jobs are submitted to spark cluster. In spark cluster UI, you can get the actual jobs number, and how many of them are success or not. Furthermore, you can try to find the yarn log of the missing points, if the job actually executed, to find the error message in the log, then we can get more information about this case. Thanks, Lionel On Mon, Jan 14, 2019 at 1:36 PM 大鹏 <[email protected]> wrote: This is attachment On 01/14/2019 13:22，Lionel Liu<[email protected]> <[email protected]> wrote： Hi DaPeng, Griffin reads your data, execute the rule steps on the data, then persist the metrics. If there's any exception like data can not find or execution error, the rule step might fail, and the following steps will not success either, the metrics is collected after the last step, thus there might be no metrics in such a situation. All the exceptions will be logged in the application log, you can find some information there. I cannot see your attachment, I'm not sure what kind of job are you running, what's the data like, is it accessible in every minute? Thanks, Lionel On Mon, Jan 14, 2019 at 11:06 AM 大鹏 <[email protected]> wrote: What is the strategy of ES to save metric?My task was executed every five minutes, and part of the metric generated in the process of execution was lost, which did not match the number of task execution. For this problem, please see the red part in the attachment.
