Hi Xiangwei, Good work! I could do the review.
Thanks, ————————————————— Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 Xiangwei Wei <[email protected]> 于2020年9月19日周六 下午3:55写道： > Hi, > > I've submitted a PR[1] for this. The design principle is show in the > former emails and the user guide doc has been added to this PR. > > Welcome to reviewing, and any ideas about this monitor module. > > > > [1] https://github.com/apache/incubator-iotdb/pull/1725/files > > Xiangwei Wei <[email protected]> 于2020年9月7日周一 下午4:40写道： > >> Hi, >> >> I've completed a PR for this(not committed yet), and used JMX for >> visualization. You can see it in Jconsole.[1] >> >> But something still needs to be thought of. >> >> 1. *What about adding device number as an attribute?* >> >> TOTAL_DEVICE(Calculate the total device number.) I think it's useful for >> users in monitoring data writing. >> >> 2. *Show monitor attributes command.* >> >> Currently, we can use commands like `select TOTAL_POINTS from >> root.stats.global` to see the monitor attributes. It will show like a >> common time series: >> >> | Time | root.stats.global.TOTAL_POINTS | >> | ... | ... | >> | ... | ... | >> >> But we can't get the latest info this way as the info is updated during >> each flush(). And sometimes users only care about the real-time value >> instead the process of data changing. >> >> Therefore I think maybe we need a new command to complete this function. >> >> But I don't carry out a good idea. If you have any ideas or some >> references, welcome to reply. >> >> >> >> >> >> [1] >> [image: image.png] >> >> >> >> >> 1. What about DEVICE_NUM >> >> Xiangwei Wei <[email protected]> 于2020年8月26日周三 下午4:57写道： >> >>> Hi, >>> >>> > I think TOTAL_REQ_SUCCESS is also important in some cases. >>> >>> Yes, we can keep the TOTAL_REQ_FAIL also as a contrast. >>> >>> >>> > By the way, there is a branch "feature/improve-monitoring-metrics" >>> which is >>> > written by Julian and use Micrometer to collect some performance >>> info. >>> >>> Any introduction or doc about this? >>> >>> >>> >>> >>> Julian Feinauer <[email protected]> 于2020年8月26日周三 下午4:49写道： >>> >>>> Hi, >>>> >>>> indeed there is potential for some more modularity (and I think I >>>> already added a submodule in that branch). >>>> Currently Xiangdong is improving the collected metrics and I would be >>>> happy to get this into the master / develop ASAP : ) >>>> >>>> Julian >>>> >>>> Am 26.08.20, 08:10 schrieb "Xiangdong Huang" <[email protected]>: >>>> >>>> Hi, >>>> >>>> I think TOTAL_REQ_SUCCESS is also important in some cases. >>>> >>>> By the way, there is a branch "feature/improve-monitoring-metrics" >>>> which is >>>> written by Julian >>>> >>>> and use Micrometer to collect some performance info. >>>> >>>> I think we can maintain that branch in the next step. >>>> (By the way, I believe that making the project more modularity is >>>> beneficial. >>>> We can extract the implementation as interfaces to let users decide >>>> using >>>> which way to get the info) >>>> >>>> Best, >>>> ----------------------------------- >>>> Xiangdong Huang >>>> School of Software, Tsinghua University >>>> >>>> 黄向东 >>>> 清华大学 软件学院 >>>> >>>> >>>> Xiangwei Wei <[email protected]> 于2020年8月26日周三 下午1:52写道： >>>> >>>> > Hi, >>>> > >>>> > Currently, the status monitor module of iotdb is abandoned and not >>>> > maintained now. It's time for us to restart this module. >>>> > >>>> > Before, it was designed including two child modules: 1. the >>>> writing data >>>> > status monitor and 2. the file size monitor. >>>> > >>>> > 1. The writing data monitor module is responsible for collecting >>>> writing >>>> > data statistics including >>>> > - TOTAL_POINTS (Calculate the total writing points number.) >>>> > - TOTAL_REQ_SUCCESS (Calculate the successful requests number.) >>>> > - TOTAL_REQ_FAIL (Calculate the failed requests number.) >>>> > - TOTAL_POINTS_FAIL (Calculate the failed writing points number.) >>>> > - TOTAL_POINTS_SUCCESS >>>> > etc. >>>> > >>>> > and presenting them to users by statistics time series like >>>> > .root.stats.write.global.TOTAL_POINTS. It's not supported >>>> completely now. >>>> > >>>> > Actually, many parts of origin design are redundant and not >>>> meaningful for >>>> > users. Therefore, *I want to just keep TOTAL_POINTS here and >>>> remove >>>> > others*. >>>> > And i did not come up with some good ideas about *what needs to >>>> be added to >>>> > this monitor* while writing data. Welcome to any ideas here! >>>> > >>>> > >>>> > >>>> > 2. The file size monitor is concerned about how much space the >>>> data file >>>> > takes. It's supported partly and very limited now. >>>> > >>>> > In my opinion, it's easily for users to see how much space the >>>> data file >>>> > takes by a visible interface or just a Linux code. And it's more >>>> flexible >>>> > for users to decide what to calculate by themselves. Therefore, >>>> there is no >>>> > sufficient reason for us to redesign and reimplement this child >>>> module. *I >>>> > want to just remove it from iotdb.* >>>> > >>>> > The impletation way of status monitor module was designed as >>>> setting a >>>> > monitor thread and returning the status info every >>>> > *back_loop_period_in_second* (default 5s), removing outdated data >>>> > every *stat_monitor_retain_interval_in_second >>>> > (default 600s). *It cost a lot and is meaningless if no write was >>>> made >>>> > then. >>>> > >>>> > Therefore, we want to collect the data while writing data and >>>> keep it in >>>> > memory temperarily, *every time flush() is invoked or user want >>>> to select >>>> > the monitor data, we update the data info to the disk and return >>>> users the >>>> > lastest data.* >>>> > >>>> > That what need to be monitored which is useful for users except >>>> > TOTAL_POINTS still need to be discussed. Welcome to any ideas. >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > -- >>>> > Best, >>>> > Xiangwei Wei >>>> > >>>> >>>> >>> >>> -- >>> Best, >>> Xiangwei Wei >>> >> >> >> -- >> Best, >> Xiangwei Wei >> > > > -- > Best, > Xiangwei Wei >
