EagleConfigFactory, LogReader, LogWriter are developed for reading/writing data to HBase only at the beginning of Eagle open source. At that time, HBase acts as both metadata storage and log/metric data storage. But some feedback from community is that Eagle sometimes is used for monitoring where HBase is not ready. There is a requirement that Eagle can be run without HBase dependency. That is why in Eagle 0.5, all metadata are stored in relational database or mongodb which is common to most users.
But for application data, for example metrics or logs, it is still not decided yet which storage is final choice. But for job related performance monitoring, all data including metrics and logs are stored in HBase. Some other storage is also evaluated for example druid for metrics and elasticsearch for log. But it is not finalized. Difference between log and metric is that metric stores numeric data with dimensions, while log stores any type of data. Some storage like TSDB, Druid they made optimization for metric storage. Thanks Edward On Mon, Aug 22, 2016 at 2:08 AM, Chang Chen <[email protected]> wrote: > What make me confused is EagleConfigFactory directly depends on HBase. But > there are 2 interfaces for persisting data, one is DataStorage, the other > are LogReader and LogWriter. Which one will be removed? > > I also noticed that eagle 0.5 architecture ( > https://cwiki.apache.org/confluence/display/EAG/Architecture+of+Eagle+0.5) > was updated. What are the difference between "log" and "metric"? > > Thanks > Chang > > On Wednesday, August 17, 2016, Liangfei.Su <[email protected]> wrote: > > > On test scenario with different config need for differest, i simply do > > ConfigFactory.invalidateCaches() in setup(). > > > > > > On Wed, Aug 17, 2016 at 9:43 AM, Edward Zhang <[email protected] > > <javascript:;>> > > wrote: > > > > > Sorry I mean "remove" :-) > > > > > > Thanks > > > Edward > > > > > > On Tue, Aug 16, 2016 at 6:35 PM, Chang Chen <[email protected] > > <javascript:;>> wrote: > > > > > > > Hi Edward > > > > > > > > Does remote mean run TestEagleConfig in different JVM? > > > > > > > > Thanks > > > > Chang > > > > > > > > On Wednesday, August 17, 2016, Edward Zhang <[email protected] > > <javascript:;>> > > > > wrote: > > > > > > > > > TestEagleConfig is not a valid test case because it assumes that it > > can > > > > > initialize EagleConfigFactory, but when we execute multiple test > > cases, > > > > > this may not work as EagleConfigFactory may already be initialized > > > before > > > > > this test case is executed. So setting "config.resource" will not > > have > > > > any > > > > > effect. > > > > > > > > > > I suggest we remote TestEagleConfig, and also going forward we can > > need > > > > > refactor EagleConfigFactor to be not a singleton but use > > > > > com.typesafe.config.Config as constructor parameter. > > > > > > > > > > Thanks > > > > > Edward > > > > > > > > > > On Tue, Aug 16, 2016 at 4:26 AM, Chang Chen <[email protected] > > <javascript:;> > > > > > <javascript:;>> wrote: > > > > > > > > > > > Hi Guys > > > > > > > > > > > > In the middle of writing UT for Eagle, we found it is difficult > to > > > mock > > > > > > EagleConfigFactory, since it has static method. The current > > > workaround > > > > > is > > > > > > introducing powermock, so that we can set manager to mock object > , > > > i.e. > > > > > > > > > > > > Whitebox.setInternalState(EagleConfigFactory.class, > "manager", > > > > > > eagleConfigFactory); > > > > > > > > > > > > This is OK in most cases, but TestEagleConfig.testInit could be > > > failed, > > > > > > because manager only initialized at first time and > > > > EagleConfigFactory's > > > > > > Constructor is private: > > > > > > > > > > > > private static EagleConfigFactory manager = new > > > > EagleConfigFactory(); > > > > > > > > > > > > EagleConfigFactory actually has two roles, one is factory and > > another > > > > is > > > > > > configuration. I propose refactoring EagleConfigFactory and > > > distinguish > > > > > > these two roles: > > > > > > > > > > > > public class EagleConfigImpl implements EagleConfig{ > > > > > > } > > > > > > > > > > > > public class EagleConfigFactory { > > > > > > public static EagleConfig load(){} > > > > > > public static void reset(EagleConfig config){ > > > > > > if (config == null) { > > > > > > //default > > > > > > }else{ > > > > > > } > > > > > > } > > > > > > } > > > > > > > > > > > > However doing so, we need call EagleConfigFactory.reset at the > > > > beginning > > > > > of > > > > > > eagle start. > > > > > > > > > > > > Any ideas? > > > > > > > > > > > > Thanks > > > > > > Chang > > > > > > > > > > > > > > > > > > > > >
