It makes sense to adjust its LOG level to reduce log size on master. 1. It affects master performance if log file size is huge.Query on hawq_toolkit.hawq_log_master_concise is slower as time going. https://issues.apache.org/jira/projects/HAWQ/issues/HAWQ-1550 2. It has little value when segments are seperated from storage nodes.If storage nodes and computing nodes are geographically distributed,it is common on Cloud service, and no distributed cache systems are available,then we have little interest to keep them in log in such cases.
data locality ratio -------------—— Kuien Liu/奎恩 ------------------------------------------------------------------发件人：Kyle Roberts <[email protected]>发送时间：2017年12月1日(星期五) 05:41收件人：dev <[email protected]>主 题：Default LOG level for "data locality ratio:" events contribute to Hawq Master log bloat Issue: Default LOG level for "data locality ratio:" events contribute to Hawq Master log bloat. Should only be needed when investigating/tuning performance. Case Scenario: *"data locality ratio:"* events can report multiple times and, depending upon the query, many times. Sample event: 2017-11-30 20:33:39.857251 GMT,"gpadmin","testdb",p280349,th-909919968,"[local]",,2017-11-30 20:06:35 GMT,1630750,con25419,cmd28,seg-1,,,,sx2,"LOG","00000","data locality ratio: 1.000; virtual segment number: 1; different host number: 1; virtual segment number per host(avg/min/max): (1/1/1); segment size(avg/min/max): (1016.000 B/1016 B/1016 B); segment size with penalty(avg/min/max): (1016.000 B/1016 B/1016 B); continuity(avg/min/max): (1.000/1.000/1.000).",,,,,"SQL statement ""select count(*)::float4 from public.test_tbl as Ta where Ta.id is null""","analyze test_tbl;",0,,"cdbdatalocality.c",3372, For example, Hawq master logs are seen with close to *20% of the file size* is from only the *"data locality ratio:" *events. Product Enhancement suggestion: Because these are only needed for performance tuning (or for investigation): In: https://github.com/apache/incubator-hawq/blob/master/src/backend/cdb/cdbdatalocality.c#L3372 Could we change default LOG level to something like DEBUG1, etc? - Kyle
