Thanks Zhanwei for the clarification. It is worthwhile to understand the highest tcp connection frequency that would cause OS consider the connection as DoS. We can then evaluate at which degree of concurrency (partitioned tables, column storage, etc) in hawq would cause that problem.
BTW, here is some detail mechanism of protecting DoS with sync cookies: http://security.stackexchange.com/questions/20904/using-syn-cookies-to-perform-a-dos-attack . Best regards, Ruilong Huo On Sun, Dec 11, 2016 at 10:09 AM, Zhanwei Wang <[email protected]> wrote: > HI Ruilong > > > According the pervious stress test, under very high workload, OS will > consider the connection from HAWQ to HDFS as Dos and reject the connect > request and then HAWQ query will fail. > > After disabling CO table in HAWQ, we have significantly reduced the file > write workload, I think it is time to reconsider this OS setting. > > > > Best Regards > > Zhanwei Wang > [email protected] > > > > > 在 2016年12月11日，上午8:38，Ruilong Huo <[email protected]> 写道： > > > > Hi hawq community, > > > > Anyone know that why we turn off DoS protection in hawq by setting > > net.ipv4.tcp_syncookies to off in /etc/sysctl.conf? Any other reason from > > hawq perspective other than below two more from operating system > > perspective? > > > > 1) increase the number of concurrent tcp clients > > > > 2) reduce cpu overhead for creating and processing the syncookies. Though > > the overhead is tiny > > > > BTW: HAWQ turn off DoS protection > > <http://hdb.docs.pivotal.io/210/hdb/install/install-cli.html> while > > Greenplum Database enable DoS protection > > <http://gpdb.docs.pivotal.io/4320/install_guide/prep_os_ > install_gpdb.html> by > > default. > > > > Best regards, > > Ruilong Huo > >
