[
https://issues.apache.org/jira/browse/HAWQ-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438486#comment-15438486
]
Paul Guo commented on HAWQ-984:
-------------------------------
1) gpscp
It still uses popen2.Popen4(). This function introduces millions of useless
close() sys call finally in above test command. Using subprocess.Popen()
without close_fds as an alternative easily resolves this.
2) gppylib/commands/base.py
def __init__(self,name,pool,timeout=5):
The worker thread will block at most 5 seconds in each loop (Queue.get()) to
fetch potential commands even we have known that there will be no more commands
to run for some threads. This really does not make sense since some idle
threads will block for 5 seconds also before exiting.
Setting timeout to zero will make python code spin. I tested a small timeout
value e.g. 0.1s and it works fine. It seems that 0.1 is a good timeout
candidate.
> hawq config is too slow.
> ------------------------
>
> Key: HAWQ-984
> URL: https://issues.apache.org/jira/browse/HAWQ-984
> Project: Apache HAWQ
> Issue Type: Bug
> Components: Command Line Tools
> Reporter: Paul Guo
> Assignee: Lei Chang
> Fix For: 2.0.1.0-incubating
>
>
> I tried to set a simple guc value via "hawq config" on my centos
> virtual system, but it spends >6 seconds. I know "hawq config" just
> simple scp-update xml files to various nodes. This should have
> been very fast if network situation is fine (My test hawq system is all-in-one
> test environment so network latency is not an issue.)
> I expect this is done on my test system (just one seg node) in < 1 second.
> But actually the time is:
> $ time hawq config -c lc_messages -v en_US.UTF-8
> GUC lc_messages already exist in hawq-site.xml
> Update it with value: en_US.UTF-8
> GUC : lc_messages
> Value : en_US.UTF-8
> real 0m6.363s
> user 0m0.943s
> sys 0m1.258s
> I quickly monitored the processes when running "hawq config",
> it seems that the scp command finishes very early and adding
> print debug code in python file gpscp shows that the code in it
> finishes also very early.
> I'm not sure if anyone sees that long like my case (my system
> is probably quite mangled after dev and test). I roughly traced
> the python stack. I suspect there is something wrong in python
> or we are not using related module well or we should replace modules.
> I just checked "hawq config" but it is possible other CLI suffers from this
> also.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
