[
https://issues.apache.org/jira/browse/OOZIE-3382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705086#comment-16705086
]
Hadoop QA commented on OOZIE-3382:
----------------------------------
Testing JIRA OOZIE-3382
Cleaning local git workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any star imports
. {color:green}+1{color} the patch does not introduce any line longer than
132
. {color:green}+1{color} the patch adds/modifies 6 testcase(s)
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} Javadoc generation succeeded with the patch
. {color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
. {color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
. Tests run: 3096
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:green}*+1 Overall result, good!, no -1s*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/934/
> Optimize SshActionExecutor's drainBuffers method
> ------------------------------------------------
>
> Key: OOZIE-3382
> URL: https://issues.apache.org/jira/browse/OOZIE-3382
> Project: Oozie
> Issue Type: Improvement
> Reporter: Andras Salamon
> Assignee: Andras Salamon
> Priority: Major
> Attachments: OOZIE-3382-01.patch, OOZIE-3382-02.patch,
> OOZIE-3382-03.patch, OOZIE-3382-04.patch
>
>
> OOZIE-3354 improved {{SshActionExecutor}} to avoid {{Process#waitFor()}}
> blocks and modified the {{drainBuffers}} method to keep draining the standard
> output (and standard error) continuously.
> Right now the speed of the drain is hardwired. As long as the process is
> running the method only reads 1024 bytes in each cycle (half a second) which
> can take very long time if we want to drain several megabytes (for instance
> {{oozie.servlet.CallbackServlet.max.data.len}} is increased).
> Let's optimize the draining.
> We can either read 1024 bytes multiple times in each cycle (as long as there
> are data in the buffer), or we can increase the value of the buffer size
> (1024).
> In the latter case the default of the buffer size could be half of the
> {{oozie.servlet.CallbackServlet.max.data.len}} value, but we also need an
> additional property to specify the buffer size (to avoid memory problems
> because of using a very big buffer). We can keep 1024 as a minimum buffer
> size.
> It would be also useful to refactor the code and put the buffer draining into
> a separate class and create unit tests for the class. Using this class in
> {{ShellMain}} to avoid code duplication would also be very useful, but we
> have to fix OOZIE-3359 first.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
