Hi, Yes, there are some site.ducc.property entries that will speed up the timing. Will respond with those tomorrow. Are you often running jobs with only a single work item?
Eddie On Sat, Nov 28, 2015 at 7:23 PM, Yi-Wen Liu <[email protected]> wrote: > Hi, > > I am using ducc to process text files(cTAKES), and one of my input is quite > short, about 10 lines. > But it takes more than two minutes to process it, as follows: > After submitting, > 00:00-00:08 > no status > 00:09-00:30 > waiting for driver > 00:31-01:00 > waiting for resources > 01:01-02:00 > initializing > 02:01-02:30 > completing > 02:31 > completed > > Is there any way to lower the preprocessing time?(Time to wait for driver, > resources, initializing...) > > I am wondering why it takes so long before completing, and have tried > different parameter values, for example lower initialization time, lower > resources needed, but didn't have much improvement. > > Here's parameters I am using now: process_memory_size 2 > process_jvm_args -Xmx4g > driver_jvm_args -Xmx4g > process_thread_count 2 > process_per_item_time_max 5 > process_deployments_max 999 > environment AE_INIT_TIME=5 AE_INIT_RANGE=5 INIT_ERROR=0 > > Any suggestion is appreciated. > > Thanks, > Yi-Wen >
