Re: Sporadic delays in task execution

2019-03-24 Thread DImuthu Upeksha
Hi Lee, Thanks for letting us know. We would happy to try out the latest version. Can you please point me to such known issues (JIRA or Github issues) in latest version and then we can decide whether those issue might affect to our use case or not. Thanks Dimuthu On Fri, Mar 22, 2019 at 4:29 PM

Re: Sporadic delays in task execution

2019-03-22 Thread Hunter Lee
Let me add a caveat to my previous email. Although it comes with scalability improvements, there are currently a few known issues with the latest version. We'd encourage you to check back to make sure your current usage isn't affected. Hunter On Fri, Mar 22, 2019 at 12:35 PM Hunter Lee wrote:

Re: Sporadic delays in task execution

2019-03-22 Thread Hunter Lee
No problem. If you have further questions, let us know what kind of load you're putting on Helix as well. The newest version of Helix contains Task Framework 2.0, and has greater scalability in scheduling tasks, so you might want to consider using the newest version as well. Hunter On Fri, Mar

Re: Sporadic delays in task execution

2019-03-21 Thread Xue Junkai
Can you try one thing? Touch the ideal state to trigger an event. If workflows are not scheduled, it should scheduling has problem. Best, Junkai On Wed, Mar 20, 2019 at 10:31 PM DImuthu Upeksha wrote: > Hi Junkai, > > We are using 0.8.1 > > Dimuthu > > On Thu, Mar 21, 2019 at 12:14 AM Xue

Re: Sporadic delays in task execution

2019-03-20 Thread DImuthu Upeksha
Hi Junkai, We are using 0.8.1 Dimuthu On Thu, Mar 21, 2019 at 12:14 AM Xue Junkai wrote: > Hi Dimuthu, > > What's the version of Helix you are using? > > Best, > > Junkai > > On Wed, Mar 20, 2019 at 8:54 PM DImuthu Upeksha < > dimuthu.upeks...@gmail.com> > wrote: > > > Hi Helix Dev, > > > >

Re: Sporadic delays in task execution

2019-03-20 Thread Xue Junkai
Hi Dimuthu, What's the version of Helix you are using? Best, Junkai On Wed, Mar 20, 2019 at 8:54 PM DImuthu Upeksha wrote: > Hi Helix Dev, > > We are again seeing this delay in task execution. Please have a look at the > screencast [1] of logs printed in participant (top shell) and

Re: Sporadic delays in task execution

2019-03-20 Thread DImuthu Upeksha
Hi Helix Dev, We are again seeing this delay in task execution. Please have a look at the screencast [1] of logs printed in participant (top shell) and controller (bottom shell). When I record this, there were about 90 - 100 workflows pending to be executed. As you can see some tasks were

Re: Sporadic delays in task execution

2018-10-04 Thread DImuthu Upeksha
Hi Junkai, I'm CCing Airavata dev list as this is directly related to the project. I just went through the zookeeper path like //EXTERNALVIEW, //CONFIGS/RESOURCE as I have noticed that helix controller is periodically monitoring for the children of those paths even though all the Workflows have

Re: Sporadic delays in task execution

2018-10-02 Thread Xue Junkai
Could you please check the log of how long for each pipeline stage takes? Also, did you set expiry for workflows? Are they piled up for long time? How long for each workflow completes? best, Junkai On Wed, Sep 26, 2018 at 8:52 AM DImuthu Upeksha wrote: > Hi Junkai, > > Average load is like

Re: Sporadic delays in task execution

2018-09-26 Thread DImuthu Upeksha
Hi Junkai, Average load is like 10 - 20 workflows per minutes. In some cases it's less than that However based on the observations, I feel like it does not depend on the load and it is sporadic. Is there a particular log lines that I can filter in controller and participant to capture the

Re: Sporadic delays in task execution

2018-09-25 Thread Xue Junkai
Hi Dimuthu, At which rate, you are keep submitting workflows? Usually, Workflow scheduling is very fast. And which version of Helix you are using? Best, Junkai On Tue, Sep 25, 2018 at 8:58 AM DImuthu Upeksha wrote: > Hi Folks, > > We have noticed some delays between workflow submission and

Sporadic delays in task execution

2018-09-25 Thread DImuthu Upeksha
Hi Folks, We have noticed some delays between workflow submission and actual picking up by participants and seems like that delay is somewhat constant around 2- 3 minutes. We used to continuously submit workflows and after 2 -3 minutes, a bulk of workflows are picked by participant and execute