[
https://issues.apache.org/jira/browse/CONNECTORS-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright reassigned CONNECTORS-1574:
---------------------------------------
Assignee: Karl Wright
> Performance tuning of manifold
> ------------------------------
>
> Key: CONNECTORS-1574
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1574
> Project: ManifoldCF
> Issue Type: Bug
> Components: File system connector, JCIFS connector, Solr 6.x
> component
> Affects Versions: ManifoldCF 2.5
> Environment: Apache manifold installed in Linux machine
> Linux version 3.10.0-327.el7.ppc64le
> Red Hat Enterprise Linux Server release 7.2 (Maipo)
> Reporter: balaji
> Assignee: Karl Wright
> Priority: Critical
> Labels: performance
>
> My team is using *Apache ManifoldCF 2.5 with SOLR Cloud* for indexing of
> data. we are currently having 450-500 jobs which needs to run simultaneously.
> We need to index json data and we are using connector type as *file system*
> along with *postgres* as backend database.
> We are facing several issues like
> 1. Scheduling works for some jobs and doesn't work for other jobs.
> 2. Some jobs gets completed and some jobs hangs and doesn't get completed.
> 3. With one job earlier 60000 documents was getting indexed in 15minutes but
> now even a directory path having 5 documents takes 20 minutes or sometimes
> doesn't get completed
> 4. "list all jobs" or "status and job management" page doesn't load sometimes
> and on seeing the pg_stat_activity we observe that 2 queries are in waiting
> state state because of which the page doesn't load. so if we kill those
> queries or restart manifold the issue gets resolved and the page loads
> properly
> queries getting stuck:
> 1. SELECT ID,FAILTIME, FAILCOUNT, SEEDINGVERSION, STATUS FROM JOBS WHERE
> (STATUS=$1 OR STATUS=$2) FOR UPDATE
> 2. UPDATE JOBS SET ERRORTEXT=NULL, ENDTIME=NULL, WINDOWEND=NULL, STATUS=$1
> WHERE ID=$2
> note : We have deployed manifold in *linux*. Our major requirement is
> scheduling of jobs which will run every 15 minutes
> Please help us in fine tuning manifold so that it runs smoothly and acts as a
> robust system.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
