Re: RSS Connection Paused

[Matt Christianson]

Thanks Karl.

I was able to get this working by joining the thread similar to here.
https://github.com/apache/manifoldcf/blob/trunk/connectors/elasticsearch/connector/src/main/java/org/apache/manifoldcf/agents/output/elasticsearch/ElasticSearchConnection.java

thanks,
—Matt

On Thu, Mar 8, 2018 at 8:09 AM Karl Wright  wrote:

> Hi Matt,
>
> These documents have been queued for rescan.  That means that there was
> possibly an error of some kind encountered when fetching them, that the RSS
> connector concluded might go away if retried.  Either that, or you are
> running the job in "continuous" mode, in which case it will rescan
> documents periodically based on how often they change.
>
> If you want to see what errors might have occurred, have a look at the log
> and also the Simple History.
>
> Thanks,
> Karl
>
>
> On Wed, Mar 7, 2018 at 10:58 PM, Matt Christianson <
> christianson.m...@gmail.com> wrote:
>
> > The Null Output Connector is also pausing after processing 8 of 11
> > documents.
> >
> > The remaining jobqueue records have a status of ‘A’ and a checkaction
> value
> > of ‘R’.
> >
> > On Wed, Mar 7, 2018 at 1:45 PM Karl Wright  wrote:
> >
> > > Hi Matt,
> > >
> > > Can you try running the same RSS configuration using the null output
> > > connector and seeing how it behaves?
> > >
> > > Thanks,
> > > Karl
> > >
> > >
> > > On Wed, Mar 7, 2018 at 12:29 PM, Matt Christianson <
> > > christianson.m...@gmail.com> wrote:
> > >
> > > > When using a RSS Connection and a custom Output Connector I’m running
> > > into
> > > > an issue where the first 8 documents process successfully, but then
> it
> > > > seems to pause and not process any other documents. Looking at the
> > > Document
> > > > Status I see that 9 other documents have processed and there are 2
> that
> > > > have a status of “Processing”. There is no activity in the logs. If I
> > > stop
> > > > and start the server the remaining documents do get processed.
> > > >
> > > > Running ManifoldCF 2.9.1 example connected to local Postgres 9.5.5.
> The
> > > > custom Output Connector is a simple post to a rest endpoint.
> > > >
> > > > Thank you,
> > > > —Matt
> > > >
> > >
> >
>

Re: RSS Connection Paused

[Karl Wright]

Hi Matt,

These documents have been queued for rescan.  That means that there was
possibly an error of some kind encountered when fetching them, that the RSS
connector concluded might go away if retried.  Either that, or you are
running the job in "continuous" mode, in which case it will rescan
documents periodically based on how often they change.

If you want to see what errors might have occurred, have a look at the log
and also the Simple History.

Thanks,
Karl


On Wed, Mar 7, 2018 at 10:58 PM, Matt Christianson <
christianson.m...@gmail.com> wrote:

> The Null Output Connector is also pausing after processing 8 of 11
> documents.
>
> The remaining jobqueue records have a status of ‘A’ and a checkaction value
> of ‘R’.
>
> On Wed, Mar 7, 2018 at 1:45 PM Karl Wright  wrote:
>
> > Hi Matt,
> >
> > Can you try running the same RSS configuration using the null output
> > connector and seeing how it behaves?
> >
> > Thanks,
> > Karl
> >
> >
> > On Wed, Mar 7, 2018 at 12:29 PM, Matt Christianson <
> > christianson.m...@gmail.com> wrote:
> >
> > > When using a RSS Connection and a custom Output Connector I’m running
> > into
> > > an issue where the first 8 documents process successfully, but then it
> > > seems to pause and not process any other documents. Looking at the
> > Document
> > > Status I see that 9 other documents have processed and there are 2 that
> > > have a status of “Processing”. There is no activity in the logs. If I
> > stop
> > > and start the server the remaining documents do get processed.
> > >
> > > Running ManifoldCF 2.9.1 example connected to local Postgres 9.5.5. The
> > > custom Output Connector is a simple post to a rest endpoint.
> > >
> > > Thank you,
> > > —Matt
> > >
> >
>

Re: RSS Connection Paused

[Matt Christianson]

The Null Output Connector is also pausing after processing 8 of 11
documents.

The remaining jobqueue records have a status of ‘A’ and a checkaction value
of ‘R’.

On Wed, Mar 7, 2018 at 1:45 PM Karl Wright  wrote:

> Hi Matt,
>
> Can you try running the same RSS configuration using the null output
> connector and seeing how it behaves?
>
> Thanks,
> Karl
>
>
> On Wed, Mar 7, 2018 at 12:29 PM, Matt Christianson <
> christianson.m...@gmail.com> wrote:
>
> > When using a RSS Connection and a custom Output Connector I’m running
> into
> > an issue where the first 8 documents process successfully, but then it
> > seems to pause and not process any other documents. Looking at the
> Document
> > Status I see that 9 other documents have processed and there are 2 that
> > have a status of “Processing”. There is no activity in the logs. If I
> stop
> > and start the server the remaining documents do get processed.
> >
> > Running ManifoldCF 2.9.1 example connected to local Postgres 9.5.5. The
> > custom Output Connector is a simple post to a rest endpoint.
> >
> > Thank you,
> > —Matt
> >
>

Re: RSS Connection Paused

[Karl Wright]

Hi Matt,

Can you try running the same RSS configuration using the null output
connector and seeing how it behaves?

Thanks,
Karl


On Wed, Mar 7, 2018 at 12:29 PM, Matt Christianson <
christianson.m...@gmail.com> wrote:

> When using a RSS Connection and a custom Output Connector I’m running into
> an issue where the first 8 documents process successfully, but then it
> seems to pause and not process any other documents. Looking at the Document
> Status I see that 9 other documents have processed and there are 2 that
> have a status of “Processing”. There is no activity in the logs. If I stop
> and start the server the remaining documents do get processed.
>
> Running ManifoldCF 2.9.1 example connected to local Postgres 9.5.5. The
> custom Output Connector is a simple post to a rest endpoint.
>
> Thank you,
> —Matt
>

RSS Connection Paused

[Matt Christianson]

When using a RSS Connection and a custom Output Connector I’m running into
an issue where the first 8 documents process successfully, but then it
seems to pause and not process any other documents. Looking at the Document
Status I see that 9 other documents have processed and there are 2 that
have a status of “Processing”. There is no activity in the logs. If I stop
and start the server the remaining documents do get processed.

Running ManifoldCF 2.9.1 example connected to local Postgres 9.5.5. The
custom Output Connector is a simple post to a rest endpoint.

Thank you,
—Matt