[
https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright resolved CONNECTORS-453.
Resolution: Fixed
> ManifoldCF running with Derby 10.8.1.1 has problems paus
anch)
> ManifoldCF running with Derby 10.8.1.1 has problems pausing and aborting jobs
> -
>
> Key: CONNECTORS-453
> URL: https://issues.apache.org/jira/br
[
https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright reopened CONNECTORS-453:
Reopening for inclusion in 0.5.1
> ManifoldCF running with Derby 10.8.
> ManifoldCF running with Derby 10.8.1.1 has problems pausing and aborting jobs
> -
>
> Key: CONNECTORS-453
> URL: https://issues.apache.org/jira/browse/CONNECTORS-453
> P
[
https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright updated CONNECTORS-453:
---
Summary: ManifoldCF running with Derby 10.8.1.1 has problems pausing and
aborting jobs
1102
> ManifoldCF running with Derby 10.8.1.1 has severe performance problems
> --
>
> Key: CONNECTORS-453
> URL: https://issues.apache.org/jira/browse/CONNECTORS-453
>
[
https://issues.apache.org/jira/browse/CONNECTORS-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright resolved CONNECTORS-453.
Resolution: Fixed
> ManifoldCF running with Derby 10.8.1.1 has severe performa
time ASC FETCH NEXT 120 ROWS ONLY
Granted XID : {157557, X} Lock : ROW, JOBS, (1,7) Waiting XID : {157557, S} ,
APP, INSERT INTO hopcount (deathmark,parentidhash,id,distance,jobid,linktype)
VALUES (?,?,?,?,?,?) . The selected victim is XID : 157800.
> ManifoldCF running with D
. The selected victim is XID : 147028.
> ManifoldCF running with Derby 10.8.1.1 has severe performance problems
> --
>
> Key: CONNECTORS-453
> URL: https://issues.apac
very beginning of a crawl. Long crawls with lots of
documents don't appear to stall, however. Still trying to figure out if this
is an actual problem or something more innocuous.
> ManifoldCF running with Derby 10.8.1.1 has severe performa
ManifoldCF running with Derby 10.8.1.1 has severe performance problems
--
Key: CONNECTORS-453
URL: https://issues.apache.org/jira/browse/CONNECTORS-453
Project: ManifoldCF
Version/s: ManifoldCF next
> Implement ability to run ManifoldCF with Derby in multiprocess mode
> ---
>
> Key: CONNECTORS-178
> URL: https://issues.apache.org/jira/browse/CONNECTORS-178
>
Version/s: ManifoldCF next
> Max activity and Max bandwidth reports don't work properly under Derby or
> HSQLDB
>
>
> Key: CONNECTORS-110
> URL: https://issues
[
https://issues.apache.org/jira/browse/CONNECTORS-244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright resolved CONNECTORS-244.
Resolution: Fixed
Fix Version/s: ManifoldCF 0.3
r1163260.
> Derby deadlo
Derby deadlocks in a new way on the IngestStatus table, which isn't caught and
retried
--
Key: CONNECTORS-244
URL: https://issues.apache.org/jira/browse/CONNECTOR
[
https://issues.apache.org/jira/browse/CONNECTORS-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright resolved CONNECTORS-225.
Resolution: Fixed
Fix Version/s: ManifoldCF 0.3
r1150502
> When using De
[
https://issues.apache.org/jira/browse/CONNECTORS-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright reassigned CONNECTORS-225:
--
Assignee: Karl Wright
> When using Derby, ManifoldCF incremental indexer someti
When using Derby, ManifoldCF incremental indexer sometimes gets a deadlock
exception
Key: CONNECTORS-225
URL: https://issues.apache.org/jira/browse/CONNECTORS-225
I have not yet made HSQLDB the official Derby replacement, but it is currently
a better embedded option for many situations than Derby is.
> Derby seems too unstable in multithreaded situations to be a good database
> for ManifoldCF, so try to add support for
have been resolved, so I'm closing this ticket.
r1131056.
> Derby seems too unstable in multithreaded situations to be a good database
> for ManifoldCF, so try to add sup
QLDB. Unfortunately, performance is extremely
slow, even when the number of rows in the temporary table is only a few dozen.
> Max activity and Max bandwidth reports don't work properly under Derby
) AS i_two
I believe this can actually be generated in a manner that fits the current
abstraction.
> Max activity and Max bandwidth reports don't work properly under Derby or
> HSQLDB
>
>
>
[
https://issues.apache.org/jira/browse/CONNECTORS-110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright updated CONNECTORS-110:
---
Summary: Max activity and Max bandwidth reports don't work properly under
Der
ll
databases could go the temporary table route, but then PostgreSQL would be
unnecessarily crippled.
> Max activity and Max bandwidth reports don't work properly under Derby
> --
>
> Key:
L.
We've still got to settle on how precisely to do the equivalent of PostgreSQL's
DISTINCT ON functionality, but that's all that is left. Also, FWIW, HSQLDB
doesn't (as yet) seem to fail so spectacularly dealing with hopcounts as Derby
does.
> Derby seems too unstable i
ite documentation property list does not include the PostgreSQL-specific
> parameters, and may be missing some of the Derby ones too
> ---
>
> Key:
the PostgreSQL-specific
> parameters, and may be missing some of the Derby ones too
> ---
>
> Key: CONNECTORS-175
>
ters org.apache.manifoldcf.dbsuperusername and
org.apache.manifoldcf.dbsuperuserpassword are definitely missing.
> The site documentation property list does not include the PostgreSQL-specific
> parameters, and may be missing some of the Derb
Implement ability to run ManifoldCF with Derby in multiprocess mode
---
Key: CONNECTORS-178
URL: https://issues.apache.org/jira/browse/CONNECTORS-178
Project: ManifoldCF
Issue
e it requires a new Derby feature to resolve. The
resolution will be to assess the current version of Derby and find out whether
the required feature has been added, and barring that, opening a Derby ticket
for the feature.
> Max activity and Max bandwidth reports don't work properly
[
https://issues.apache.org/jira/browse/CONNECTORS-110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright updated CONNECTORS-110:
---
Summary: Max activity and Max bandwidth reports don't work properly under
Derby
B is not yet ready for the kinds of demands that
ManifoldCF will put on it. Working with Derby seems more appropriate since
they've been able to respond to bugs.
> Derby seems too unstable in multithreaded situations to be a good database
> for ManifoldCF, so try to add sup
The site documentation property list does not include the PostgreSQL-specific
parameters, and may be missing some of the Derby ones too
---
Key
[
https://issues.apache.org/jira/browse/CONNECTORS-170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright resolved CONNECTORS-170.
Resolution: Fixed
Fix Version/s: ManifoldCF 0.2
r1082598.
> Derby datab
Derby database driver needs to periodically update statistics
-
Key: CONNECTORS-170
URL: https://issues.apache.org/jira/browse/CONNECTORS-170
Project: ManifoldCF
Issue Type: Bug
[
https://issues.apache.org/jira/browse/CONNECTORS-170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright reassigned CONNECTORS-170:
--
Assignee: Karl Wright
> Derby database driver needs to periodically upd
up when running Derby
> --
>
> Key: CONNECTORS-166
> URL: https://issues.apache.org/jira/browse/CONNECTORS-166
> Project: ManifoldCF
> Issue Type: Bug
> Components: Framework crawler age
s to pass. The only remaining issue is that the
version of Derby built from trunk has upgrade blocked. I will therefore need
to build a version of Derby based on the latest release plus the patch instead.
> Crawl seizes up when runnin
[
https://issues.apache.org/jira/browse/CONNECTORS-166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13006583#comment-13006583
]
Karl Wright commented on CONNECTORS-166:
According to the Derby team, D
Crawl seizes up when running Derby
--
Key: CONNECTORS-166
URL: https://issues.apache.org/jira/browse/CONNECTORS-166
Project: ManifoldCF
Issue Type: Bug
Components: Framework crawler agent
ent version of Derby, to try and avoid internal deadlocks
> ---
>
> Key: CONNECTORS-163
> URL: https://issues.apache.org/jira/browse/CONNECTORS-163
> Project: ManifoldCF
>
[
https://issues.apache.org/jira/browse/CONNECTORS-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright updated CONNECTORS-163:
---
Description: Derby 10.5.3.0 irrecoverably deadlocks on the straightforward
correlated
Go to current version of Derby, to try and avoid internal deadlocks
---
Key: CONNECTORS-163
URL: https://issues.apache.org/jira/browse/CONNECTORS-163
Project: ManifoldCF
Issue
[
https://issues.apache.org/jira/browse/CONNECTORS-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright reassigned CONNECTORS-163:
--
Assignee: Karl Wright
> Go to current version of Derby, to try and avoid inter
[
https://issues.apache.org/jira/browse/CONNECTORS-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright resolved CONNECTORS-123.
Resolution: Fixed
It looks like this is a Derby bug, but it can be worked around by
tus under Derby
> --
>
> Key: CONNECTORS-123
> URL: https://issues.apache.org/jira/browse/CONNECTORS-123
> Project: ManifoldCF
> Issue Type: Bug
>
Document status report does not display the correct status under Derby
--
Key: CONNECTORS-123
URL: https://issues.apache.org/jira/browse/CONNECTORS-123
Project: ManifoldCF
functions to perform regular expression matching in
Derby. r1029455.
> Queue status report fails under Derby
> -
>
> Key: CONNECTORS-109
> URL: https://issues.apache.org/jira/browse/CONNECTORS-109
> P
[
https://issues.apache.org/jira/browse/CONNECTORS-109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright reassigned CONNECTORS-109:
--
Assignee: Karl Wright
> Queue status report fails under De
CPreparedStatement.fetchResult(Unknown Source)
at org.hsqldb.jdbc.JDBCPreparedStatement.executeUpdate(Unknown Source)
- locked <0x29a65798> (a org.hsqldb.jdbc.JDBCPreparedStatement)
at org.apache.manifoldcf.core.database.Database.execute(Database.java:56
6)
at org
d in.
However, when I try to use hsqldb for an actual crawl, in less than 10 seconds
I wind up with a java-level thread deadlock. I've posted the thread dump to
connectors-dev. All the locks seem to be deep inside hsqldb, FWIW, which leads
me to believe that perhaps hsqldb is even less stable t
Derby seems too unstable in multithreaded situations to be a good database for
ManifoldCF, so try to add support for HSQLDB
---
Key: CONNECTORS-114
things.
> Encountering deadlock using quick-start & derby
> ---
>
> Key: CONNECTORS-111
> URL: https://issues.apache.org/jira/browse/CONNECTORS-111
> Project: ManifoldCF
>
[
https://issues.apache.org/jira/browse/CONNECTORS-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karl Wright reassigned CONNECTORS-111:
--
Assignee: Karl Wright
> Encountering deadlock using quick-start &
e to run the job successfully the
first time after setup. Seems to be resolved. I ran a few other experiments
and they were fine too. Thanks!
> Encountering deadlock using quick-start & derby
> ---
>
>
and more to do
with an internal lock-ordering problem in Derby itself. So, each database
modification has a potential of throwing one of these exceptions.
I tried to fix that issue by retrying the operation should I be outside of a
transaction. r1004915.
> Encountering deadlock using quic
the derby ManifoldCF database implementation
more robust against exceptions thrown during commit or rollback. r1004786. See
if this makes any difference in your setup.
> Encountering deadlock using quick-start & derby
> ---
>
>
code in question is not apparently within a
database transaction, so it's puzzling to me how a deadlock could develop. The
possibilities are:
(1) Derby detects deadlocks in part by timeout. Perhaps the derby timeout time
is too short.
(2) It could be a plain old Derby bug.
(3) There co
o be happening the first time after a clean
setup of ManifoldCF. Subsequent jobs including the same job ran successfully.
> Encountering deadlock using quick-start & derby
> ---
>
> Key: CONNECTORS-111
> URL: ht
Encountering deadlock using quick-start & derby
---
Key: CONNECTORS-111
URL: https://issues.apache.org/jira/browse/CONNECTORS-111
Project: ManifoldCF
Issue Type: Bug
Components: Exam
S endtime FROM
>>>>>> (...) t3
>>>>> Do you have primary key in your t3 table?
>>>>>
>>>>>> In Postgresql, what this does is to return the FIRST entire row matching
>>>>>> each distinct idbucket result.
>>>
In Postgresql, what this does is to return the FIRST entire row matching
>>>>> each distinct idbucket result.
>>>> FIRST based on which sort?
>>>>
>>>> Lets say you want to return FIRST row based on t3.windowstart column
>>>> and you
3.bucket AS idbucket, t3.activitycount AS
>>>>>> activitycount, t3.windowstart AS starttime, t3.windowend AS endtime FROM
>>>>>> (...) t3
>>>>> Do you have primary key in your t3 table?
>>>>>
>>>>>> In Postgresql
.) t3
>>>> Do you have primary key in your t3 table?
>>>>
>>>>> In Postgresql, what this does is to return the FIRST entire row matching
>>>>> each distinct idbucket result.
>>>> FIRST based on which sort?
>>>>
>>>&
ts say you want to return FIRST row based on t3.windowstart column
>>> and you have primary key in t3 table. Then I believe your query can be
>>> rewritten in the following ways:
>>>
>>> 1. Using subqueries
>>> SELECT
>>> bucket, primary_key, windowstart
art column
>> and you have primary key in t3 table. Then I believe your query can be
>> rewritten in the following ways:
>>
>> 1. Using subqueries
>> SELECT
>> bucket, primary_key, windowstart, etc
>> FROM
>> table AS t1
>> WHERE
&
ucket, primary_key, windowstart, etc
> FROM
> table AS t1
> WHERE
> windowstart=( SELECT max(windowstart) FROM table AS t2 WHERE
> bucket = t1.bucket )
>
> 2. Using joins instead of subqueries ( in case Derby doesn't support
> subqueries - not sure about that )
> SE
start, etc
FROM
table AS t1
WHERE
windowstart=( SELECT max(windowstart) FROM table AS t2 WHERE
bucket = t1.bucket )
2. Using joins instead of subqueries ( in case Derby doesn't support
subqueries - not sure about that )
SELECT
t1.bucket, t1.primary_key, windowstart, etc
FROM
table
this issue. At least the reports don't fail
with an exception now, but they also list all time intervals on Derby instead
of collapsing and reporting just the maximum, which will make these reports far
less useful. r998635.
> Max activity and Max bandwidth reports fail under Derby
Max activity and Max bandwidth reports fail under Derby with a stack trace
--
Key: CONNECTORS-110
URL: https://issues.apache.org/jira/browse/CONNECTORS-110
Project: Apache
nges necessary to use the DERBY-4066 fix properly when
it becomes available. r998576.
> Queue status report fails under Derby
> -
>
> Key: CONNECTORS-109
> URL: https://issues.apache.org/jira/browse/CONNECTORS-109
&g
code changes to correct this problem locally, but
can't commit them yet because Derby's functions are limited in the current
release to not allow CLOB arguments. This issue is going to be addressed in
the next release of Derby, see DERBY-4066. The alternative is to build a trunk
version of
The Derby table-result function syntax requires all output columns to
be declared as part of the function definition, and more importantly
it does not seem to allow calls into Derby itself to get results. So
this would not seem to be a viable option for that reason.
Back to square 1, I guess
For what it's worth, defining a Derby function seems like the only way
to do it. These seem to call arbitrary java that can accept a query
as an argument and return a resultset as the result. But in order to
write such a thing I will need the ability to call Derby at a java
level, I
Hi Folks,
For two of the report queries, ACF uses the following Postgresql
construct, which sadly seems to have no Derby equivalent:
SELECT DISTINCT ON (idbucket) t3.bucket AS idbucket, t3.activitycount
AS activitycount, t3.windowstart AS starttime, t3.windowend AS endtime
FROM (...) t3
In
imum activity report, maximum bandwidth report, and
result code report as well.
> Queue status report fails under Derby
> -
>
> Key: CONNECTORS-109
> URL: https://issues.apache.org/jira/browse/CONNECTORS-109
>
Queue status report fails under Derby
-
Key: CONNECTORS-109
URL: https://issues.apache.org/jira/browse/CONNECTORS-109
Project: Apache Connectors Framework
Issue Type: Bug
Components: Framework
This actually did work, oddly enough. I wonder how Derby is undoing the
read-only attribute on those directories? But in any case, I'm revamping the
core setup/shutdown code again so that there's a decent hook in place to do the
derby shutdown.
Karl
-Original Message-
On 6/8/10 6:35 AM, karl.wri...@nokia.com wrote:
I've been trying to get some basic tests working under Junit. Unfortunately,
I've run into a Derby problem which prevents these tests from working.
What happens is this. Derby, when it creates a database, forces a number of directori
t Karl (Nokia-S/Cambridge)
Sent: Wednesday, June 09, 2010 5:24 AM
To: connectors-dev@incubator.apache.org
Subject: RE: Derby/JUnit bad interaction - any ideas?
Open jdk does not seem to work properly with most java applications at this
time, although it has continued to improve. Its switch in
Olivier Bourgeat [olivier.bourg...@polyspot.com]
Sent: Wednesday, June 09, 2010 4:03 AM
To: connectors-dev@incubator.apache.org
Subject: RE: Derby/JUnit bad interaction - any ideas?
Debian Lenny have openjdk-6:
http://packages.debian.org/fr/source/lenny/openjdk-6
Olivier
Le mardi 08 juin 2010 à 22:37
;
> Karl
>
>
> -Original Message-
> From: ext Jack Krupansky [mailto:jack.krupan...@lucidimagination.com]
> Sent: Tuesday, June 08, 2010 4:36 PM
> To: connectors-dev@incubator.apache.org
> Subject: Re: Derby/JUnit bad interaction - any ideas?
>
> If we need to require Java 1.6
: Derby/JUnit bad interaction - any ideas?
If we need to require Java 1.6, that is probably okay. I am fine with that.
Does anybody have a serious objection to requiring Java 1.6 for LCF?
-- Jack Krupansky
--
From:
Sent: Tuesday, June 08, 2010 6:35
If we need to require Java 1.6, that is probably okay. I am fine with that.
Does anybody have a serious objection to requiring Java 1.6 for LCF?
-- Jack Krupansky
--
From:
Sent: Tuesday, June 08, 2010 6:35 AM
To:
Subject: Derby/JUnit bad
a strange mess.
Still, things are currently working, so I guess I'll leave them as they are,
for now.
Karl
-Original Message-
From: ext Koji Sekiguchi [mailto:k...@r.email.ne.jp]
Sent: Tuesday, June 08, 2010 10:30 AM
To: connectors-dev@incubator.apache.org
Subject: Re: Derby/J
(10/06/08 23:14), karl.wri...@nokia.com wrote:
Yeah, I was pretty surprised too. But on windows it is likely that
File.makeReadOnly() (which is what Derby must be using) doesn't actually do
anything to directories, which would explain the discrepancy.
Karl
If so, luckily Ant hac
Yeah, I was pretty surprised too. But on windows it is likely that
File.makeReadOnly() (which is what Derby must be using) doesn't actually do
anything to directories, which would explain the discrepancy.
Karl
-Original Message-
From: ext Mark Miller [mailto:markrmil...@gmai
, 2010 10:08 AM
To: connectors-dev@incubator.apache.org
Subject: Re: Derby/JUnit bad interaction - any ideas?
(10/06/08 22:35), karl.wri...@nokia.com wrote:
> I've been trying to get some basic tests working under Junit. Unfortunately,
> I've run into a Derby problem which prevents
(10/06/08 22:35), karl.wri...@nokia.com wrote:
I've been trying to get some basic tests working under Junit. Unfortunately,
I've run into a Derby problem which prevents these tests from working.
What happens is this. Derby, when it creates a database, forces a number of directori
On 6/8/10 6:35 AM, karl.wri...@nokia.com wrote:
I've been trying to get some basic tests working under Junit. Unfortunately,
I've run into a Derby problem which prevents these tests from working.
What happens is this. Derby, when it creates a database, forces a number of directori
I've been trying to get some basic tests working under Junit. Unfortunately,
I've run into a Derby problem which prevents these tests from working.
What happens is this. Derby, when it creates a database, forces a number of
directories within the database to "read-only&q
Yup.
Karl
-Original Message-
From: ext Jack Krupansky [mailto:jack.krupan...@lucidimagination.com]
Sent: Friday, June 04, 2010 12:27 AM
To: connectors-dev@incubator.apache.org
Subject: Re: Derby
Just to be clear, the full sequence would be:
1) Start UI app. Agent process should not be
The reason this occurs is because I am using Derby in embedded mode, and the
restriction appears to be a limitation of that mode of operation. However,
this mode is necessary to meet the testing goal, which was the prime motivator
behind doing a Derby implementation. I am sure that if we were
--
From:
Sent: Thursday, June 03, 2010 5:34 PM
To:
Subject: Derby
For what it's worth, after some 5 days of work, and a couple of schema
changes to boot, LCF now runs with Derby.
Some caveats:
(1) You can't run more than one LCF process at a time. That means you
need to
e idle.
6) Possibly commit to Solr.
7) AgentStop.
8) Back to step 1 for additional jobs.
Correct?
-- Jack Krupansky
--
From:
Sent: Thursday, June 03, 2010 7:24 PM
To:
Subject: RE: Derby
The daemon does not need to interact with the UI directly, onl
: Thursday, June 03, 2010 5:51 PM
To: connectors-dev@incubator.apache.org
Subject: Re: Derby
> (1) You can't run more than one LCF process at a time. That means you
> need to either run the daemon or the crawler-ui web application, but you
> can't run both at the same time.
H
awling?
Thanks for all of this effort!
-- Jack Krupansky
--
From:
Sent: Thursday, June 03, 2010 5:34 PM
To:
Subject: Derby
For what it's worth, after some 5 days of work, and a couple of schema
changes to boot, LCF now runs with Derby.
S
For what it's worth, after some 5 days of work, and a couple of schema changes
to boot, LCF now runs with Derby.
Some caveats:
(1) You can't run more than one LCF process at a time. That means you need
to either run the daemon or the crawler-ui web application, but you can'
98 matches
Mail list logo
