Hi Erlend, This is actually the result of a bug fix that was made in the 1.7 time frame. The problem actually arose in the first place in 1.5, when a lock needed to go from being a critical section to a cross-process lock. But, due to an oversight, this was only fixed now.
We can lock based on the uri's hash value; that's the only viable solution. File locking, though, will be removed in MCF 2.0 entirely, in favor of Zookeeper. I'll open a ticket. Karl On Fri, Aug 15, 2014 at 4:11 AM, Erlend Garåsen <[email protected]> wrote: > > Another thing. It's not possible to abort the job due to this problem. > LockManager still tries to set locks over and over again. It's not just the > previous URL/filename I entered, but several others: > > WARN 2014-08-15 10:07:46,178 (Worker thread '31') - Attempt to set file > lock '/www/var/data/mcf/mcf-1/conf/../data/synchdir/664/756/lock- > Solr&58!https&58!&47!&47!www.journals.uio.no&58!443&47! > index.php&47!nordina&47!search&47!advancedResults? > subject&61!Small%20group%20learning%2C%203rd%20graders% > 2C%20learning%20of%20DC-circuit%20phenomena%2C% > 20active%20and%20spontaneous%20learning.lock' failed: File name too long > > Erlend > > > On 15.08.14 09:46, Erlend Garåsen wrote: > >> >> -1 >> >> All my first tests pass, but I think I found a blocker when I ran the >> last one. >> >> By running MCF using FileLockManager, I'm getting the following error >> and MCF just tries to run this task over and over again. My synch folder >> now contains a lot of files and it still grows. I think MCF should >> handle long URLs and just strip the length of the filename if it becomes >> too large. >> >> INFO 2014-08-15 09:30:54,485 (Worker thread '9') - WEB: FETCH >> URL|https://www.journals.uio.no/index.php/nordina/search/ >> advancedResults?subject=effective%20continuing% >> 20professional%20development%2C%20authentic%20and% >> 20entrepreneurial%20learning%2C%20science%20and%20technology%20education| >> 1408087853848+633|200|15735| >> <https://www.journals.uio.no/index.php/nordina/search/advancedResults?subject=effective%20continuing%20professional%20development%2C%20authentic%20and%20entrepreneurial%20learning%2C%20science%20and%20technology%20education%7C1408087853848+633%7C200%7C15735%7C> >> >> WARN 2014-08-15 09:30:54,609 (Worker thread '9') - Attempt to set file >> lock >> '/www/var/data/mcf/mcf-1/conf/../data/synchdir/948/350/lock- >> Solr&58!https&58!&47!&47!www.journals.uio.no&58!443&47! >> index.php&47!nordina&47!search&47!advancedResults?subject&61!effective% >> 20continuing%20professional%20development%2C%20authentic% >> 20and%20entrepreneurial%20learning%2C%20science%20and% >> 20technology%20education.lock' >> failed: File name too long >> java.io.IOException: File name too long >> at java.io.UnixFileSystem.createFileExclusively(Native Method) >> at java.io.File.createNewFile(File.java:947) >> at >> org.apache.manifoldcf.core.lockmanager.FileLockObject. >> grabFileLock(FileLockObject.java:221) >> >> at >> org.apache.manifoldcf.core.lockmanager.FileLockObject. >> obtainGlobalWriteLockNoWait(FileLockObject.java:77) >> >> at >> org.apache.manifoldcf.core.lockmanager.LockObject.obtainGlobalWriteLock( >> LockObject.java:121) >> >> at >> org.apache.manifoldcf.core.lockmanager.LockObject. >> enterWriteLock(LockObject.java:74) >> >> at >> org.apache.manifoldcf.core.lockmanager.LockGate. >> enterWriteLock(LockGate.java:177) >> >> at >> org.apache.manifoldcf.core.lockmanager.BaseLockManager. >> enter(BaseLockManager.java:1473) >> >> at >> org.apache.manifoldcf.core.lockmanager.BaseLockManager. >> enterLocks(BaseLockManager.java:803) >> >> at >> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$ >> OutputAddEntryPoint.addOrReplaceDocumentWithExcept >> ion(IncrementalIngester.java:3329) >> >> at >> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$ >> PipelineAddFanout.sendDocument(IncrementalIngester.java:3051) >> >> >> >> >> On 12.08.14 05:13, Mingchun Zhao wrote: >> >>> Hi all, >>> >>> Please vote on whether to release the ManifoldCF, version 1.7, RC0. >>> >>> You can find the artifact at: >>> >>> http://people.apache.org/~mingchun/apache-manifoldcf-1.7-RC0 >>> >>> There is also a tag at: >>> >>> https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.7-RC0 >>> >>> Vote will remain open at least 72 hours. >>> >>> Thanks! >>> Mingchun Zhao >>> >>> >> >
