Re: How to process files in a sorted order

2013-11-19 Thread Chris Mattmann
Mavrommatis kmavromma...@celgene.com Reply-To: dev@oodt.apache.org Date: Thursday, November 7, 2013 8:44 PM To: dev@oodt.apache.org dev@oodt.apache.org Subject: How to process files in a sorted order Hi, In my environment I am using cas-crawler to process directories of 1000s of files

Re: How to process files in a sorted order

2013-11-15 Thread Konstantinos Mavrommatis
Thanks Cameron, I will try to play with it and I will post a solution if I find one. Konstantinos On 11/13/13 3:18 PM, Cameron Goodale sigep...@gmail.com wrote: Konstantinos, My name is Cameron and I am a committer on the Apache OODT project. I am not familiar with the internals of crawler,

Re: How to process files in a sorted order

2013-11-13 Thread Cameron Goodale
Konstantinos, My name is Cameron and I am a committer on the Apache OODT project. I am not familiar with the internals of crawler, but I don't believe there is a way to accomplish your goal of enforcing a sorting algorithm within the crawler config. I think you will have to write your own

How to process files in a sorted order

2013-11-07 Thread Konstantinos Mavrommatis
Hi, In my environment I am using cas-crawler to process directories of 1000s of files. The metadata for these files are extracted automatically using the mimetypes definitions and small wrapper scripts. In these directories some of the files are derived from other files and metadata from the