[
https://issues.apache.org/jira/browse/CAMEL-7525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Knut-Håvard Aksnes updated CAMEL-7525:
--------------------------------------
Description:
We do use the file component to read files from a particular area. We pick up
new files from there using their content as basis for a database import. There
are a couple of important points:
# We must not add or delete files to the area we read from, not even
temporarily.
# We need to pick up files once.
To achive this we use noop=true as well as a file based idempotentRepository
where the repository is placed in a different location. We also use
readLock=changed which is the big problem. To cite the component documentation:
Notice from Camel 2.10 onwards the read locks changed, fileLock and rename will
also use a markerFile as well, to ensure not picking up files that may be in
process by another Camel consumer running on another node (eg cluster).
This is a huge problem as I can't find any documented way of storing the marker
files in a different location. What is really needed is something similar to
inProgressRepository as an option to keep track of the readLock related
information. Alternatively an option to disable the marker files could be
created, this option could however lead to failure if applications are
clustered.
This problem is for us serious breakage, as there are other applications not
under our control reading the same areas, some of these crashes due to the
marker files.
was:
We do use the file component to read files from a particular area. We pick up
new files from there using their content as basis for a database import. There
are a couple of important points:
# We must not add or delete files to the area we read from, not even
temporarily.
# We need to pick up files once.
To achive this we use noop=true as well as a file based idempotentRepository
where the repository is placed in a different location. We also use
readLock=changed which is the big problem. To cite the component documentation:
Notice from Camel 2.10 onwards the read locks changed, fileLock and rename will
also use a markerFile as well, to ensure not picking up files that may be in
process by another Camel consumer running on another node (eg cluster).
This is a huge problem as I can't find any documented way of storing the marker
files in a different location. What is really needed is something similar to
inProgressRepository as an option to keep track of the readLock related
information.
This problem is for us serious breakage, as there are other applications not
under our control reading the same areas, some of these crashes due to the
marker files.
Alternatively an option to disable the marker files, this option could however
lead to failure if applications are clustered.
> Behavior change for file component in 2.10 causes problems with no workaround
> available
> ---------------------------------------------------------------------------------------
>
> Key: CAMEL-7525
> URL: https://issues.apache.org/jira/browse/CAMEL-7525
> Project: Camel
> Issue Type: Bug
> Reporter: Knut-Håvard Aksnes
>
> We do use the file component to read files from a particular area. We pick
> up new files from there using their content as basis for a database import.
> There are a couple of important points:
> # We must not add or delete files to the area we read from, not even
> temporarily.
> # We need to pick up files once.
> To achive this we use noop=true as well as a file based idempotentRepository
> where the repository is placed in a different location. We also use
> readLock=changed which is the big problem. To cite the component
> documentation:
> Notice from Camel 2.10 onwards the read locks changed, fileLock and rename
> will also use a markerFile as well, to ensure not picking up files that may
> be in process by another Camel consumer running on another node (eg cluster).
> This is a huge problem as I can't find any documented way of storing the
> marker files in a different location. What is really needed is something
> similar to inProgressRepository as an option to keep track of the readLock
> related information. Alternatively an option to disable the marker files
> could be created, this option could however lead to failure if applications
> are clustered.
> This problem is for us serious breakage, as there are other applications not
> under our control reading the same areas, some of these crashes due to the
> marker files.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
