Re: Camel Main: RoutesIncludePattern doesn't seem to respect asterisks as a directory placeholder
Hi, the API seems to have changed from the 3.20.4 that I'm using.
Can I just replace the File Filter that the default Route Reloader is
using? I've noticed it uses the Ant Matcher but deliberately ignores the
whole path:
https://github.com/apache/camel/blob/af9e3b4b9559ef18abf7eaba382a2991998d83da/core/camel-support/src/main/java/org/apache/camel/support/RouteWatcherReloadStrategy.java#L128
I would suggest to change this one line so it starts to work a little
more as expected.
//path = FileUtil.stripPath(path);
path = path.substring(base.length()+1); //
TODO: consider using nio instead of strings
Turned out it wasn't easy to replace the File Filter only, I came up
with the following workaround. Do you think it's legit and will survive
a couple of the next Camel upgrades?
public static void main(String[] args) throws Exception {
Main main = new Main();
// could come from the properties file
main.configure().withRoutesIncludePattern("file:deploy/**/PRIVATE/EXCHANGE/*.yaml");
main.configure().withRoutesReloadEnabled(true);
main.configure().withRoutesReloadDirectory("deploy");
main.configure().withRoutesReloadDirectoryRecursive(true);
main.configure().withRoutesReloadPattern("**/PRIVATE/EXCHANGE/*.yaml");
main.configure().withRoutesReloadRemoveAllRoutes(true);
if (main.configure().isRoutesReloadEnabled()) { // in case
we're loading from a properties file
//copied from the DefaultConfigurationConfigurer
RouteWatcherReloadStrategy reloader = new
RouteWatcherReloadStrategy(
main.configure().getRoutesReloadDirectory(),
main.configure().isRoutesReloadDirectoryRecursive());
reloader.setPattern(main.configure().getRoutesReloadPattern());
reloader.setRemoveAllRoutes(main.configure().isRoutesReloadRemoveAllRoutes());
// clear so that the DefaultConfigurationConfigurer doesn't
mess everything up
main.configure().withRoutesReloadEnabled(false);
// replace with the better file filter
//
// copied and modified from the RouteWatcherReloadStrategy
final String[] parts = reloader.getPattern().split(",");
final String base = new
File(reloader.getFolder()).getAbsolutePath();
final AntPathMatcher matcher = new AntPathMatcher();
reloader.setFileFilter(new FileFilter() {
@Override
public boolean accept(File f) {
for (String part : parts) {
// strip starting directory, so we have a
relative name to the starting folder
String path = f.getAbsolutePath();
if (path.startsWith(base)) {
//path = FileUtil.stripPath(path); - why
the whole path? lets strip the base only
path = path.substring(base.length() + 1);
// TODO: consider using nio instead of strings
}
String name = FileUtil.compactPath(f.getPath());
boolean exact = name.equals(part);
boolean result = exact || matcher.match(part,
path, false);
//LOG.trace("Accepting file pattern:{} path:{}
-> {}", part, path, result);
if (result) {
return true;
}
}
return false;
}
});
// add the reloader with the better file filter
main.configure().addConfiguration(new CamelConfiguration() {
@Override
public void configure(CamelContext camelContext) throws
Exception {
camelContext.addService(reloader);
}
});
}
main.run();
}
On 18.05.2023 11:03, Claus Ibsen wrote:
Hi
You can implement a custom org.apache.camel.spi.ResourceReload where you in
the onReload can program what to do.
The current defauly just do
if (name.endsWith(".properties")) {
onPropertiesReload(resource, true);
} else {
onRouteReload(List.of(resource), false);
}
And then create an instance of FileWatcherResourceReloadStrategy which you
can configure with folder / pattern etc, and then add this as a bean to
camel context.
On Thu, May 18, 2023 at 9:39 AM Fyodor Kravchenko
wrote:
Hi,
Thank you! I'm new to the Camel 3 API, can you please point me how can I
load a route programmatically if I have a yaml or xml String (or
Stream/Reader)?
Regarding the ticket, loading the new files after the startup, renaming
the route files and deleting them would also be a useful development
feature. Currently it works independently from the initial
Re: Camel Main: RoutesIncludePattern doesn't seem to respect asterisks as a directory placeholder
Hi
You can implement a custom org.apache.camel.spi.ResourceReload where you in
the onReload can program what to do.
The current defauly just do
if (name.endsWith(".properties")) {
onPropertiesReload(resource, true);
} else {
onRouteReload(List.of(resource), false);
}
And then create an instance of FileWatcherResourceReloadStrategy which you
can configure with folder / pattern etc, and then add this as a bean to
camel context.
On Thu, May 18, 2023 at 9:39 AM Fyodor Kravchenko
wrote:
> Hi,
>
> Thank you! I'm new to the Camel 3 API, can you please point me how can I
> load a route programmatically if I have a yaml or xml String (or
> Stream/Reader)?
>
> Regarding the ticket, loading the new files after the startup, renaming
> the route files and deleting them would also be a useful development
> feature. Currently it works independently from the initial load, so it
> does re-load all of the routes it encounters under the base directory,
> but the pattern it uses to choose which files to (re/un)load follows the
> different syntax and even the directory base is different.
>
> A jira ticket that I would file would require a developer-friendly
> "fileReload"-true-false configuration parameter, that would try to use
> the same pattern that the "include" configuration uses, if it is a
> "file" one. I can imagine it is not a trivial development for a rarely
> used development feature and is likely to be marked as "will not
> implement" so let's leave it as it is.
>
> On 18.05.2023 09:06, Claus Ibsen wrote:
> > On Wed, May 17, 2023 at 10:59 PM Fyodor Kravchenko <
> feddkr...@hotmail.com>
> > wrote:
> >
> >> Hi,
> >>
> >> I'm integrating Camel into my application, or better say, application
> >> platform, that may run different arbitrary Camel routes for integration,
> >> and I'd like to provide an ability for a developer who makes
> >> applications for this platform to edit the routes during the
> >> development. I'm already monitoring the filesystem for other artifact
> >> edits, so If there is an API to reload an individual route taken from a
> >> file I might use that.
> >>
> >> I've considered to use a separate Camel standalone or as a containerized
> >> service that I've noticed the project is encouraging now, but this would
> >> require me to develop some protocol to interact with my platform during
> >> its lifecycle. For Camel 2 I managed to instantiate xml snippets to
> >> custom Processors but the API has changed and this Main load-reload
> >> function seemed to do what I want for Camel 3.
> >>
> >>
> > Okay so for making the reload respect only already loaded files, then we
> > can match the "changed file event" form the JDK
> > and match if it was previously loaded or not. And then skip it if not.
> >
> > Then users can turn this on via some new option (maybe name it only
> > existing files, or some better name? ).
> > withRoutesReloadOnlyExistingFiles(true)
> >
> >
> > You are welcome to create a JIRA ticket for that.
> >
> >
> >
> >> On 17.05.2023 22:58, Claus Ibsen wrote:
> >>> Hi
> >>>
> >>> What is your goal with this?
> >>>
> >>> The reload stuff is for development and not a production app server to
> >>> "redeploy apps" or any sort of that.
> >>> The intention is that you work on a single application.
> >>>
> >>> The reload is using Java APIs for "change file events" and this does
> not
> >>> support an ANT style way and whatnot.
> >>>
> >>>
> >>> On Wed, May 17, 2023 at 8:45 PM Fyodor Kravchenko <
> feddkr...@hotmail.com
> >>>
> >>> wrote:
> >>>
> I'm sorry please disregard my previous ramblings, moving the "deploy"
> directory to the working directory of the java app did the "include"
> trick. It worked without the "file:" prefix because it was looking
> into
> the "target" directry which was the classpath.. My fault.
>
> However, the additional question remains: how do I make it _reload_
> what
> it loaded previosly, without catching the extra *.yaml files that may
> be
> located in the wrong directories?
>
> I'm trying this,
>
> Main main = new Main();
>
>
> >>
> main.configure().withRoutesIncludePattern("file:deploy/**/PRIVATE/EXCHANGE/*.yaml");
> main.configure().withRoutesReloadEnabled(true);
> main.configure().withRoutesReloadDirectory("deploy");
> main.configure().withRoutesReloadDirectoryRecursive(true);
> //
> >> main.configure().withRoutesReloadPattern("**/PRIVATE/EXCHANGE/*.yaml");
> //
> main.configure().withRoutesReloadPattern("PRIVATE/EXCHANGE/*.yaml");
> main.configure().withRoutesReloadPattern("*.yaml");
> main.configure().withRoutesReloadRemoveAllRoutes(true);
> main.run();
>
> but this will catch and load any .yaml file under the "deploy"
> directory, event those that wasn't loaded at
Re: Camel Main: RoutesIncludePattern doesn't seem to respect asterisks as a directory placeholder
Hi,
Thank you! I'm new to the Camel 3 API, can you please point me how can I
load a route programmatically if I have a yaml or xml String (or
Stream/Reader)?
Regarding the ticket, loading the new files after the startup, renaming
the route files and deleting them would also be a useful development
feature. Currently it works independently from the initial load, so it
does re-load all of the routes it encounters under the base directory,
but the pattern it uses to choose which files to (re/un)load follows the
different syntax and even the directory base is different.
A jira ticket that I would file would require a developer-friendly
"fileReload"-true-false configuration parameter, that would try to use
the same pattern that the "include" configuration uses, if it is a
"file" one. I can imagine it is not a trivial development for a rarely
used development feature and is likely to be marked as "will not
implement" so let's leave it as it is.
On 18.05.2023 09:06, Claus Ibsen wrote:
On Wed, May 17, 2023 at 10:59 PM Fyodor Kravchenko
wrote:
Hi,
I'm integrating Camel into my application, or better say, application
platform, that may run different arbitrary Camel routes for integration,
and I'd like to provide an ability for a developer who makes
applications for this platform to edit the routes during the
development. I'm already monitoring the filesystem for other artifact
edits, so If there is an API to reload an individual route taken from a
file I might use that.
I've considered to use a separate Camel standalone or as a containerized
service that I've noticed the project is encouraging now, but this would
require me to develop some protocol to interact with my platform during
its lifecycle. For Camel 2 I managed to instantiate xml snippets to
custom Processors but the API has changed and this Main load-reload
function seemed to do what I want for Camel 3.
Okay so for making the reload respect only already loaded files, then we
can match the "changed file event" form the JDK
and match if it was previously loaded or not. And then skip it if not.
Then users can turn this on via some new option (maybe name it only
existing files, or some better name? ).
withRoutesReloadOnlyExistingFiles(true)
You are welcome to create a JIRA ticket for that.
On 17.05.2023 22:58, Claus Ibsen wrote:
Hi
What is your goal with this?
The reload stuff is for development and not a production app server to
"redeploy apps" or any sort of that.
The intention is that you work on a single application.
The reload is using Java APIs for "change file events" and this does not
support an ANT style way and whatnot.
On Wed, May 17, 2023 at 8:45 PM Fyodor Kravchenko
I'm sorry please disregard my previous ramblings, moving the "deploy"
directory to the working directory of the java app did the "include"
trick. It worked without the "file:" prefix because it was looking into
the "target" directry which was the classpath.. My fault.
However, the additional question remains: how do I make it _reload_ what
it loaded previosly, without catching the extra *.yaml files that may be
located in the wrong directories?
I'm trying this,
Main main = new Main();
main.configure().withRoutesIncludePattern("file:deploy/**/PRIVATE/EXCHANGE/*.yaml");
main.configure().withRoutesReloadEnabled(true);
main.configure().withRoutesReloadDirectory("deploy");
main.configure().withRoutesReloadDirectoryRecursive(true);
//
main.configure().withRoutesReloadPattern("**/PRIVATE/EXCHANGE/*.yaml");
// main.configure().withRoutesReloadPattern("PRIVATE/EXCHANGE/*.yaml");
main.configure().withRoutesReloadPattern("*.yaml");
main.configure().withRoutesReloadRemoveAllRoutes(true);
main.run();
but this will catch and load any .yaml file under the "deploy"
directory, event those that wasn't loaded at startup, when I touch any
of the yamls.
Any tips on reloading patterns?
Thank you!
On 17.05.2023 21:06, Fyodor Kravchenko wrote:
Hi, thanks but this
.
Netbeans uses the following command to run the program:
cd /home/fedd/NetBeansProjects/camelmaintry;
JAVA_HOME=/home/fedd/Programs/graalvm-ce-java19-22.3.0
/snap/netbeans/76/netbeans/java/maven/bin/mvn -Dexec.vmArgs=
"-Dexec.args=${exec.vmArgs} -classpath %classpath ${exec.mainClass}
${exec.appArgs}" -Dexec.appArgs=
-Dexec.mainClass=camelmaintry.CamelMainTry
-Dexec.executable=/home/fedd/Programs/graalvm-ce-java19-22.3.0/bin/java
org.codehaus.mojo:exec-maven-plugin:3.1.0:exec
(nothing special in the "exec.mainClass" and other placeholders there)
I can't grasp how to make it look into this pattern:
deploy/**/PRIVATE/EXCHANGE/*.yaml
Thanks!
--fedd
On 17.05.2023 20:15, Claus Ibsen wrote:
Hi
Okay for file system, you should favour prefixing with file:
main.configure().withRoutesIncludePattern("file:deploy/customer/PRIVATE/EXCHANGE/*.yaml");
On Wed, May 17, 2023 at 3:53 PM Fyodor
Re: Camel Main: RoutesIncludePattern doesn't seem to respect asterisks as a directory placeholder
On Wed, May 17, 2023 at 10:59 PM Fyodor Kravchenko
wrote:
> Hi,
>
> I'm integrating Camel into my application, or better say, application
> platform, that may run different arbitrary Camel routes for integration,
> and I'd like to provide an ability for a developer who makes
> applications for this platform to edit the routes during the
> development. I'm already monitoring the filesystem for other artifact
> edits, so If there is an API to reload an individual route taken from a
> file I might use that.
>
> I've considered to use a separate Camel standalone or as a containerized
> service that I've noticed the project is encouraging now, but this would
> require me to develop some protocol to interact with my platform during
> its lifecycle. For Camel 2 I managed to instantiate xml snippets to
> custom Processors but the API has changed and this Main load-reload
> function seemed to do what I want for Camel 3.
>
>
Okay so for making the reload respect only already loaded files, then we
can match the "changed file event" form the JDK
and match if it was previously loaded or not. And then skip it if not.
Then users can turn this on via some new option (maybe name it only
existing files, or some better name? ).
withRoutesReloadOnlyExistingFiles(true)
You are welcome to create a JIRA ticket for that.
> On 17.05.2023 22:58, Claus Ibsen wrote:
> > Hi
> >
> > What is your goal with this?
> >
> > The reload stuff is for development and not a production app server to
> > "redeploy apps" or any sort of that.
> > The intention is that you work on a single application.
> >
> > The reload is using Java APIs for "change file events" and this does not
> > support an ANT style way and whatnot.
> >
> >
> > On Wed, May 17, 2023 at 8:45 PM Fyodor Kravchenko >
> > wrote:
> >
> >> I'm sorry please disregard my previous ramblings, moving the "deploy"
> >> directory to the working directory of the java app did the "include"
> >> trick. It worked without the "file:" prefix because it was looking into
> >> the "target" directry which was the classpath.. My fault.
> >>
> >> However, the additional question remains: how do I make it _reload_ what
> >> it loaded previosly, without catching the extra *.yaml files that may be
> >> located in the wrong directories?
> >>
> >> I'm trying this,
> >>
> >> Main main = new Main();
> >>
> >>
> main.configure().withRoutesIncludePattern("file:deploy/**/PRIVATE/EXCHANGE/*.yaml");
> >> main.configure().withRoutesReloadEnabled(true);
> >> main.configure().withRoutesReloadDirectory("deploy");
> >> main.configure().withRoutesReloadDirectoryRecursive(true);
> >> //
> main.configure().withRoutesReloadPattern("**/PRIVATE/EXCHANGE/*.yaml");
> >> // main.configure().withRoutesReloadPattern("PRIVATE/EXCHANGE/*.yaml");
> >> main.configure().withRoutesReloadPattern("*.yaml");
> >> main.configure().withRoutesReloadRemoveAllRoutes(true);
> >> main.run();
> >>
> >> but this will catch and load any .yaml file under the "deploy"
> >> directory, event those that wasn't loaded at startup, when I touch any
> >> of the yamls.
> >>
> >> Any tips on reloading patterns?
> >>
> >> Thank you!
> >>
> >> On 17.05.2023 21:06, Fyodor Kravchenko wrote:
> >>> Hi, thanks but this
> >>>
> >>> .
> >>> Netbeans uses the following command to run the program:
> >>>
> >>> cd /home/fedd/NetBeansProjects/camelmaintry;
> >>> JAVA_HOME=/home/fedd/Programs/graalvm-ce-java19-22.3.0
> >>> /snap/netbeans/76/netbeans/java/maven/bin/mvn -Dexec.vmArgs=
> >>> "-Dexec.args=${exec.vmArgs} -classpath %classpath ${exec.mainClass}
> >>> ${exec.appArgs}" -Dexec.appArgs=
> >>> -Dexec.mainClass=camelmaintry.CamelMainTry
> >>> -Dexec.executable=/home/fedd/Programs/graalvm-ce-java19-22.3.0/bin/java
> >>> org.codehaus.mojo:exec-maven-plugin:3.1.0:exec
> >>>
> >>> (nothing special in the "exec.mainClass" and other placeholders there)
> >>>
> >>> I can't grasp how to make it look into this pattern:
> >>>
> >>> deploy/**/PRIVATE/EXCHANGE/*.yaml
> >>>
> >>> Thanks!
> >>> --fedd
> >>>
> >>> On 17.05.2023 20:15, Claus Ibsen wrote:
> Hi
>
> Okay for file system, you should favour prefixing with file:
>
>
> >>
> main.configure().withRoutesIncludePattern("file:deploy/customer/PRIVATE/EXCHANGE/*.yaml");
> >>
>
>
> On Wed, May 17, 2023 at 3:53 PM Fyodor Kravchenko
>
> wrote:
>
> > Hi,
> >
> > I'm trying to load the routes from the file system directly, it's
> Camel
> > 3.20.4, and it's java 19 from the GraalVM distribution running on
> > Ubuntu
> > 22.04. The fact it sees the file when no wildcard is present tells me
> > that I'm missing something in the wildcards and the docs.
> >
> > Thank you!
> >
> > --fedd
> >
> > (sorry for the late response, only now I've got the rejection
> > notification for the html-formatted email)
> >
> > On
