Re: [HEADS UP] - More flexible route configuration in all DSLs
On Tue, Aug 3, 2021 at 7:58 PM Zoran Regvart wrote: > > Hi Claus, > this is really useful. I think a blog post taking bits from the > documentation and showcasing one of the examples would help spread the > word about this. > > (not saying that you need to write the blog post, perhaps someone else > could volunteer) > Yeah I also think it would be good to blog about this upcoming feature. Then also we can link back to it when we do the whats new in 3.12 announcement blog. > zoran > > On Tue, Aug 3, 2021 at 5:51 PM Claus Ibsen wrote: > > > > Hi > > > > The work has been merged, and the website has been updated > > https://camel.apache.org/manual/latest/route-configuration.html > > > > > > On Tue, Aug 3, 2021 at 2:49 PM Claus Ibsen wrote: > > > > > > Hi > > > > > > Back from PTO then I continued this work > > > https://issues.apache.org/jira/browse/CAMEL-16757 > > > > > > I am so far that I am working on a giant PR to be ready for merging. > > > Review is welcome but it's big work so it's not so easy to understand > > > just by browsing a lot of code changes. > > > In the PR there is documentation of this new feature which is a > > > starting point to see what this fuss is all about. > > > > > > There are also two examples (standalone and spring boot) that are in > > > the github examples repo with the name routes-configuration. > > > https://github.com/apache/camel-spring-boot-examples/tree/main/routes-configuration > > > https://github.com/apache/camel-examples/tree/main/examples/routes-configuration > > > > > > There is more work to come, as I would like to add configuration of > > > errorHandler as well. The errorHandler has always been a special case, > > > as it was only in the DSL for spring xml and blueprint xml, > > > and not in the general DSL. However Java DSL had some special for > > > errorHandler too. There are some other JIRAs about this work. > > > > > > > > > > > > > > > On Thu, Jul 8, 2021 at 2:12 PM Claus Ibsen wrote: > > > > > > > > Hi > > > > > > > > I am working on (branch on git with the ticket number) > > > > https://issues.apache.org/jira/browse/CAMEL-16757 > > > > > > > > Its work in progress, but I got a prototype in a new example > > > > https://github.com/apache/camel-examples/tree/main/examples/routes-configuration > > > > > > > > In this example we have routes in java, xml and yaml in the myroutes > > > > folder. For Java you can of course also have it in src/main/java. > > > > > > > > But the point is to make this work for dynamic loaded routes (also > > > > Java) and hence why its in the myroutes folder. > > > > > > > > These routes causes exceptions, and then we have global configured > > > > error handling in separate source code, in the myerror folder. > > > > > > > > Those are also polyglot and you can define this in XML, YAML or Java > > > > etc. > > > > > > > > There is more to work on but wanted to share the heads-up. > > > > I will be on PTO 2 weeks from tomorrow afternoon. So will work on > > > > finishing this later. > > > > > > > > The potential is also to allow users to define their global error > > > > handler, route configurations, and you can put that into separate JAR > > > > and just add as dependency. And then you can refer to these route > > > > configurations from your routes or to use the global configured etc. > > > > > > > > Feedback is welcome, the work is "not set in stone". > > > > > > > > > > > > > > > > -- > > > > Claus Ibsen > > > > - > > > > http://davsclaus.com @davsclaus > > > > Camel in Action 2: https://www.manning.com/ibsen2 > > > > > > > > > > > > -- > > > Claus Ibsen > > > - > > > http://davsclaus.com @davsclaus > > > Camel in Action 2: https://www.manning.com/ibsen2 > > > > > > > > -- > > Claus Ibsen > > - > > http://davsclaus.com @davsclaus > > Camel in Action 2: https://www.manning.com/ibsen2 > > > > -- > Zoran Regvart -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
Re: [HEADS UP] - More flexible route configuration in all DSLs
Hi Claus, this is really useful. I think a blog post taking bits from the documentation and showcasing one of the examples would help spread the word about this. (not saying that you need to write the blog post, perhaps someone else could volunteer) zoran On Tue, Aug 3, 2021 at 5:51 PM Claus Ibsen wrote: > > Hi > > The work has been merged, and the website has been updated > https://camel.apache.org/manual/latest/route-configuration.html > > > On Tue, Aug 3, 2021 at 2:49 PM Claus Ibsen wrote: > > > > Hi > > > > Back from PTO then I continued this work > > https://issues.apache.org/jira/browse/CAMEL-16757 > > > > I am so far that I am working on a giant PR to be ready for merging. > > Review is welcome but it's big work so it's not so easy to understand > > just by browsing a lot of code changes. > > In the PR there is documentation of this new feature which is a > > starting point to see what this fuss is all about. > > > > There are also two examples (standalone and spring boot) that are in > > the github examples repo with the name routes-configuration. > > https://github.com/apache/camel-spring-boot-examples/tree/main/routes-configuration > > https://github.com/apache/camel-examples/tree/main/examples/routes-configuration > > > > There is more work to come, as I would like to add configuration of > > errorHandler as well. The errorHandler has always been a special case, > > as it was only in the DSL for spring xml and blueprint xml, > > and not in the general DSL. However Java DSL had some special for > > errorHandler too. There are some other JIRAs about this work. > > > > > > > > > > On Thu, Jul 8, 2021 at 2:12 PM Claus Ibsen wrote: > > > > > > Hi > > > > > > I am working on (branch on git with the ticket number) > > > https://issues.apache.org/jira/browse/CAMEL-16757 > > > > > > Its work in progress, but I got a prototype in a new example > > > https://github.com/apache/camel-examples/tree/main/examples/routes-configuration > > > > > > In this example we have routes in java, xml and yaml in the myroutes > > > folder. For Java you can of course also have it in src/main/java. > > > > > > But the point is to make this work for dynamic loaded routes (also > > > Java) and hence why its in the myroutes folder. > > > > > > These routes causes exceptions, and then we have global configured > > > error handling in separate source code, in the myerror folder. > > > > > > Those are also polyglot and you can define this in XML, YAML or Java etc. > > > > > > There is more to work on but wanted to share the heads-up. > > > I will be on PTO 2 weeks from tomorrow afternoon. So will work on > > > finishing this later. > > > > > > The potential is also to allow users to define their global error > > > handler, route configurations, and you can put that into separate JAR > > > and just add as dependency. And then you can refer to these route > > > configurations from your routes or to use the global configured etc. > > > > > > Feedback is welcome, the work is "not set in stone". > > > > > > > > > > > > -- > > > Claus Ibsen > > > - > > > http://davsclaus.com @davsclaus > > > Camel in Action 2: https://www.manning.com/ibsen2 > > > > > > > > -- > > Claus Ibsen > > - > > http://davsclaus.com @davsclaus > > Camel in Action 2: https://www.manning.com/ibsen2 > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2 -- Zoran Regvart
Re: [HEADS UP] - More flexible route configuration in all DSLs
Hi The work has been merged, and the website has been updated https://camel.apache.org/manual/latest/route-configuration.html On Tue, Aug 3, 2021 at 2:49 PM Claus Ibsen wrote: > > Hi > > Back from PTO then I continued this work > https://issues.apache.org/jira/browse/CAMEL-16757 > > I am so far that I am working on a giant PR to be ready for merging. > Review is welcome but it's big work so it's not so easy to understand > just by browsing a lot of code changes. > In the PR there is documentation of this new feature which is a > starting point to see what this fuss is all about. > > There are also two examples (standalone and spring boot) that are in > the github examples repo with the name routes-configuration. > https://github.com/apache/camel-spring-boot-examples/tree/main/routes-configuration > https://github.com/apache/camel-examples/tree/main/examples/routes-configuration > > There is more work to come, as I would like to add configuration of > errorHandler as well. The errorHandler has always been a special case, > as it was only in the DSL for spring xml and blueprint xml, > and not in the general DSL. However Java DSL had some special for > errorHandler too. There are some other JIRAs about this work. > > > > > On Thu, Jul 8, 2021 at 2:12 PM Claus Ibsen wrote: > > > > Hi > > > > I am working on (branch on git with the ticket number) > > https://issues.apache.org/jira/browse/CAMEL-16757 > > > > Its work in progress, but I got a prototype in a new example > > https://github.com/apache/camel-examples/tree/main/examples/routes-configuration > > > > In this example we have routes in java, xml and yaml in the myroutes > > folder. For Java you can of course also have it in src/main/java. > > > > But the point is to make this work for dynamic loaded routes (also > > Java) and hence why its in the myroutes folder. > > > > These routes causes exceptions, and then we have global configured > > error handling in separate source code, in the myerror folder. > > > > Those are also polyglot and you can define this in XML, YAML or Java etc. > > > > There is more to work on but wanted to share the heads-up. > > I will be on PTO 2 weeks from tomorrow afternoon. So will work on > > finishing this later. > > > > The potential is also to allow users to define their global error > > handler, route configurations, and you can put that into separate JAR > > and just add as dependency. And then you can refer to these route > > configurations from your routes or to use the global configured etc. > > > > Feedback is welcome, the work is "not set in stone". > > > > > > > > -- > > Claus Ibsen > > - > > http://davsclaus.com @davsclaus > > Camel in Action 2: https://www.manning.com/ibsen2 > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2 -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
Re: [HEADS UP] - More flexible route configuration in all DSLs
Hi Back from PTO then I continued this work https://issues.apache.org/jira/browse/CAMEL-16757 I am so far that I am working on a giant PR to be ready for merging. Review is welcome but it's big work so it's not so easy to understand just by browsing a lot of code changes. In the PR there is documentation of this new feature which is a starting point to see what this fuss is all about. There are also two examples (standalone and spring boot) that are in the github examples repo with the name routes-configuration. https://github.com/apache/camel-spring-boot-examples/tree/main/routes-configuration https://github.com/apache/camel-examples/tree/main/examples/routes-configuration There is more work to come, as I would like to add configuration of errorHandler as well. The errorHandler has always been a special case, as it was only in the DSL for spring xml and blueprint xml, and not in the general DSL. However Java DSL had some special for errorHandler too. There are some other JIRAs about this work. On Thu, Jul 8, 2021 at 2:12 PM Claus Ibsen wrote: > > Hi > > I am working on (branch on git with the ticket number) > https://issues.apache.org/jira/browse/CAMEL-16757 > > Its work in progress, but I got a prototype in a new example > https://github.com/apache/camel-examples/tree/main/examples/routes-configuration > > In this example we have routes in java, xml and yaml in the myroutes > folder. For Java you can of course also have it in src/main/java. > > But the point is to make this work for dynamic loaded routes (also > Java) and hence why its in the myroutes folder. > > These routes causes exceptions, and then we have global configured > error handling in separate source code, in the myerror folder. > > Those are also polyglot and you can define this in XML, YAML or Java etc. > > There is more to work on but wanted to share the heads-up. > I will be on PTO 2 weeks from tomorrow afternoon. So will work on > finishing this later. > > The potential is also to allow users to define their global error > handler, route configurations, and you can put that into separate JAR > and just add as dependency. And then you can refer to these route > configurations from your routes or to use the global configured etc. > > Feedback is welcome, the work is "not set in stone". > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2 -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
Re: [HEADS UP] - More flexible route configuration in all DSLs
Hello Claus, thanks for the clarification! On Fri, Jul 9, 2021 at 9:42 AM Claus Ibsen wrote: > Hi > > The CAMEL-16757 is not really about loading routes (we can do that > already today with the routes loader and resource abstractions). > What it is about is to allow to separate your routes from route > configurations, and also allow such configurations to be easily > reusable. > Today you have either global vs route, but with this you can have > reusable route configurations you can easily on a per route choose to > refer to an existing configuration. > This allow DRY (dont repeat yourself) as otherwise users would have to > copy the same route configuration on their routes. > > Also the configuration can be loaded separated from the actual routes, > so you can have Camel expert users setup a common set of route > configurations and put that in a JAR. > And then novice Camel users can add the JAR and have it enabled out of > the box, or they can refer to the route configuration by a logical > name. > > Today you can't really do that, and you would have to put route > configuration and route together (or if you use Java you can do some > tricks with the xxxDefinition classes). > Also this is to make this unified across all the DSLs. So it's the > same way to do this. For example error handler (not onException) is > different in XML vs Java vs YAML etc. > > > > You have some great ideas for camel and jbang in your email. > I suggest to start a new mail thread on Camel DEV where we can discuss > this. > > > In the mean time take good care of the Camel wile I am on PTO ;) > > > > > On Thu, Jul 8, 2021 at 3:06 PM Otavio Rodolfo Piske > wrote: > > > > Hello Claus, > > > > This looks very interesting! > > > > If I understand correctly, with this change, we'll be able to replace the > > usage of KameletMain in CAMEL-16612 [1] and then we can load all sorts of > > routes out of the box, without relying on or implementing different kinds > > of "Main". > > > > So, if my understanding is correct, then it will make it much easier to > > load them on Camel JBang as the new loader does the bulk of the work. > > > > In terms of ideas ... > > > > One scenario that I have been thinking of - but haven't fully matured > yet - > > is whether it's worth pursuing a type of "pipe into/out of shell" feature > > for CAMEL-16612. Specifically: whether we could modify Camel JBang so > that > > we could use it as part of shell scripts. For example: > > > > CamelJBang run /path/to/route.yaml | ... shell script stuff here (sed, > > grep, awk, etc) > > > > One challenge is that we need to find a way to flexibilize the end of the > > execution dynamically. Maybe even from the standard input ... so that > would > > behave similar to this: > > > > - echo "" | CamelJBang run > > /path/to/route.yaml | ... shell script stuff here (sed, grep, awk, etc) > > > > or > > > > - CamelJBang run /path/to/route.yaml /path/to/termination.yaml | ... > shell > > script stuff here (sed, grep, awk, etc) > > > > With this, we might be able to offer a client that plugs everything to > > everything almost out of the box and is easily scriptable in shell. > > > > Obs.: enjoy your PTO! > > > > 1. https://github.com/apache/camel/pull/5770 > > > > Kind regards > > > > On Thu, Jul 8, 2021 at 2:12 PM Claus Ibsen > wrote: > > > > > Hi > > > > > > I am working on (branch on git with the ticket number) > > > https://issues.apache.org/jira/browse/CAMEL-16757 > > > > > > Its work in progress, but I got a prototype in a new example > > > > > > > https://github.com/apache/camel-examples/tree/main/examples/routes-configuration > > > > > > In this example we have routes in java, xml and yaml in the myroutes > > > folder. For Java you can of course also have it in src/main/java. > > > > > > But the point is to make this work for dynamic loaded routes (also > > > Java) and hence why its in the myroutes folder. > > > > > > These routes causes exceptions, and then we have global configured > > > error handling in separate source code, in the myerror folder. > > > > > > Those are also polyglot and you can define this in XML, YAML or Java > etc. > > > > > > There is more to work on but wanted to share the heads-up. > > > I will be on PTO 2 weeks from tomorrow afternoon. So will work on > > > finishing this later. > > > > > > The potential is also to allow users to define their global error > > > handler, route configurations, and you can put that into separate JAR > > > and just add as dependency. And then you can refer to these route > > > configurations from your routes or to use the global configured etc. > > > > > > Feedback is welcome, the work is "not set in stone". > > > > > > > > > > > > -- > > > Claus Ibsen > > > - > > > http://davsclaus.com @davsclaus > > > Camel in Action 2: https://www.manning.com/ibsen2 > > > > > > > > > -- > > Otavio R. Piske > > http://orpiske.net > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus >
Re: [HEADS UP] - More flexible route configuration in all DSLs
Hi The CAMEL-16757 is not really about loading routes (we can do that already today with the routes loader and resource abstractions). What it is about is to allow to separate your routes from route configurations, and also allow such configurations to be easily reusable. Today you have either global vs route, but with this you can have reusable route configurations you can easily on a per route choose to refer to an existing configuration. This allow DRY (dont repeat yourself) as otherwise users would have to copy the same route configuration on their routes. Also the configuration can be loaded separated from the actual routes, so you can have Camel expert users setup a common set of route configurations and put that in a JAR. And then novice Camel users can add the JAR and have it enabled out of the box, or they can refer to the route configuration by a logical name. Today you can't really do that, and you would have to put route configuration and route together (or if you use Java you can do some tricks with the xxxDefinition classes). Also this is to make this unified across all the DSLs. So it's the same way to do this. For example error handler (not onException) is different in XML vs Java vs YAML etc. You have some great ideas for camel and jbang in your email. I suggest to start a new mail thread on Camel DEV where we can discuss this. In the mean time take good care of the Camel wile I am on PTO ;) On Thu, Jul 8, 2021 at 3:06 PM Otavio Rodolfo Piske wrote: > > Hello Claus, > > This looks very interesting! > > If I understand correctly, with this change, we'll be able to replace the > usage of KameletMain in CAMEL-16612 [1] and then we can load all sorts of > routes out of the box, without relying on or implementing different kinds > of "Main". > > So, if my understanding is correct, then it will make it much easier to > load them on Camel JBang as the new loader does the bulk of the work. > > In terms of ideas ... > > One scenario that I have been thinking of - but haven't fully matured yet - > is whether it's worth pursuing a type of "pipe into/out of shell" feature > for CAMEL-16612. Specifically: whether we could modify Camel JBang so that > we could use it as part of shell scripts. For example: > > CamelJBang run /path/to/route.yaml | ... shell script stuff here (sed, > grep, awk, etc) > > One challenge is that we need to find a way to flexibilize the end of the > execution dynamically. Maybe even from the standard input ... so that would > behave similar to this: > > - echo "" | CamelJBang run > /path/to/route.yaml | ... shell script stuff here (sed, grep, awk, etc) > > or > > - CamelJBang run /path/to/route.yaml /path/to/termination.yaml | ... shell > script stuff here (sed, grep, awk, etc) > > With this, we might be able to offer a client that plugs everything to > everything almost out of the box and is easily scriptable in shell. > > Obs.: enjoy your PTO! > > 1. https://github.com/apache/camel/pull/5770 > > Kind regards > > On Thu, Jul 8, 2021 at 2:12 PM Claus Ibsen wrote: > > > Hi > > > > I am working on (branch on git with the ticket number) > > https://issues.apache.org/jira/browse/CAMEL-16757 > > > > Its work in progress, but I got a prototype in a new example > > > > https://github.com/apache/camel-examples/tree/main/examples/routes-configuration > > > > In this example we have routes in java, xml and yaml in the myroutes > > folder. For Java you can of course also have it in src/main/java. > > > > But the point is to make this work for dynamic loaded routes (also > > Java) and hence why its in the myroutes folder. > > > > These routes causes exceptions, and then we have global configured > > error handling in separate source code, in the myerror folder. > > > > Those are also polyglot and you can define this in XML, YAML or Java etc. > > > > There is more to work on but wanted to share the heads-up. > > I will be on PTO 2 weeks from tomorrow afternoon. So will work on > > finishing this later. > > > > The potential is also to allow users to define their global error > > handler, route configurations, and you can put that into separate JAR > > and just add as dependency. And then you can refer to these route > > configurations from your routes or to use the global configured etc. > > > > Feedback is welcome, the work is "not set in stone". > > > > > > > > -- > > Claus Ibsen > > - > > http://davsclaus.com @davsclaus > > Camel in Action 2: https://www.manning.com/ibsen2 > > > > > -- > Otavio R. Piske > http://orpiske.net -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
Re: [HEADS UP] - More flexible route configuration in all DSLs
Hello Claus, This looks very interesting! If I understand correctly, with this change, we'll be able to replace the usage of KameletMain in CAMEL-16612 [1] and then we can load all sorts of routes out of the box, without relying on or implementing different kinds of "Main". So, if my understanding is correct, then it will make it much easier to load them on Camel JBang as the new loader does the bulk of the work. In terms of ideas ... One scenario that I have been thinking of - but haven't fully matured yet - is whether it's worth pursuing a type of "pipe into/out of shell" feature for CAMEL-16612. Specifically: whether we could modify Camel JBang so that we could use it as part of shell scripts. For example: CamelJBang run /path/to/route.yaml | ... shell script stuff here (sed, grep, awk, etc) One challenge is that we need to find a way to flexibilize the end of the execution dynamically. Maybe even from the standard input ... so that would behave similar to this: - echo "" | CamelJBang run /path/to/route.yaml | ... shell script stuff here (sed, grep, awk, etc) or - CamelJBang run /path/to/route.yaml /path/to/termination.yaml | ... shell script stuff here (sed, grep, awk, etc) With this, we might be able to offer a client that plugs everything to everything almost out of the box and is easily scriptable in shell. Obs.: enjoy your PTO! 1. https://github.com/apache/camel/pull/5770 Kind regards On Thu, Jul 8, 2021 at 2:12 PM Claus Ibsen wrote: > Hi > > I am working on (branch on git with the ticket number) > https://issues.apache.org/jira/browse/CAMEL-16757 > > Its work in progress, but I got a prototype in a new example > > https://github.com/apache/camel-examples/tree/main/examples/routes-configuration > > In this example we have routes in java, xml and yaml in the myroutes > folder. For Java you can of course also have it in src/main/java. > > But the point is to make this work for dynamic loaded routes (also > Java) and hence why its in the myroutes folder. > > These routes causes exceptions, and then we have global configured > error handling in separate source code, in the myerror folder. > > Those are also polyglot and you can define this in XML, YAML or Java etc. > > There is more to work on but wanted to share the heads-up. > I will be on PTO 2 weeks from tomorrow afternoon. So will work on > finishing this later. > > The potential is also to allow users to define their global error > handler, route configurations, and you can put that into separate JAR > and just add as dependency. And then you can refer to these route > configurations from your routes or to use the global configured etc. > > Feedback is welcome, the work is "not set in stone". > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2 > -- Otavio R. Piske http://orpiske.net
