Hi, For however as far as you read, thank you, I value your expertise, and your time... you are simply the only folks with whom I could share these "dumb" questions and discuss these topics.
My story: A few months ago I inherited a custom data management system. Its goal in life is to allow clients to look up data via a service call. Data had always been loaded by hand-coding an Excel spreadsheet using copy-and-pasted functions to generate SQL. This needs to go away. Recently we got a "little" requirement like this: For each DTO, load data into the DB like this: 1. Read the CSV file. 2. Populate the DTO. 3. Call a service method 'save' which accepts any DTO of the same supertype. We had some proof of concept work lying around (that I had written a few months ago, with less understanding of the eventual goal) for parsing CSV files and loading DTOs. It is really basic, it uses SimpleCSV to parse the files, the specification for mapping columns to properties of the DTO are done in an intermediary "model" class so the BAs can just look at the class and say "Oh yea, the property state actually parses a string from column 4", and then Apache Commons Beanutils copies the properties from the model over to the DTO (can't do this in the DTOs because of dependency issues). This works well enough but for the horrible hand-coding of the mappings.Friday night I figured it would be nice to use annotations instead, and then decided that it is a bad idea to write yet another framework and set out searching for one. I found Apache Camel Bindy. I was delighted. It is exactly what I wanted. The only problem is that I am totally ignorant of Apache Camel. I spent some time reading up on the EIPs and about how Camel Works. I need help figuring out of I should dig deeper and whether our project is a good fit. I feel like the basic flow of above is a good candidate for implementation as a route in Camel. <file rows> -> [ftp endpoint] -> [import queue] -> [CSV translation via Bindy] -> <POJOs> -> [POJO Import Queue] -> (Database) What do you think, is it a good candidate? I keep in mind that even if we didn't use Camel, we would end up doing this exact same thing, using custom code... and again we do not want to write a custom framework! :). I think it is a good candidate since in the future, it can only get more complex. Thinking ahead to the future, I do know that the business wants to do things like: 1. Allow a user to submit specific files to the input queue via a UI; so now there are two ways to get get files queued up for importation. 2. Allow a user to define logical "jobs" for importing files that would allow: A. Tracking of what files came in with the job. B. What rows of data came in with the job. C. Such that we could view all of the information involved, and even "back it out" at some point. 3. Import data from the legacy system. 4. Eventually we'll have hundreds of DTOs to map, every couple of weeks. It seems like #1 is straightforward two sources into one end point scenario. It seems like #2 is a content enricher scenario. It seems like #3 is a scenario that would benefit from an Enteprise Integration System. It seems like #4 is again a good scenario. It looks like long term it would be a good addition because I don't want to write a framework or custom code to handle the intricacies (though the world would not end if we did, it certainly would not be any easier). What do you think? At this point, nobody in the organization will be using an ESB, however, I feel like Camel would work fine as an Enteprise Integration System, deployed in the EAR that contains the service that provides this system I've inherited. Does that make sense? Is there a precedent? Is there any documentation/resources for this usage scenario other than the one in the user guide (the one that doesn't use Spring)? Is Spring a best-practice for using Camel? I would prefer to avoid it if I can and keep things as simple as possible, then again I never learned Spring, so perhaps I am the problem :). Is it easy to do things like add a file to the queue via a web application? What I mean is rather than dropping in a file somewhere, just upload it via the web app and plunk it into the queue? I'm thinking that it is, just manually get a handle on the endpoint and drop in the message. How has your experience been with testing? Must you use Spring? If we move forward with this, I'm planning on using the Camel In Action book (along with reading EIP as I know next to nothing about both). Is that the best one? Best wishes, Grant Rettke -- http://www.wisdomandwonder.com/ ACM, AMA, COG, IEEE
