Dave, It sounds like many of the issues you are having with specific processors can be answered with a combination of mailing list questions and the processor documentation. I would also encourage you to read the Getting Started Guide [1], User Guide [2], and Admin Guide [3]. They are long, but really offer a good overview of the entire system. Once you understand the framework (which can be complicated), a lot of the little pieces which can seem weird start to fall into place.
There are also a number of good independent blogs (Bryan Bende [4], Pierre Villard [5], Matt Burgess [6]) which give step-by-step instructions on various topics, and some excellent overview videos on YouTube by Jenn Barnabee [7]. [1] https://nifi.apache.org/docs/nifi-docs/html/getting-started.html [2] https://nifi.apache.org/docs/nifi-docs/html/user-guide.html [3] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html [4] https://bryanbende.com/ <https://bryanbende.com/> [5] https://pierrevillard.com/ <https://pierrevillard.com/> [6] https://funnifi.blogspot.com/ <https://funnifi.blogspot.com/> [7] https://www.youtube.com/playlist?list=PLHre9pIBAgc4e-tiq9OIXkWJX8bVXuqlG <https://www.youtube.com/playlist?list=PLHre9pIBAgc4e-tiq9OIXkWJX8bVXuqlG> Andy LoPresto [email protected] [email protected] PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jun 18, 2018, at 10:28 AM, Dave <[email protected]> wrote: > > Thanks.. I will try to implement this suggestion also. > > Actually since I earlier have managed a Datawarehouse project, I am trying > to pick up scenarios based on that experience. I have actually visualised > this specific scenario. > > The source file will be a CSV file. And all the fields will be be stored as > strings in the csv file(Similar to the source files in a DWH project). Once > the fields are validated for their length, then the data will be stored in > DB table with a unique ID to keep a track of failure at a later stage.. My > plan is to store the data in the appropriate format at this point with the > unique ID.. for eg) ID and Zipcode to be numeric. > > Once I am able to achieve the above I would like to add a few more fields > later like date, some descriptions of the city also will be added. The > description will also have invalid characters. After this I would like to > implement another set of validations like date format, remove invalid > characters etc. and then store in a different DB. > > At every stage I want to also capture the error records and either publish > them or store in a different DB. > > This is goal for now. > > Thanks > Dave > > > > -- > Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
signature.asc
Description: Message signed with OpenPGP using GPGMail
