I think that what you want is a union and then a join
On Wed, Apr 7, 2010 at 4:11 PM, Larry R <[email protected]> wrote:
> I have a product lookup table to join against.
> foreach (string inputFile in
> Directory.GetFiles(Settings.Default.InputFilePath,"*.xls"))
> {
> //JoinData();
> Register ( new JoinModelMasterOperation()
> .Left( new
> PortfolioReadOperation(inputFile) )
> .Right( new
> ModelMasterReadOperation(AFM_CONNECTION_NAME) )
> );
>
> }
> //ApplyBusinessRules();
> Register(new ApplyBusinessRulesOperation());
> ...
>
> I was hoping to read these files ( 38 x 1000 records) in one pass of
> the application.
>
>
> On Apr 7, 8:36 am, Ayende Rahien <[email protected]> wrote:
> > What are you joining against here?
> > Your best bet is to write an operation that simple reads from all the
> files.
> > What you want in not a join, it is a union.
> >
> > On Wed, Apr 7, 2010 at 3:20 PM, Larry R <[email protected]> wrote:
> > > Ok, so in example "a", wouldn't all the records in all the files be
> > > joined, then proceed to the BusinessRules?
> >
> > > On Apr 7, 1:45 am, webpaul <[email protected]> wrote:
> > > > I'm not sure either of these will work - Keep in mind the output of
> > > > the prior operation goes to the next operation. If you "unroll" your
> > > > loop you'll see you'd have some issues there. It may be simpler to
> > > > have a process that takes an input file as a parameter and launch the
> > > > entire process multiple times.
> >
> > > > On Apr 6, 9:15 pm, Larry R <[email protected]> wrote:
> >
> > > > > I have a question on which pattern to use. I have x number of input
> > > > > files (XLS), which are merged, custom rules applied, magic pixie
> dust
> > > > > applied then inserted into a database.
> >
> > > > > { My syntax is off as I don't have the project in front of me }
> >
> > > > > So, should I do a)
> > > > > foreach( string file in Directory.GetFiles("*.xls")
> > > > > {
> > > > > Register(new JoinOperationHere ( ).Left(new
> > > > > ReadFileOperation(file).Right(new DatabaseModelRecords() ) );
> >
> > > > > )}
> >
> > > > > Register ( new BusinessRulesOperation() );
> > > > > Register( new WriteToDbOperation() );
> > > > > ...etc
> >
> > > > > or
> > > > > b)
> > > > > foreach( string file in Directory.GetFiles("*.xls")
> > > > > {
> > > > > Register(new JoinOperationHere ( ).Left(new
> > > > > ReadFileOperation(file).Right(new DatabaseModelRecords() ) );
> > > > > Register ( new BusinessRulesOperation() );
> > > > > Register( new WriteToDbOperation() );
> >
> > > > > )
> >
> > > > > }
> >
> > > > > And, for bonus points, after you've done an aggregation, how would
> you
> > > > > use that after the end of the EtlProcess to report the number of
> > > > > records ?
> >
> > > > > Thanks. I really like this toolset. I've done what a previous team
> > > > > took 2 months to do, in the past 3 days.
> >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Rhino Tools Dev" group.
> > > To post to this group, send email to [email protected].
> > > To unsubscribe from this group, send email to
> > > [email protected]<rhino-tools-dev%[email protected]>
> <rhino-tools-dev%[email protected]<rhino-tools-dev%[email protected]>
> >
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/rhino-tools-dev?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Rhino Tools Dev" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected]<rhino-tools-dev%[email protected]>
> .
> For more options, visit this group at
> http://groups.google.com/group/rhino-tools-dev?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Rhino Tools Dev" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/rhino-tools-dev?hl=en.
