So does the Spring-Batch framework only process Lists? What are the options for feeding it?
Larry On 10/4/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > > > > Thanks for the insights Jeff. > > > > Shifting a paradigm and letting iBATIS control the flow is not an option > this case, but I think iBATIS can still be used in a slightly different > "driving query" scenario. In this case a "driving query" is run first which > retrieves all primary keys. When framework asks for next item, the item > provider takes the next key and asks iBATIS for the corresponding object – > it is not what I was originally looking for, but still useful. > > > > Btw. the motivation for my post is to figure out how ibatis can be > integrated as an input source for the emerging Spring-Batch framework. It > provides support for ordinary sql with manual mapping, but supporting ORM > frameworks (especially Hibernate & iBATIS) would be a valuable addition. > > > > Robert > > > > ________________________________ > > > From: Jeff Butler [mailto:[EMAIL PROTECTED] > Sent: Thursday, October 04, 2007 14:15 > > To: user-java@ibatis.apache.org > Subject: Re: how to map huge resultsets? > > > > > > Your question is not that confusing. Here are the answers: > > > > > > 1. No (although the approach suggested by Christopher Lamey is very close if > you are willing to shift your paradigm) > > > > > > 2. If the result set is truly huge, then approach #1 is a bad idea because > iBATIS will have to read through huge amounts of data to get to the latter > parts of the result set in subsequent calls (this is why Larry asked about > selecting by primary key rather than returning such a huge result set) > > > > > > I guess you'll have to stick with Hibernate :) > > > > > > Jeff Butler > > > > > On 10/4/07, [EMAIL PROTECTED] <[EMAIL PROTECTED] > > wrote: > > Seems like my question was quite confusing, so I'll try to explain what > I need once again from scratch. > > 1. I need to process a potentially huge number of rows (so > queryForList(query) is not an option because it would try to load all > objects into memory at once). > > 2. I need to control the iteration over the query results, so I can't > use queryWithRowHandler (in this case iBATIS iterates and I only tell > iBATIS how each record should be processed - I need to be able to ask > iBATIS for the next record instead - it is the internal vs. external > iterator difference, or SAX vs. StAX in case of XML processing or how > collections are typically iterated in ruby/groovy vs. Java or ... you > name it). Simply framework iterates, iBATIS knows the query and must > provide next record when it is asked for it. > > First approach: > It is possible to implement these requirements by using > queryForList(query, skipSize, maxSize) and query the database > TOTAL_ROW_COUNT / maxSize times (issuing a new query when records from > current list have been processed - this is invisible to the user > (framework) who just iteratively asks for next record for processing). > > Second approach: > Hibernate allows to get a ScrollableResults object for a query allowing > the user to move the cursor and ask for an object that corresponds to > the current row. > > Questions: > 1. is the second approach possible with iBATIS? > 2. first vs. second approach pros & cons? > > Thanks > Robert > > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf > Of Larry Meadors > Sent: Thursday, October 04, 2007 01:18 > To: user-java@ibatis.apache.org > Subject: Re: how to map huge resultsets? > > I'm confused, how would grabbing chunks of a huge result set be more > efficient than grabbing the records by PK? > > Couldn't you just have a select that grabbed the requested item, and > passed it back to the consumer? > > Larry > > > On 10/3/07, [EMAIL PROTECTED] > <[EMAIL PROTECTED]> wrote: > > My problem with RowHandler is that iBATIS controls the iteration. I > just say > > > > sqlMap.queryWithRowHandler ("getAllItems", rowHandler); > > > > and all items get processed by the rowHandler. > > > > But in my case I need to make iBATIS return items one-by-one when it > is asked to do so because the framework controls the iteration. > > This is a very simplified basic logic of the framework: > > > > while (itemProvider.hasNext()) { > > Object item = itemProvider.next (); > > process(item); > > } > > > > > > > > -----Original Message----- > > From: Christopher Lamey [mailto:[EMAIL PROTECTED] > > Sent: Wed 10/3/2007 11:55 PM > > To: user-java@ibatis.apache.org > > Subject: Re: how to map huge resultsets? > > > > Hmm...I don't see how having an external framework prevents you from > using a > > RowHandler. Your item provider could implement the RowHandler > interface and > > the external code wouldn't know or care about it. Or your item > provider > > could wrap something that does implement RowHandler so the external > code > > doesn't know it exists. The main point is that you can pull mapped > objects > > on a row by row basis from the database. > > > > How is a RowHandler different that what you were describing in your > first > > mail? > > > > On 10/3/07 3:33 PM, "[EMAIL PROTECTED]" > > <[EMAIL PROTECTED] > wrote: > > > > > > > > Thanks for your reply. > > > > > > I can't use the rowhandler callback because the iteration is > external to > > > iBATIS. In my case a batch framework iteratively asks for an item > and > > > processes it - and I am trying to implement an iBATIS item provider > (I realize > > > now I should have explained this in the initial post). > > > > > > Robert > > > > > > > > > -----Original Message----- > > > From: Christopher Lamey [mailto:[EMAIL PROTECTED] > > > Sent: Wed 10/3/2007 11:06 PM > > > To: user-java@ibatis.apache.org > > > Subject: Re: how to map huge resultsets? > > > > > > Hello, > > > > > > You should take a look at the RowHandler interface and the > > > queryWithRowHandler calls in SqlMapClient (page 61 of the pdf). > Basically, > > > the RowHandler gets invoked for every row returned rather than > mapping all > > > the rows into objects in a collection. > > > > > > Cheers, > > > Chris > > > > > > > > > On 10/3/07 2:37 PM, "[EMAIL PROTECTED]" > > > <[EMAIL PROTECTED] > wrote: > > > > > >> Hello, > > >> > > >> I am wondering whether it possible to implement the following > scenario with > > >> iBATIS: > > >> > > >> 1. run an iBATIS-managed select > > >> 2. get a scrollable result set instead of a list of mapped > objects > > >> 3. manually scroll the result set and ask iBATIS for object > corresponding > > >> to current row > > >> > > >> Hibernate provides this possibility > > >> > (http://www.hibernate.org/hib_docs/reference/en/html/batch.html > ) so > I thought > > >> it would be feasible with iBATIS too, but I couldn't figure out a > way. The > > >> motivation is a batch scenario where the select returns a huge > number of rows > > >> so all mapped objects can't be loaded into memory at once. > > >> > > >> The iBATIS way I am aware of is to use queryForList(String > statementName, int > > >> skipResults, int maxResults), but this means querying the database > > >> (TOTAL_NUMBER_OF_ROWS / maxResults) times. > > >> > > >> Can somebody give advice about pros & cons of the two approaches? > > >> > > >> Thanks > > >> Robert > > >> > > >> > > >> This message is for the designated recipient only and may contain > privileged, > > >> proprietary, or otherwise private information. If you have > received it in > > >> error, please notify the sender immediately and delete the > original. Any > > >> other use of the email by you is prohibited. > > > > > > > > > > > > > > > This message is for the designated recipient only and may contain > privileged, > > > proprietary, or otherwise private information. If you have received > it in > > > error, please notify the sender immediately and delete the original. > Any > > > other use of the email by you is prohibited. > > > > > > > > > > This message is for the designated recipient only and may contain > privileged, proprietary, or otherwise private information. If you have > received it in error, please notify the sender immediately and delete > the original. Any other use of the email by you is prohibited. > > > > > > > This message is for the designated recipient only and may contain > privileged, proprietary, or otherwise private information. If you have > received it in error, please notify the sender immediately and delete the > original. Any other use of the email by you is prohibited. > > > > > > > > This message is for the designated recipient only and may contain > privileged, proprietary, or otherwise private information. If you have > received it in error, please notify the sender immediately and delete the > original. Any other use of the email by you is prohibited.
