Storing ArrayList with over 10000 objects

2011-03-09 Thread Jan Juno
I have One Array-list(with over 1 objects in it) what is the best
practice for caching it so I don't have to load it over and over again in
each request?

Jan


Re: Storing ArrayList with over 10000 objects

2011-03-09 Thread Martijn Dashorst
On Wed, Mar 9, 2011 at 1:00 PM, Jan Juno janko...@gmail.com wrote:
 I have One Array-list(with over 1 objects in it) what is the best
 practice for caching it so I don't have to load it over and over again in
 each request?

Usually at our company we expect our ORM mapper (hibernate) to take
care of caching, so it is not a problem for us to ask the ORM for the
same query again and again and again.

I'd use a cache for that. Either something home grown (usually not a
great idea, but for a single use case it might work), or something
like ehcache. If the item can't be found in the cache, retrieve it
from data store and put it in cache.

I wouldn't keep the 10k objects in a page instance or the user session
since that will be serialized with each request.

Martijn

-- 
Become a Wicket expert, learn from the best: http://wicketinaction.com

-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org



Re: Storing ArrayList with over 10000 objects

2011-03-09 Thread Jan Juno
can you point me to a good wicket + ehcache tutorial?

On 9 March 2011 13:13, Martijn Dashorst martijn.dasho...@gmail.com wrote:

 On Wed, Mar 9, 2011 at 1:00 PM, Jan Juno janko...@gmail.com wrote:
  I have One Array-list(with over 1 objects in it) what is the best
  practice for caching it so I don't have to load it over and over again in
  each request?

 Usually at our company we expect our ORM mapper (hibernate) to take
 care of caching, so it is not a problem for us to ask the ORM for the
 same query again and again and again.

 I'd use a cache for that. Either something home grown (usually not a
 great idea, but for a single use case it might work), or something
 like ehcache. If the item can't be found in the cache, retrieve it
 from data store and put it in cache.

 I wouldn't keep the 10k objects in a page instance or the user session
 since that will be serialized with each request.

 Martijn

 --
 Become a Wicket expert, learn from the best: http://wicketinaction.com

 -
 To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
 For additional commands, e-mail: users-h...@wicket.apache.org




Re: Storing ArrayList with over 10000 objects

2011-03-09 Thread robert.mcguinness
Martijn,

Wouldn't this be a safe way to cache a list at the expense of memory (using
transient)?   In a lot of my projects I have to do some aggregation of the
results from multiple data stores that are slow even with the caching layer. 


private class SomeDataProvider extends SortableDataProvider {

/* cache the page results but don't serialize them to session 
or disk */
private transient SearchResults searchResults;

private static final long serialVersionUID = 1L;
private SortParam sortParam;

public SomeDataProvider () {}

@Override
public int size() {

/*
 * if null (back button/forward button case) rebuild 
the results using
the search parmas that are stored with page
 */
if (searchResults == null) {

searchResults = 
searchService.findSomeStuff(serializableSearchParams);
}

return searchResults.size();;
}

@Override
public IModel model(Object object) {
return new 
SearchHitModel(object).setShouldDetach(false);
}

@Override
public Iterator iterator(int first, int count) {

SortParam sp = getSort();

// only sort results when the sort command changes
if (!sortParam.equals(sp)) {
Sorter sorter = new Sorter(sp.getProperty(), 
sp.isAscending());

Collections.sort(searchResults.getSearchResultList(), sorter);
sortParam = sp;
}

return searchResults.getPagedResults(first, 
count).iterator();
}

}


--
View this message in context: 
http://apache-wicket.1842946.n4.nabble.com/Storing-ArrayList-with-over-1-objects-tp3343442p3343499.html
Sent from the Users forum mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org



Re: Storing ArrayList with over 10000 objects

2011-03-09 Thread Martijn Dashorst
Should work, though perhaps going to the datastore for each
back/forward button press might be overkill. Retrieving it from a
cache, like ehcache, allows you to set an expiry on the data, keep it
thread safe and even memory safe (i.e. let ehcache throw away the
objects when memory is tight, or let it write to disk if not used for
a while).

AFAIK there are no examples on the web utilizing ehcache and wicket
specifically. Just use a plain ehcache example, and access the cache
store in a loadable detacheble model, or possibly even in your service
method.

Martijn

On Wed, Mar 9, 2011 at 1:48 PM, robert.mcguinness
robert.mcguinness@gmail.com wrote:
 Martijn,

 Wouldn't this be a safe way to cache a list at the expense of memory (using
 transient)?   In a lot of my projects I have to do some aggregation of the
 results from multiple data stores that are slow even with the caching layer.


 private class SomeDataProvider extends SortableDataProvider {

                /* cache the page results but don't serialize them to session 
 or disk */
                private transient SearchResults searchResults;

                private static final long serialVersionUID = 1L;
                private SortParam sortParam;

                public SomeDataProvider () {}

                @Override
                public int size() {

                        /*
                         * if null (back button/forward button case) rebuild 
 the results using
 the search parmas that are stored with page
                         */
                        if (searchResults == null) {

                                searchResults = 
 searchService.findSomeStuff(serializableSearchParams);
                        }

                        return searchResults.size();;
                }

                @Override
                public IModel model(Object object) {
                        return new 
 SearchHitModel(object).setShouldDetach(false);
                }

                @Override
                public Iterator iterator(int first, int count) {

                        SortParam sp = getSort();

                        // only sort results when the sort command changes
                        if (!sortParam.equals(sp)) {
                                Sorter sorter = new Sorter(sp.getProperty(), 
 sp.isAscending());
                                
 Collections.sort(searchResults.getSearchResultList(), sorter);
                                sortParam = sp;
                        }

                        return searchResults.getPagedResults(first, 
 count).iterator();
                }

        }


 --
 View this message in context: 
 http://apache-wicket.1842946.n4.nabble.com/Storing-ArrayList-with-over-1-objects-tp3343442p3343499.html
 Sent from the Users forum mailing list archive at Nabble.com.

 -
 To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
 For additional commands, e-mail: users-h...@wicket.apache.org





-- 
Become a Wicket expert, learn from the best: http://wicketinaction.com

-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org



Re: Storing ArrayList with over 10000 objects

2011-03-09 Thread Michael O'Cleirigh

Hi,

Another way is to have a service manage the list.  We have a reference 
data list (list of streets) in our application that has 100k elements 
that is stored in memory and we have a service hold one instance that is 
shared between all accessors (typically the autocomplete search fields) 
of the data.


The list is built when the application starts up and has defined refresh 
points and then all users can just access the data through the service.


e.g.

public ListStreetData referenceDataService.find(String streetNamePrefix);

Since we use spring and inject the service using @SpringBean it is 
wrapped in a proxy that prevents the 100k element list from being 
serialized with the page.


Regards,

Mike



I have One Array-list(with over 1 objects in it) what is the best
practice for caching it so I don't have to load it over and over again in
each request?

Jan




-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org