Hi,
I try to use Ignite 2.1 to simulate the situation where a table doesn't fit
in memory and data is read from disk every time a full table scan occurs.
I've adapted the PersistentStoreExample to create a cache of a million items
which is stored in 200MB of data files and 120MB of index files. I call
PersistentStoreConfiguration.setFileIOFactory() to register a custom
FileIOFactory that tracks what file I/O is occurring. This tracker shows the
files are read during startup of the example, but not while the SQL query is
running. The cache shouldn't fit in 10MB and I would expect that during the
query the data would be read from disk. Why is this not happening and what
do I need to change to test this?
Thanks for your help.
Pascal
<bean class="org.apache.ignite.configuration.MemoryConfiguration">
<property name="pageSize" value="2048"/>
<property name="systemCacheInitialSize" value="#{10 * 1024 * 1024}"/>
<property name="systemCacheMaxSize" value="#{10 * 1024 * 1024}"/>
<property name="defaultMemoryPolicyName" value="default_mem_plc"/>
<property name="memoryPolicies">
<list>
<bean
class="org.apache.ignite.configuration.MemoryPolicyConfiguration">
<property name="name" value="default_mem_plc"/>
<property name="initialSize" value="#{10 * 1024
* 1024}"/>
<property name="maxSize" value="#{10 * 1024 *
1024}"/>
</bean>
</list>
</property>
</bean>
public class PersistentStoreExample {
private static final boolean UPDATE = false;
private static final long COUNT = 1000000;
public static void doMain() throws Exception {
Ignition.setClientMode(false);
Ignite ig =
Ignition.start("D:/Ignite/2.1/examples/config/persistentstore/example-persistent-store.xml");
IgniteConfiguration ic = ig.configuration();
PersistentStoreConfiguration pc =
ic.getPersistentStoreConfiguration();
pc.setPersistentStorePath("C:\\tmp\\");
MyFileIOFactory myFact = new MyFileIOFactory();
pc.setFileIOFactory(myFact);
ig.active(true);
IgniteCache<Long, MyPerson> cache = ig.cache("personCache");
if (UPDATE) {
System.out.println("Populating the cache...");
try (IgniteDataStreamer<Long, MyPerson> streamer =
ig.dataStreamer("personCache")) {
streamer.allowOverwrite(true);
for (long i = 0; i < COUNT; i++) {
MyPerson o = new MyPerson(i, "per-" + i);
streamer.addData(i, o);
if (i > 0 && i % 10_000 == 0) System.out.println("Done:
" + i);
}
}
}
while(true)
{
Thread.sleep(3000);
System.out.println("\nFull scan " + COUNT + " times...");
for(int getId = 0; getId< COUNT; getId++) {
QueryCursor cur = cache.query(new SqlFieldsQuery("select id,
name from MyPerson where name like ?").setArgs("per-" + getId));
if(getId % 100000 == 0) {
System.out.println(" SQL Result: " + cur.getAll());
}
}
}
}
}
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/MemoryPolicy-size-vs-cache-size-tp15892.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
