Michael,
This happens even with Single node. I have a total of 41k addresses and the
code fails while inserting 5600th node. Tried in batches of 1000s to single
digit. Could this be a data problem? Same data and code works fine in 1.9.5
even with commit interval of 1000. I am attaching the SDN class and the
method that loads data.
Thanks
On Fri, Feb 7, 2014 at 9:05 PM, Michael Hunger <
[email protected]> wrote:
> How many nodes are you adding at once and how big is your Java heap?
>
> You don't have to create a layer, the index automatically creates the
> layer.
>
> Michael
>
> Am 07.02.2014 um 21:55 schrieb Pattabi <[email protected]>:
>
> Getting this error while loading bean which has a 'wkt/spatial' index
> using - Spring data-neo4j 3.0.0.BUILD-20140207...., neo4j 2.0.0
>
> *Note*: I do not know how to do batch insert a spatial (as it is my
> understanding that I need to create a layer with that WKT, add these nodes
> to that layer)
> I looked at the code 'SpatialIndexImplementation' nodeIndex. *If I have
> to do batch insert, *adding to this index would be sufficient?
>
> The stack trace is as shown below:
>
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.lucene.store.BufferedIndexOutput.<init>(BufferedIndexOutput.java:26)
> at
> org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:438)
> at
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:306)
> at
> org.apache.lucene.index.FormatPostingsPositionsWriter.<init>(FormatPostingsPositionsWriter.java:43)
> at
> org.apache.lucene.index.FormatPostingsDocsWriter.<init>(FormatPostingsDocsWriter.java:57)
> at
> org.apache.lucene.index.FormatPostingsTermsWriter.<init>(FormatPostingsTermsWriter.java:33)
> at
> org.apache.lucene.index.FormatPostingsFieldsWriter.<init>(FormatPostingsFieldsWriter.java:51)
> at
> org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
> at org.apache.lucene.index.TermsHash.flush(TermsHash.java:113)
> at org.apache.lucene.index.DocInverter.flush(DocInverter.java:70)
> at
> org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:60)
> at
> org.apache.lucene.index.DocumentsWriter.flush(DocumentsWriter.java:581)
> at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3587)
> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3552)
> at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:450)
> at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:399)
> at
> org.apache.lucene.index.DirectoryReader.doOpenFromWriter(DirectoryReader.java:413)
> at
> org.apache.lucene.index.DirectoryReader.doOpenIfChanged(DirectoryReader.java:432)
> at
> org.apache.lucene.index.DirectoryReader.doOpenIfChanged(DirectoryReader.java:375)
> at
> org.apache.lucene.index.DirectoryReader.doOpenIfChanged(DirectoryReader.java:392)
> at
> org.apache.lucene.index.IndexReader.openIfChanged(IndexReader.java:634)
> at
> org.neo4j.index.impl.lucene.LuceneDataSource.refreshSearcher(LuceneDataSource.java:505)
> at
> org.neo4j.index.impl.lucene.LuceneDataSource.refreshSearcherIfNeeded(LuceneDataSource.java:635)
> at
> org.neo4j.index.impl.lucene.LuceneDataSource.getIndexSearcher(LuceneDataSource.java:577)
> at org.neo4j.index.impl.lucene.LuceneIndex.query(LuceneIndex.java:293)
> at org.neo4j.index.impl.lucene.LuceneIndex.query(LuceneIndex.java:251)
> at
> org.neo4j.gis.spatial.indexprovider.LayerNodeIndex.findExistingNode(LayerNodeIndex.java:152)
> at
> org.neo4j.gis.spatial.indexprovider.LayerNodeIndex.add(LayerNodeIndex.java:132)
> at
> org.neo4j.gis.spatial.indexprovider.LayerNodeIndex.add(LayerNodeIndex.java:1)
> at
> org.springframework.data.neo4j.fieldaccess.IndexingPropertyFieldAccessorListenerFactory$IndexingPropertyFieldAccessorListener.valueChanged(IndexingPropertyFieldAccessorListenerFactory.java:86)
> at
> org.springframework.data.neo4j.fieldaccess.DefaultEntityState.notifyListeners(DefaultEntityState.java:137)
> at
> org.springframework.data.neo4j.fieldaccess.DefaultEntityState.setValue(DefaultEntityState.java:114)
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Neo4j" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Neo4j" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/neo4j/TnwBO3xKrv4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
@NodeEntity
@TypeAlias("Address")
public class Address extends AbstractEntity {
@RelatedTo(type = "riskLocation", direction = Direction.INCOMING)
// @Fetch
Set<Policy> policy;
@Indexed(indexType = IndexType.FULLTEXT, indexName = "address")
String addressLine1;
String addressLine2, country;
@RelatedTo(type="RiskZip")
// @Fetch
Zipcode zipCode;
double latitude, longitude;
@Indexed(indexType = POINT, indexName = "Geocode")
String wkt;
@RelatedTo(type="RiskCity")
// @Fetch
City city;
@RelatedTo(type="RiskState")
// @Fetch
State state;
@RelatedTo(type="RiskCounty")
// @Fetch
County county;
int locationId;
@RelatedTo(type = "assessment")
// @Fetch
Set<WindAssessmentReport> windAssessment;
public void loadAllAddresses(){
List<Address> addresses = dmDao.getAllAddresses();
Map<String, String> uniqueAddresses = new
HashMap<String,String>();
int i= 0;
for(Address addr: addresses){
String key = String.format("%s_%s_%s",
addr.getAddressLine1(), addr.getCity(), addr.getZipCode());
if(uniqueAddresses.get(key)!=null)continue;
transaction.prepareTransaction();
System.out.println("%%%%%%%%%%%%%%%%%%%%%%%%%% Saving " +
i + " of " + addresses.size() + addr);
addressRepository.save(addr);
uniqueAddresses.put(key,key);
i++;
}
}
