Hi Luca;
You are so kind. I develop some framework for spring orintdb so it takes
time to understand code. I write transaction manager so data access layer
is like hibernate or others. I start commit and rollback over transaction
manager I don't call this methods explicitly. When you see any interaction
like save edge,save vertex or load something I get current connection from
thread local. At the following code first transaction manager gets
connection, then code finds Cookie, Ip and User vertexes( all has index)
from DB, then I create Login vertex and then I create three edges and
commit transaction over transaction manager. Service layer is this at the
following show implemetaion of dao methods.
@Override
@Transactional(value = "orientTransactionManager", propagation =
Propagation.REQUIRED)
public void insertLogins(List<LoginInformationType> loginTypes) {
for (LoginInformationType loginInformationType : loginTypes){
List<BaseOrientEdgeEntity> edges = new ArrayList<>();
UserEntity userEntity =
userGraphDao.load(loginInformationType.getUserId());
CookieEntity cookieEntity =
cookieGraphDao.load(loginInformationType.getCompId());
IpEntity ipEntity = ipGraphDao.load(loginInformationType.getIpNumber());
LoginEntity loginEntity = new
LoginEntity(loginInformationType.getLoginDate());
loginGraphDao.save(loginEntity);
CookieLoginByEdge cookieLoginByEdge = new
CookieLoginByEdge(cookieEntity,loginEntity);
edges.add(cookieLoginByEdge);
IpLogInByEdge ipLogInByEdge = new IpLogInByEdge(ipEntity,loginEntity);
edges.add(ipLogInByEdge);
LoginByEdge loginByEdge = new LoginByEdge(loginEntity,userEntity);
edges.add(loginByEdge);
userRelationGraphDao.saveEdges(edges);
}
Save vertex is same for all vertex types. At the above Service I used it once
for Login Vertex.
@Override
public Vertex createVertex(String className, Map<String, Object>
vertexProperties) {
String vertexType = "class:" + className;
Vertex vertex =
orientDBFactory.getCurrentDb().addVertex(vertexType,vertexProperties);
return vertex;
}
Save edge is similar for all edges. I call following code for 3 times.
@Override
public void createOneDirectionalEdge(String className, Vertex from, Vertex to) {
orientDBFactory.getCurrentDb().addEdge("class:" + className, from, to,
className);
}
Load method is similar for all vertex types ip,cookie and user. I call it
three times.
public Vertex findOneVertex(String query, Map<String, Object> params) {
Vertex vertex = null;
OCommandRequest command = orientDBFactory.getCurrentDb().command(new
OCommandSQL(query));
for (Vertex v : (Iterable<Vertex>) command.execute(params)) {
vertex = v;
break;
}
return vertex;
}
For dao operations I need OrientGraphNoTx object so I call getCurrentDb().
This method gets OrientGraphNoTx object from thread local. I call this method.
I call it seven times.
public OrientBaseGraph getCurrentDb(){
OrientGraphNoTx tx = (OrientGraphNoTx)OrientBaseGraph.getActiveGraph();
log.debug("orientdbfactory hash" + tx.hashCode());
return tx;
}
Thank you again for your quick response.
7 Mart 2016 Pazartesi 20:12:29 UTC+2 tarihinde kurtuluş yılmaz yazdı:
>
> Hi;
> I am trying to migrate data from mysql to orientdb. I can insert 16
> million vertexes per hour with multiple threads and it is very impressive.
> After inserting vertexes I try to insert edges but it is very slow. I
> looked at internet but ı coulnt find any useful information. What is the
> best practice for massive insertion of EDGES. I send my configuration . Any
> help appreciated.
>
> OrientDb version : 2.1.11
> Transaction managemet = OrientdBNoTx
>
> OrientDB 2.1.11 (build 2.1.x@rddb5c0b4761473ae9549c3ac94871ab56ef5af2c;
> 2016-02-15 10:45:12+0000) configuration dump:
> - ENVIRONMENT
> + environment.dumpCfgAtStartup = true
> + environment.concurrent = true
> + environment.allowJVMShutdown = true
> - SCRIPT
> + script.pool.maxSize = 20
> - MEMORY
> + memory.useUnsafe = true
> + memory.directMemory.safeMode = true
> + memory.directMemory.trackMode = false
> + memory.directMemory.onlyAlignedMemoryAccess = true
> - JVM
> + jvm.gc.delayForOptimize = 600
> - STORAGE
> + storage.diskCache.pinnedPages = 20
> + storage.diskCache.bufferSize = 1256
> + storage.diskCache.writeCachePart = 15
> + storage.diskCache.writeCachePageTTL = 86400
> + storage.diskCache.writeCachePageFlushInterval = 25
> + storage.diskCache.writeCacheFlushInactivityInterval = 60000
> + storage.diskCache.writeCacheFlushLockTimeout = -1
> + storage.diskCache.diskFreeSpaceLimit = 100
> + storage.diskCache.diskFreeSpaceCheckInterval = 5
> + storage.configuration.syncOnUpdate = true
> + storage.compressionMethod = nothing
> + storage.useWAL = false
> + storage.wal.syncOnPageFlush = false
> + storage.wal.cacheSize = 3000
> + storage.wal.maxSegmentSize = 128
> + storage.wal.maxSize = 4096
> + storage.wal.commitTimeout = 1000
> + storage.wal.shutdownTimeout = 10000
> + storage.wal.fuzzyCheckpointInterval = 300
> + storage.wal.reportAfterOperationsDuringRestore = 10000
> + storage.wal.restore.batchSize = 50000
> + storage.wal.readCacheSize = 1000
> + storage.wal.fuzzyCheckpointShutdownWait = 600
> + storage.wal.fullCheckpointShutdownTimeout = 600
> + storage.wal.path = null
> + storage.makeFullCheckpointAfterCreate = true
> + storage.makeFullCheckpointAfterOpen = true
> + storage.makeFullCheckpointAfterClusterCreate = true
> + storage.diskCache.pageSize = 64
> + storage.lowestFreeListBound = 16
> + storage.cluster.usecrc32 = false
> + storage.lockTimeout = 0
> + storage.record.lockTimeout = 2000
> + storage.useTombstones = false
> - RECORD
> + record.downsizing.enabled = true
> - OBJECT
> + object.saveOnlyDirty = false
> - DB
> + db.pool.min = 1
> + db.pool.max = 50
> + db.pool.idleTimeout = 0
> + db.pool.idleCheckDelay = 0
> + db.mvcc.throwfast = false
> + db.validation = true
> - NONTX
> + nonTX.recordUpdate.synch = false
> + nonTX.clusters.sync.immediately = manindex
> - TX
> + tx.trackAtomicOperations = false
> - INDEX
> + index.embeddedToSbtreeBonsaiThreshold = 40
> + index.sbtreeBonsaiToEmbeddedThreshold = -1
> - HASHTABLE
> + hashTable.slitBucketsBuffer.length = 1500
> - INDEX
> + index.auto.synchronousAutoRebuild = true
> + index.auto.lazyUpdates = 10000
> + index.flushAfterCreate = true
> + index.manual.lazyUpdates = 1
> + index.durableInNonTxMode = false
> + index.txMode = FULL
> + index.cursor.prefetchSize = 500000
> - SBTREE
> + sbtree.maxDepth = 64
> + sbtree.maxKeySize = 10240
> + sbtree.maxEmbeddedValueSize = 40960
> - SBTREEBONSAI
> + sbtreebonsai.bucketSize = 2
> + sbtreebonsai.linkBagCache.size = 100000
> + sbtreebonsai.linkBagCache.evictionSize = 1000
> + sbtreebonsai.freeSpaceReuseTrigger = 0.5
> - RIDBAG
> + ridBag.embeddedDefaultSize = 4
> + ridBag.embeddedToSbtreeBonsaiThreshold = -1
> + ridBag.sbtreeBonsaiToEmbeddedToThreshold = -1
> - COLLECTIONS
> + collections.preferSBTreeSet = false
> - FILE
> + file.trackFileClose = false
> + file.lock = true
> + file.deleteDelay = 10
> + file.deleteRetry = 50
> - JNA
> + jna.disable.system.library = true
> - NETWORK
> + network.maxConcurrentSessions = 1000
> + network.socketBufferSize = 32768
> + network.lockTimeout = 15000
> + network.socketTimeout = 15000
> + network.requestTimeout = 3600000
> + network.retry = 5
> + network.retryDelay = 500
> + network.binary.loadBalancing.enabled = false
> + network.binary.loadBalancing.timeout = 2000
> + network.binary.maxLength = 32736
> + network.binary.readResponse.maxTimes = 20
> + network.binary.debug = false
> + network.http.maxLength = 1000000
> + network.http.charset = utf-8
> + network.http.jsonResponseError = true
> + network.http.jsonp = false
> - OAUTH2
> + oauth2.secretkey =
> - NETWORK
> + network.http.sessionExpireTimeout = 300
> + network.http.useToken = false
> + network.token.secretyKey =
> + network.token.encriptionAlgorithm = HmacSHA256
> + network.token.expireTimeout = 60
> - PROFILER
> + profiler.enabled = true
> + profiler.config = null
> + profiler.autoDump.interval = 0
> + profiler.maxValues = 200
> - LOG
> + log.console.level = info
> + log.file.level = fine
> - COMMAND
> + command.timeout = 0
> - QUERY
> + query.scanThresholdTip = 50000
> + query.limitThresholdTip = 10000
> - SQL
> + sql.graphConsistencyMode = notx_async_repair
> - CLIENT
> + client.channel.maxPool = 100
> + client.connectionPool.waitTimeout = 5000
> + client.channel.dbReleaseWaitTimeout = 10000
> + client.ssl.enabled = false
> + client.ssl.keyStore = null
> + client.ssl.keyStorePass = null
> + client.ssl.trustStore = null
> + client.ssl.trustStorePass = null
> + client.session.tokenBased = false
> - SERVER
> + server.channel.cleanDelay = 5000
> + server.cache.staticFile = false
> + server.log.dumpClientExceptionLevel = FINE
> + server.log.dumpClientExceptionFullStackTrace = false
> - DISTRIBUTED
> + distributed.crudTaskTimeout = 3000
> + distributed.commandTaskTimeout = 10000
> + distributed.commandLongTaskTimeout = 86400000
> + distributed.deployDbTaskTimeout = 1200000
> + distributed.deployChunkTaskTimeout = 15000
> + distributed.deployDbTaskCompression = 7
> + distributed.queueTimeout = 5000
> + distributed.asynchQueueSize = 0
> + distributed.asynchResponsesTimeout = 15000
> + distributed.purgeResponsesTimerDelay = 15000
> + distributed.queueMaxSize = 10000
> + distributed.backupDirectory = ../backup/databases
> + distributed.concurrentTxMaxAutoRetry = 10
> + distributed.concurrentTxAutoRetryDelay = 100
> - DB
> + db.makeFullCheckpointOnIndexChange = true
> + db.makeFullCheckpointOnSchemaChange = true
> + db.document.serializer = ORecordSerializerBinary
> - LAZYSET
> + lazyset.workOnStream = true
> - DB
> + db.mvcc = true
> + db.use.distributedVersion = false
> - MVRBTREE
> + mvrbtree.timeout = 0
> + mvrbtree.nodePageSize = 256
> + mvrbtree.loadFactor = 0.7
> + mvrbtree.optimizeThreshold = 100000
> + mvrbtree.entryPoints = 64
> + mvrbtree.optimizeEntryPointsFactor = 1.0
> + mvrbtree.entryKeysInMemory = false
> + mvrbtree.entryValuesInMemory = false
> + mvrbtree.ridBinaryThreshold = -1
> + mvrbtree.ridNodePageSize = 64
> + mvrbtree.ridNodeSaveMemory = false
> - TX
> + tx.commit.synch = false
> + tx.autoRetry = 1
> + tx.log.fileType = classic
> + tx.log.synch = false
> + tx.useLog = false
> - INDEX
> + index.auto.rebuildAfterNotSoftClose = true
> - CLIENT
> + client.channel.minPool = 1
> - STORAGE
> + storage.keepOpen = false
> - CACHE
> + cache.local.enabled = false
>
>
>
>
--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
