I have configured and run IGFS as Hadoop FileSystem successfully. There are three issues, that I couldn't solve. Any help will be appreciated!
1) Out of memory error when I try to load to IGFS 400Mb file 2) FileSystemConfiguration - metaCacheName and dataCacheName properties 3) includeEventTypes and task tasks execution statistics See details below Environment -MacOS Sierra -YARN 2.8 in Pseudo-distributed mode -Ignite 2.0 -JVM setting in ignite.sh JVM_OPTS="-server -Xms6g -Xmx6g -XX:NewSize=512m -XX:SurvivorRatio=6 -XX:+AlwaysPreTouch -XX:+UseG1GC -XX:MaxGCPauseMillis=2000 -XX:GCTimeRatio=4 -XX:InitiatingHeapOccupancyPercent=30 -XX:G1HeapRegionSize=8M -XX:ConcGCThreads=8 -XX:G1HeapWastePercent=10 -XX:+UseTLAB -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC" -Ignite configuration files https://github.com/ssemichev/ignite-configs/blob/master/igfs-hadoop-fs/igfs-hadoop-fs-config.xml https://github.com/ssemichev/ignite-configs/blob/master/igfs-hadoop-fs/ignite_conf/core-site.xml https://github.com/ssemichev/ignite-configs/blob/master/igfs-hadoop-fs/ignite_conf/hdfs-site.xml https://github.com/ssemichev/ignite-configs/blob/master/igfs-hadoop-fs/ignite_conf/mapred-site.xml ============================================= 1) Out of memory error when I try to load to IGFS 400Mb file I can copy a 60Mb file, but it fails when I try to copy large 410Mb file >./bin/ignite.sh ./configs/igfs-hadoop-fs/igfs-hadoop-fs-config.xml Ignite node started OK (id=9c9a20a5) Topology snapshot [ver=1, servers=1, clients=0, CPUs=8, heap=6.0GB] visor> top Hosts: 1 Summary: +-------------------------------------+ | Total hosts | 1 | | Total nodes | 1 | | Total CPUs | 8 | | Avg. CPU load | 0.17 % | | Avg. free heap | 95.00 % | | Avg. Up time | 00:05:46 | | Snapshot time | 05/30/17, 14:49:22 | visor> node | OS info | Mac OS X x86_64 10.12.5 | | OS user | ssemichev | | Deployment mode | SHARED | | Language runtime | Java Platform API Specification ver. 1.8 | | Ignite version | 2.0.0 | Busy time % | 4.07% | | Current CPU load % | 0.23% | | Average CPU load % | 0.42% | | Heap memory initialized | 6gb | | Heap memory used | 224mb | | Heap memory committed | 6gb | | Heap memory maximum | 6gb | | Non-heap memory initialized | 2mb | | Non-heap memory used | 51mb | | Non-heap memory committed | 52mb | | Non-heap memory maximum | 0 | | Current thread count | 55 | | Maximum thread count | 61 | | Total started thread count | 131 | | Current daemon thread count | 13 | >ls -alh /opt/hadoop-2.8.0.tar.gz 410M May 22 12:25 /opt/hadoop-2.8.0.tar.gz >hdfs --config ./ignite_conf dfs -put /opt/hadoop-2.8.0.tar.gz /in put: Failed to flush data during stream close [path=/in._COPYING_, fileInfo=IgfsFileInfo [len=0, blockSize=65536, lockId=fdc3caa5c51-9c9a20a5-2b17-460a-bec4-ea909f8e49a0, affKey=null, fileMap=IgfsFileMap [ranges=null], evictExclude=true]] >From Ignite terminal output [ERROR][igfs-igfs-ipc-#81%null%][GridNearTxLocal] Heuristic transaction failure. class org.apache.ignite.internal.transactions.IgniteTxHeuristicCheckedException: Failed to locally write to cache (all transaction entries will be invalidated, however there was a window when entries for this transaction were visible to others): GridNearTxLocal [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, colocatedLocallyMapped=true, needCheckBackup=null, hasRemoteLocks=false, thread=igfs-igfs-ipc-#81%null%, mappings=IgniteTxMappingsImpl [], super=GridDhtTxLocalAdapter [nearOnOriginatingNode=false, nearNodes=[], dhtNodes=[], explicitLock=false, super=IgniteTxLocalAdapter [completedBase=null, sndTransformedVals=false, depEnabled=false, txState=IgniteTxStateImpl [activeCacheIds=GridIntList [idx=1, arr=[-313518151]], txMap={IgniteTxKey [key=KeyCacheObjectImpl [part=1, val=cdc3caa5c51-0d6b159c-291a-4630-a948-dcd9fa98ec1c, hasValBytes=true], cacheId=-313518151]=IgniteTxEntry [key=KeyCacheObjectImpl [part=1, val=cdc3caa5c51-0d6b159c-291a-4630-a948-dcd9fa98ec1c, hasValBytes=true], cacheId=-313518151, txKey=IgniteTxKey [key=KeyCacheObjectImpl [part=1, val=cdc3caa5c51-0d6b159c-291a-4630-a948-dcd9fa98ec1c, hasValBytes=true], cacheId=-313518151], val=[op=TRANSFORM, val=CacheObjectImpl [val=IgfsFileInfo [len=0, blockSize=65536, lockId=fdc3caa5c51-9c9a20a5-2b17-460a-bec4-ea909f8e49a0, affKey=null, fileMap=IgfsFileMap [ranges=null], evictExclude=true], hasValBytes=true]], prevVal=[op=TRANSFORM, val=CacheObjectImpl [val=IgfsFileInfo [len=0, blockSize=65536, lockId=fdc3caa5c51-9c9a20a5-2b17-460a-bec4-ea909f8e49a0, affKey=null, fileMap=IgfsFileMap [ranges=null], evictExclude=true], hasValBytes=true]], oldVal=[op=NOOP, val=null], entryProcessorsCol=[IgniteBiTuple [val1=IgfsMetaFileUnlockProcessor [modificationTime=1496170357665, updateSpace=false, space=0, affRange=null], val2=[Ljava.lang.Object;@6a5ba764]], ttl=-1, conflictExpireTime=-1, conflictVer=null, explicitVer=null, dhtVer=null, filters=[], filtersPassed=false, filtersSet=true, entry=GridDhtColocatedCacheEntry [super=GridDhtCacheEntry [rdrs=[], locPart=GridDhtLocalPartition [rmvQueueMaxSize=256, rmvdEntryTtl=10000, id=1, store=org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl@6f95f794, lastApplied=0, shouldBeRenting=false, state=OWNING, reservations=0, empty=false, createTime=05/30/2017 14:43:42], super=GridDistributedCacheEntry [super=GridCacheMapEntry [key=KeyCacheObjectImpl [part=1, val=cdc3caa5c51-0d6b159c-291a-4630-a948-dcd9fa98ec1c, hasValBytes=true], val=CacheObjectImpl [val=IgfsFileInfo [len=0, blockSize=65536, lockId=fdc3caa5c51-9c9a20a5-2b17-460a-bec4-ea909f8e49a0, affKey=null, fileMap=IgfsFileMap [ranges=null], evictExclude=true], hasValBytes=true], startVer=1496169896316, ver=GridCacheVersion [topVer=107649824, order=1496169890081, nodeOrder=1], hash=-684243145, extras=GridCacheMvccEntryExtras [mvcc=GridCacheMvcc [locs=[GridCacheMvccCandidate [nodeId=9c9a20a5-2b17-460a-bec4-ea909f8e49a0, ver=GridCacheVersion [topVer=107649824, order=1496169896315, nodeOrder=1], threadId=209, id=1568, topVer=AffinityTopologyVersion [topVer=2, minorTopVer=0], reentry=null, otherNodeId=9c9a20a5-2b17-460a-bec4-ea909f8e49a0, otherVer=GridCacheVersion [topVer=107649824, order=1496169896315, nodeOrder=1], mappedDhtNodes=null, mappedNearNodes=null, ownerVer=null, serOrder=null, key=KeyCacheObjectImpl [part=1, val=cdc3caa5c51-0d6b159c-291a-4630-a948-dcd9fa98ec1c, hasValBytes=true], masks=local=1|owner=1|ready=1|reentry=0|used=0|tx=1|single_implicit=0|dht_local=1|near_local=0|removed=0|read=0, prevVer=null, nextVer=null]], rmts=null]], flags=2]]]], prepared=0, locked=true, nodeId=9c9a20a5-2b17-460a-bec4-ea909f8e49a0, locMapped=false, expiryPlc=null, transferExpiryPlc=false, flags=4, partUpdateCntr=0, serReadVer=null, xidVer=GridCacheVersion [topVer=107649824, order=1496169896315, nodeOrder=1]]}, recovery=false], super=IgniteTxAdapter [xidVer=GridCacheVersion [topVer=107649824, order=1496169896315, nodeOrder=1], writeVer=GridCacheVersion [topVer=107649824, order=1496169896317, nodeOrder=1], implicit=false, loc=true, threadId=209, startTime=1496170357663, nodeId=9c9a20a5-2b17-460a-bec4-ea909f8e49a0, startVer=GridCacheVersion [topVer=107649824, order=1496169896315, nodeOrder=1], endVer=null, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, timeout=0, sysInvalidate=false, sys=true, plc=2, commitVer=GridCacheVersion [topVer=107649824, order=1496169896315, nodeOrder=1], finalizing=NONE, invalidParts=null, state=COMMITTING, timedOut=false, topVer=AffinityTopologyVersion [topVer=2, minorTopVer=0], duration=0ms, onePhaseCommit=true], size=1]]] [ERROR][igfs-igfs-ipc-#81%null%][GridNearTxLocal] Failed to invalidate transaction entries while reverting a commit. class org.apache.ignite.internal.mem.IgniteOutOfMemoryException: Not enough memory allocated (consider increasing memory policy size or enabling evictions) [policyName=sysMemPlc, size=104.9 MB] After the failed attempt visor> node | Heap memory initialized | 6gb | | Heap memory used | 1gb | | Heap memory committed | 6gb | | Heap memory maximum | 6gb | | Non-heap memory initialized | 2mb | | Non-heap memory used | 60mb | | Non-heap memory committed | 61mb | | Non-heap memory maximum | 0 | | Current thread count | 63 | | Maximum thread count | 73 | | Total started thread count | 336 | | Current daemon thread count | 13 | >ls -alh /opt/apache-ignite-hadoop-2.0.0-bin.zip 56M May 23 13:10 /opt/apache-ignite-hadoop-2.0.0-bin.zip >hdfs --config ./ignite_conf dfs -put /opt/apache-ignite-hadoop-2.0.0-bin.zip / >hdfs --config ./ignite_conf dfs -ls -h / 55.7 M 2017-05-30 15:05 /apache-ignite-hadoop-2.0.0-bin.zip ============================================= 2) FileSystemConfiguration - metaCacheName and dataCacheName properties It seems that Ignite 2.0 doesn't have them. Do I need to configure a cache(s) for IGFS setup manually? How can I check how much memory is used for IGFS? Visor statistics is not very useful, it shows only total Heap and Non-heap values. ============================================= 3) includeEventTypes and task tasks execution statistics I added includeEventTypes section in the config https://github.com/ssemichev/ignite-configs/blob/master/igfs-hadoop-fs/igfs-hadoop-fs-config.xml When I run visor>tasks it prints the error in the ignite.sh terminal Failed to execute job [jobId=4ceffda5c51-05e150ae-87db-427c-8e68-e6bc83158476, ses=GridJobSessionImpl [ses=GridTaskSessionImpl [taskName=o.a.i.i.v.node.VisorNodeEventsCollectorTask, dep=LocalDeployment [super=GridDeployment [ts=1496171110859, depMode=SHARED, clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, clsLdrId=9ed5fba5c51-53c69937-a6b3-483d-ba88-0ba5c7f86d35, userVer=0, loc=true, sampleClsName=java.lang.String, pendingUndeploy=false, undeployed=false, usage=0]], taskClsName=o.a.i.i.v.node.VisorNodeEventsCollectorTask, sesId=3ceffda5c51-05e150ae-87db-427c-8e68-e6bc83158476, startTime=1496173301316, endTime=9223372036854775807, taskNodeId=05e150ae-87db-427c-8e68-e6bc83158476, clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, internal=true, subjId=05e150ae-87db-427c-8e68-e6bc83158476, mapFut=IgniteFuture [orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1743289220]], execName=null], jobId=4ceffda5c51-05e150ae-87db-427c-8e68-e6bc83158476]] class org.apache.ignite.IgniteException: Failed to query events because default no-op event storage SPI is used. Consider configuring MemoryEventStorageSpi or another EventStorageSpi implementation via IgniteConfiguration.setEventStorageSpi() configuration property. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IGFS-as-Hadoop-FileSystem-tp13249.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
