Hello! I think that your CacheStore implementation is confused by nested fields or binary object values (what is the type of apn_id?). Consider using CacheJdbcBlobStoreFactory instead which will serialize value to one big field in BinaryObject formar.
Regards, -- Ilya Kasnacheev ср, 23 янв. 2019 г. в 15:47, Premachandran, Mahesh (Nokia - IN/Bangalore) < [email protected]>: > Hi all, > > > > I am trying to stream some data from Kafka to Ignite using > IgniteDataStreamer and use 3rd party persistence to move it to HIVE. The > data on Kafka is in avro format, which I am deserailising, populating an > Ignite BinaryObject using the binary builder and pushing it to Ignite. It > works well when I do not enable 3rd party persistence, but once that is > enabled, it throws the following exception. > > > > [12:32:07] (err) Failed to execute compound future reducer: > GridCompoundFuture [rdc=null, initFlag=1, lsnrCalls=2, done=true, > cancelled=false, err=class o.a.i.IgniteCheckedException: DataStreamer > request failed [node=292ab229-61fb-4d61-8f08-33c8abd310a2], futs=[true, > true, true]]class org.apache.ignite.IgniteCheckedException: DataStreamer > request failed [node=292ab229-61fb-4d61-8f08-33c8abd310a2] > > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1912) > > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:346) > > at > org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556) > > at > org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184) > > at > org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125) > > at > org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091) > > at > org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:511) > > at java.lang.Thread.run(Thread.java:748) > > Caused by: javax.cache.integration.CacheWriterException: class > org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException: > Failed to update keys (retry update if possible).: [2] > > at > org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1280) > > at > org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1734) > > at > org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1087) > > at > org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:788) > > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerCacheUpdaters$Individual.receive(DataStreamerCacheUpdaters.java:121) > > at > org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:140) > > at > org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.localUpdate(DataStreamProcessor.java:400) > > at > org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.processRequest(DataStreamProcessor.java:305) > > at > org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.access$000(DataStreamProcessor.java:60) > > at > org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$1.onMessage(DataStreamProcessor.java:90) > > ... 6 more > > Caused by: class > org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException: > Failed to update keys (retry update if possible).: [2] > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:397) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:253) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:303) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:300) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.map(GridDhtAtomicAbstractUpdateFuture.java:390) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1805) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1628) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:299) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:483) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:443) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:248) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1117) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put0(GridDhtAtomicCache.java:606) > > at > org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2372) > > at > org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2349) > > at > org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1084) > > ... 13 more > > Suppressed: class org.apache.ignite.IgniteCheckedException: Failed > to update keys. > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.UpdateErrors.addFailedKey(UpdateErrors.java:108) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateResponse.addFailedKey(GridNearAtomicUpdateResponse.java:329) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2560) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:1883) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1736) > > ... 23 more > > Suppressed: class > org.apache.ignite.IgniteCheckedException: Runtime failure on search row: > org.apache.ignite.internal.processors.cache.tree.SearchRow@78ca4051 > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1637) > > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1249) > > at > org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:1529) > > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:352) > > at > org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:1767) > > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2421) > > ... 25 more > > Caused by: class org.apache.ignite.IgniteCheckedException: > javax.cache.CacheException: Failed to set statement parameter name: apn_id > > at > org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.put(GridCacheStoreManagerAdapter.java:597) > > at > org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.update(GridCacheMapEntry.java:4927) > > at > org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:4746) > > at > org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:4460) > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:3083) > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$6200(BPlusTree.java:2977) > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1726) > > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1610) > > ... 30 more > > Caused by: javax.cache.integration.CacheWriterException: > javax.cache.CacheException: Failed to set statement parameter name: apn_id > > ... 38 more > > Caused by: javax.cache.CacheException: Failed to set > statement parameter name: apn_id > > at > org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.fillParameter(CacheAbstractJdbcStore.java:1391) > > at > org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.fillValueParameters(CacheAbstractJdbcStore.java:1443) > > at > org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.writeUpsert(CacheAbstractJdbcStore.java:919) > > at > org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.write(CacheAbstractJdbcStore.java:1027) > > at > org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.put(GridCacheStoreManagerAdapter.java:586) > > ... 37 more > > Caused by: java.sql.SQLException: Can't infer the SQL type > to use for an instance of > org.apache.ignite.internal.binary.BinaryObjectImpl. Use setObject() with an > explicit Types value to specify the type to use. > > at > org.apache.hive.jdbc.HivePreparedStatement.setObject(HivePreparedStatement.java:624) > > at > org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.fillParameter(CacheAbstractJdbcStore.java:1385) > > ... 41 more > > > > > > > > Is this a configuration mistake on my end? I used Ignite Web Console to > get the config XML to create the table on Ignite and connect to hive. > > > > > > Regards, > > Mahesh > > > > >
