Hi,
Blow SQL reports error
select * from
onlinesessioncount_by_appversion_childapptypeid_countryid_networktype_parentapptypeid_regionid_sessiontimelevel_usertype
limit 10;
But SQL without asterisk is ok:
select timestamp from
onlinesessioncount_by_appversion_childapptypeid_countryid_networktype_parentapptypeid_regionid_sessiontimelevel_usertype
limit 10;
Zeppelin complains:
Failed resolve class for ID: -869249411
Ignite server report below errors:
[02:37:29,212][SEVERE][pub-#118%null%][GridJobWorker] Failed to execute job
[jobId=300f0750951-7c383220-1fe9-47b9-a101-aa273e72f0ba, ses=GridJobSessionImpl
[ses=GridTaskSessionImpl
[taskName=o.a.i.i.processors.cache.query.jdbc.GridCacheQueryJdbcTask,
dep=LocalDeployment [super=GridDeployment [ts=1481854956302, depMode=SHARED,
clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6,
clsLdrId=2e030750951-c9dac7eb-827c-4bb5-99d1-c347192977e0, userVer=0, loc=true,
sampleClsName=java.lang.String, pendingUndeploy=false, undeployed=false,
usage=0]],
taskClsName=o.a.i.i.processors.cache.query.jdbc.GridCacheQueryJdbcTask,
sesId=200f0750951-7c383220-1fe9-47b9-a101-aa273e72f0ba,
startTime=1481855849160, endTime=9223372036854775807,
taskNodeId=7c383220-1fe9-47b9-a101-aa273e72f0ba,
clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, closed=false, cpSpi=null,
failSpi=null, loadSpi=null, usage=1, fullSup=false, internal=false,
subjId=7c383220-1fe9-47b9-a101-aa273e72f0ba, mapFut=IgniteFuture
[orig=GridFutureAdapter [resFlag=0, res=null, startTime=1481855849202,
endTime=0, ignoreInterrupts=false, state=INIT]]],
jobId=300f0750951-7c383220-1fe9-47b9-a101-aa273e72f0ba]]
class org.apache.ignite.binary.BinaryObjectException: Failed resolve class for
ID: -869249411
at
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:696)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1491)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1450)
at
org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:637)
at
org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:142)
at
org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinary(CacheObjectContext.java:272)
at
org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinariesIfNeeded(CacheObjectContext.java:185)
at
org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinariesIfNeeded(CacheObjectContext.java:169)
at
org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinariesIfNeeded(GridCacheContext.java:1749)
at
org.apache.ignite.internal.processors.query.GridQueryCacheObjectsIterator.next(GridQueryCacheObjectsIterator.java:64)
at
org.apache.ignite.internal.processors.query.GridQueryCacheObjectsIterator.next(GridQueryCacheObjectsIterator.java:29)
at
org.apache.ignite.internal.processors.cache.query.jdbc.GridCacheQueryJdbcTask$JdbcDriverJob.execute(GridCacheQueryJdbcTask.java:308)
at
org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:556)
at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6564)
at
org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:550)
at
org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:479)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at
org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1180)
at
org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1894)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1082)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:710)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:102)
at
org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:673)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteCheckedException: Class definition was
not found at marshaller cache and local file. [id=-869249411,
file=/home/ec2-user/apache-ignite-fabric/work/marshaller/-869249411.classname]
at
org.apache.ignite.internal.MarshallerContextImpl.className(MarshallerContextImpl.java:218)
at
org.apache.ignite.internal.MarshallerContextAdapter.getClass(MarshallerContextAdapter.java:174)
at
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:680)
... 25 more
This is cache configuration:
Cache
'onlineviewcount_by_appversion_childapptypeid_countryid_networktype_parentapptypeid_regionid_streamtype_usertype_viewtimelevel(@c13)':
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Name(@) |
onlineviewcount_by_appversion_childapptypeid_countryid_networktype_parentapptypeid_regionid_streamtype_usertype_viewtimelevel(@c13)
|
| Nodes | 1
|
| Total size Min/Avg/Max | 191225 / 191225.00 / 191225
|
| Heap size Min/Avg/Max | 191225 / 191225.00 / 191225
|
| Off-heap size Min/Avg/Max | 0 / 0.00 / 0
|
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Nodes for:
onlineviewcount_by_appversion_childapptypeid_countryid_networktype_parentapptypeid_regionid_streamtype_usertype_viewtimelevel(@c13)
+===============================================================================================================+
| Node ID8(@), IP | CPUs | Heap Used | CPU Load | Up Time |
Size | Hi/Mi/Rd/Wr |
+===============================================================================================================+
| C9DAC7EB(@n0), 172.31.33.77 | 8 | 8.45 % | 0.20 % | 00:18:55:843 |
Total: 191225 | Hi: 0 |
| | | | | |
Heap: 191225 | Mi: 0 |
| | | | | |
Off-Heap: 0 | Rd: 0 |
| | | | | |
Off-Heap Memory: 0 | Wr: 0 |
+---------------------------------------------------------------------------------------------------------------+
'Hi' - Number of cache hits.
'Mi' - Number of cache misses.
'Rd' - number of cache reads.
'Wr' - Number of cache writes.
Aggregated queries metrics:
Minimum execution time: 00:00:00:000
Maximum execution time: 00:00:00:000
Average execution time: 00:00:00:000
Total number of executions: 0
Total number of failures: 0
Cache configuration:
+==========================================================================================================+
| Name | Value
|
+==========================================================================================================+
| Mode | PARTITIONED
|
| Atomicity Mode | ATOMIC
|
| Atomic Write Ordering Mode | PRIMARY
|
| Statistic Enabled | off
|
| Management Enabled | off
|
| Time To Live Eager Flag | true
|
| Write Synchronization Mode | PRIMARY_SYNC
|
| Swap Enabled | off
|
| Invalidate | off
|
| Start Size | 10000
|
| Affinity Function |
o.a.i.cache.affinity.rendezvous.RendezvousAffinityFunction |
| Affinity Backups | 0
|
| Affinity Partitions | 1024
|
| Affinity Exclude Neighbors | false
|
| Affinity Mapper |
o.a.i.i.processors.cache.CacheDefaultBinaryAffinityKeyMapper |
| Rebalance Mode | ASYNC
|
| Rebalance Batch Size | 524288
|
| Rebalance Thread Pool size | 2
|
| Rebalance Timeout | 10000
|
| Rebalance Delay | 0
|
| Time Between Rebalance Messages | 0
|
| Eviction Policy Enabled | off
|
| Eviction Policy | <n/a>
|
| Eviction Policy Max Size | <n/a>
|
| Eviction Filter | <n/a>
|
| Eviction Key Buffer Size | 1024
|
| Eviction Synchronized | off
|
| Eviction Overflow Ratio | 10.0
|
| Synchronous Eviction Timeout | 10000
|
| Synchronous Eviction Concurrency Level | 4
|
| Near Cache Enabled | off
|
| Near Start Size | 0
|
| Near Eviction Policy | <n/a>
|
| Near Eviction Policy Max Size | <n/a>
|
| Default Lock Timeout | 0
|
| Metadata type count | 1
|
| Cache Interceptor | <n/a>
|
| Store Enabled | on
|
| Store Class |
com.neulion.qos.ignite.store.MetricCacheStore |
| Store Factory Class |
com.neulion.qos.ignite.store.MetricCacheStoreFactory |
| Store Keep Binary | true
|
| Store Read Through | off
|
| Store Write Through | on
|
| Write-Behind Enabled | on
|
| Write-Behind Flush Size | 10240
|
| Write-Behind Frequency | 5000
|
| Write-Behind Flush Threads Count | 1
|
| Write-Behind Batch Size | 512
|
| Concurrent Asynchronous Operations Number | 500
|
| Memory Mode | ONHEAP_TIERED
|
| Off-Heap Size | <n/a>
|
| Loader Factory Class Name | <n/a>
|
| Writer Factory Class Name | <n/a>
|
| Expiry Policy Factory Class Name |
javax.cache.configuration.FactoryBuilder$SingletonFactory |
| Query Execution Time Threshold | 3000
|
| Query Schema Name |
|
| Query Escaped Names | off
|
| Query Onheap Cache Size | 10240
|
| Query SQL functions | <n/a>
|
| Query Indexed Types | <n/a>
|
+----------------------------------------------------------------------------------------------------------+
Thanks
Shawn
发件人: Andrey Mashenkov [mailto:[email protected]]
发送时间: 2016年12月15日 18:21
收件人: [email protected]
主题: Re: 答复: 答复: query binaryobject cache in zeppelin
Hi Shawn,
I've got Zeppelin sources, rewrite test query () with arterisk
(org.apache.zeppelin.ignite.IgniteSqlInterpreterTest). It works fine for me as
with ignite 1.7 as 1.8.
Would you please share sql query that failed, jdbc connection string that used
by zeppelin, ignite cache configuration and also ignite logs if possible?
On Tue, Dec 13, 2016 at 3:34 AM, Shawn Du <[email protected]
<mailto:[email protected]> > wrote:
Hi Andrey,
I had code to add entities to configuration. It seems that it doesn’t support
using Asterisk in select clause.
If I use column names, zeppelin can show the data.
Also some background about the testing:
Using ignite 1.8 and zeppelin 0.6.2. in order to make zeppelin work with
ignite 1.8, I had to build from source and change pom.xml to use ignite 1.8.0.
All Seems good.
Thanks
Shawn
发件人: Andrey Mashenkov [mailto:[email protected]
<mailto:[email protected]> ]
发送时间: 2016年12月12日 18:54
收件人: [email protected] <mailto:[email protected]>
主题: Re: 答复: query binaryobject cache in zeppelin
Hi Shawn,
Looks strange that Query Indexed Types is "n/a"
Are you forget to add query entities to configuration? I can't see in your
code: cacheCfg.setQueryEntities(Arrays.asList(entity))
On Mon, Dec 12, 2016 at 1:41 PM, Shawn Du <[email protected]
<mailto:[email protected]> > wrote:
Hi,
Just this is part of my cache configuration. See the red part. Query schema
Name is empty, it is so-called *table name*?
|Store Write Through | off
|
| Write-Behind Enabled | off
|
| Write-Behind Flush Size | 10240
|
| Write-Behind Frequency | 5000
|
| Write-Behind Flush Threads Count | 1
|
| Write-Behind Batch Size | 512
|
| Concurrent Asynchronous Operations Number | 500
|
| Memory Mode | ONHEAP_TIERED
|
| Off-Heap Size | <n/a>
|
| Loader Factory Class Name | <n/a>
|
| Writer Factory Class Name | <n/a>
|
| Expiry Policy Factory Class Name |
javax.cache.configuration.FactoryBuilder$SingletonFactory |
| Query Execution Time Threshold | 3000
|
| Query Schema Name |
|
| Query Escaped Names | off
|
| Query Onheap Cache Size | 10240
|
| Query SQL functions | <n/a>
|
| Query Indexed Types | <n/a>
|
+----------------------------------------------------------------------------------------------------------+
I create queryEntity by the following code, ColumnScheme is my own class, just
contains information column name/type and ensureIndex or not.
Is the entity value type is “Table Name”? I set it the same with the cache
name. I try to use this as the table name in SQL, still not work. Please
help. Thanks.
QueryEntity entity = new QueryEntity();
entity.setKeyType(keyType);
entity.setValueType(valueType);
LinkedHashMap<String, String> fields = new LinkedHashMap<>();
List<QueryIndex> indexes = new ArrayList<>();
for (ColumnScheme columnScheme : columns)
{
fields.put(columnScheme.getName(), columnScheme.getType());
if (columnScheme.isEnsureIndex())
{
indexes.add(new QueryIndex(columnScheme.getName()));
}
}
entity.setIndexes(indexes);
entity.setFields(fields);
Shawn
发件人: Andrey Mashenkov [mailto:[email protected]
<mailto:[email protected]> ]
发送时间: 2016年12月12日 17:24
收件人: [email protected] <mailto:[email protected]>
主题: Re: query binaryobject cache in zeppelin
Hi Shawn,
Classes that you want to use in queries should be set via setQueryEntities.
*Table name* in Ignite has name of type. E.g if you want to get some record of
class "my.org.Person" you should use "Person" as table name: Select * from
Person.
To make cross cache query you should use full table name as
"cache_name".class_name: Select ... from Person, "other_cache".Org Where ...
For JDK classes like java.lang.Integer, table name will be "Integer"
On Mon, Dec 12, 2016 at 12:09 PM, Shawn Du <[email protected]
<mailto:[email protected]> > wrote:
Hi,
Today, I had a try of zeppelin. After setup a zeppelin node and began to issue
SQL for query, I don’t know how to input the *table name* of SQL.
For my cache are built with BinaryObject and config with setQueryEntities.
It is possible to do query in zeppinlin for these caches?
Thanks
Shawn
--
С уважением,
Машенков Андрей Владимирович
Тел. +7-921-932-61-82
Best regards,
Andrey V. Mashenkov
Cerr: +7-921-932-61-82
--
С уважением,
Машенков Андрей Владимирович
Тел. +7-921-932-61-82
Best regards,
Andrey V. Mashenkov
Cerr: +7-921-932-61-82
