Re: Two network interface on os
Hi Valentin, Your suggestion fits my needs. Thank you very much. Tolga On Tue, Oct 4, 2016 at 1:49 AM, vkulichenko wrote: > Hi, > > You can set IgniteConfiguration.localHost property to 172.10.2.1. This will > force Ignite to bind only to this interface. > > -Val > > > > -- > View this message in context: http://apache-ignite-users. > 70518.x6.nabble.com/Two-network-interface-on-os-tp8058p8061.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. > -- *Tolga KAVUKÇU*
Re: Accessing igfs file through java
You need to configure the igfs in the HDFS configuration file. Then you use the
standard APIs to access HDFS files and it will go automatically through the
cache.
> On 4 Oct 2016, at 07:35, Sateesh Karuturi wrote:
>
> Hello experts,
> I am new to the Apache Ignite , and i need to access igfs file through java.
> Till now what i tried is :
>
> import java.io.InputStream;
>
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteFileSystem;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.igfs.IgfsPath;
>
> public class Ignite_Spark {
>
> public static void main(String[] args) throws Exception {
> Ignite ignite =
> Ignition.start("/usr/local/hadoop/apache-ignite-hadoop-1.6.0-bin/config/default-config.xml");
> Ignite ignites = Ignition.ignite();
>
> IgniteFileSystem fs = ignites.fileSystem("igfs");
>
> IgfsPath workDir = new
> IgfsPath("igfs://localhost:8020/ignite/fs");
> fs.mkdirs(workDir);
>
>
> IgfsPath filePath = new IgfsPath(workDir, "ttypes.csv");
>
> try (InputStream in = fs.open(filePath)) {
> in.read();
> }
> }
>
> }
>
>
> But getting java.lang.IllegalArgumentException: Path should be absolute:
> igfs://localhost:8020/ignite/fs exception.
> anyone please help me out.
> Thanks .
>
Re: Delay in obtaining connection to Ignite server
Hi, Thanks for the response. Will try out the suggestions provided. OS - CentOS 7. Should we look into any specific OS configuration ? Thanks Ganesh -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Delay-in-obtaining-connection-to-Ignite-server-tp8035p8074.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Accessing igfs file through java
Hello experts,
I am new to the Apache Ignite , and i need to access igfs file through java.
Till now what i tried is :
import java.io.InputStream;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteFileSystem;
import org.apache.ignite.Ignition;
import org.apache.ignite.igfs.IgfsPath;
public class Ignite_Spark {
public static void main(String[] args) throws Exception {
Ignite ignite =
Ignition.start("/usr/local/hadoop/apache-ignite-hadoop-1.6.0-bin/config/default-config.xml");
Ignite ignites = Ignition.ignite();
IgniteFileSystem fs = ignites.fileSystem("igfs");
IgfsPath workDir = new IgfsPath("igfs://localhost:8020/ignite/fs");
fs.mkdirs(workDir);
IgfsPath filePath = new IgfsPath(workDir, "ttypes.csv");
try (InputStream in = fs.open(filePath)) {
in.read();
}
}
}
But getting java.lang.IllegalArgumentException: Path should be absolute:
igfs://localhost:8020/ignite/fs exception.
anyone please help me out.
Thanks .
Re: spark SQL thriftserver over ignite and cassandra
I am not sure that this will be performant. What do you want to achieve here?
Fast lookups? Then the Cassandra Ignite store might be the right solution. If
you want to do more analytic style of queries then you can put the data on
HDFS/Hive and use the Ignite HDFS cache to cache certain partitions/tables in
Hive in-memory. If you want to go to iterative machine learning algorithms you
can go for Spark on top of this. You can use then also Ignite cache for Spark
RDDs.
> On 4 Oct 2016, at 02:24, Alexey Kuznetsov wrote:
>
> Hi, Vincent!
>
> Ignite also has SQL support (also scalable), I think it will be much faster
> to query directly from Ignite than query from Spark.
> Also please mind, that before executing queries you should load all needed
> data to cache.
> To load data from Cassandra to Ignite you may use Cassandra store [1].
>
> [1] https://apacheignite.readme.io/docs/ignite-with-apache-cassandra
>
>> On Tue, Oct 4, 2016 at 4:19 AM, vincent gromakowski
>> wrote:
>> Hi,
>> I am evaluating the possibility to use Spark SQL (and its scalability) over
>> an Ignite cache with Cassandra persistent store to increase read workloads
>> like OLAP style analytics.
>> Is there any way to configure Spark thriftserver to load an external table
>> in Ignite like we can do in Cassandra ?
>> Here is an example of config for spark backed by cassandra
>>
>> CREATE EXTERNAL TABLE MyHiveTable
>> ( id int, data string )
>> STORED BY 'org.apache.hadoop.hive.cassandra.cql.CqlStorageHandler'
>> TBLPROPERTIES ("cassandra.host" = "x.x.x.x", "cassandra.ks.name" =
>> "test" ,
>> "cassandra.cf.name" = "mytable" ,
>> "cassandra.ks.repfactor" = "1" ,
>> "cassandra.ks.strategy" =
>> "org.apache.cassandra.locator.SimpleStrategy" );
>>
>
>
>
> --
> Alexey Kuznetsov
Re: Performance issues with Ignite L2 cache compared to ehcache
Hi, If you compare local ehcache and distributed Ignite, then it's not really fair. Ignite gives you the biggest improvement when you need to scale out and fit more data in memory than one computer allows. If memory of one node is enough for you, start with a single server node embedded into the application and measure performance. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Performance-issues-with-Ignite-L2-cache-compared-to-ehcache-tp8069p8071.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Performance issues with Ignite L2 cache compared to ehcache
Hi! Could you share with us: 1) How many hosts you are using for starting Ignite? 2) How many Ignite nodes you are staring? 3) Ignite cache config. 4) Ignite version. 5) How often data changed in cache? May be Near Cache may help in your case? See [1] section "Near Caches on Server Nodes" [1] https://apacheignite.readme.io/docs/near-caches On Tue, Oct 4, 2016 at 7:40 AM, Himetic wrote: > So I've got the L2 ignite cache working, confirmed with beans, logging, > etc, > and we're comparing it to our existing ehcache implementation. > Unfortunately, so far it seems to be running far fewer operations per > second > than the ehcache version. However, it does seem like the timing on each > cache query (we query by id) is fairly fast, around 5ms per query. > > My suspicion is that it's because of how we're doing the batch queries - we > get the ids from querying the database with whatever parameters we're > getting from the web api, and then we go through each id and try the cache, > pooling the rest into a big database query. I'm guessing that, since > ignite > has the overhead of a network call, even if it's relatively minor when > querying for a single id, it could add up to a big delay when querying for > hundreds of IDs, compared to hitting local ehcache, even if the ehcache has > fewer hits and results in a larger query against the database ultimately. > > Is there a way to mitigate this problem? Maybe a way to do a batch query > against the L2 database? I'm guessing not, but otherwise I'm not sure what > else we could do to avoid these performance issues. > > > > -- > View this message in context: http://apache-ignite-users. > 70518.x6.nabble.com/Performance-issues-with-Ignite-L2-cache-compared-to- > ehcache-tp8069.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. > -- Alexey Kuznetsov
Performance issues with Ignite L2 cache compared to ehcache
So I've got the L2 ignite cache working, confirmed with beans, logging, etc, and we're comparing it to our existing ehcache implementation. Unfortunately, so far it seems to be running far fewer operations per second than the ehcache version. However, it does seem like the timing on each cache query (we query by id) is fairly fast, around 5ms per query. My suspicion is that it's because of how we're doing the batch queries - we get the ids from querying the database with whatever parameters we're getting from the web api, and then we go through each id and try the cache, pooling the rest into a big database query. I'm guessing that, since ignite has the overhead of a network call, even if it's relatively minor when querying for a single id, it could add up to a big delay when querying for hundreds of IDs, compared to hitting local ehcache, even if the ehcache has fewer hits and results in a larger query against the database ultimately. Is there a way to mitigate this problem? Maybe a way to do a batch query against the L2 database? I'm guessing not, but otherwise I'm not sure what else we could do to avoid these performance issues. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Performance-issues-with-Ignite-L2-cache-compared-to-ehcache-tp8069.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: spark SQL thriftserver over ignite and cassandra
Hi, Vincent!
Ignite also has SQL support (also scalable), I think it will be much faster
to query directly from Ignite than query from Spark.
Also please mind, that before executing queries you should load all needed
data to cache.
To load data from Cassandra to Ignite you may use Cassandra store [1].
[1] https://apacheignite.readme.io/docs/ignite-with-apache-cassandra
On Tue, Oct 4, 2016 at 4:19 AM, vincent gromakowski <
vincent.gromakow...@gmail.com> wrote:
> Hi,
> I am evaluating the possibility to use Spark SQL (and its scalability)
> over an Ignite cache with Cassandra persistent store to increase read
> workloads like OLAP style analytics.
> Is there any way to configure Spark thriftserver to load an external table
> in Ignite like we can do in Cassandra ?
> Here is an example of config for spark backed by cassandra
>
> CREATE EXTERNAL TABLE MyHiveTable
> ( id int, data string )
> STORED BY 'org.apache.hadoop.hive.cassandra.cql.CqlStorageHandler'
>
> TBLPROPERTIES ("cassandra.host" = "x.x.x.x", "cassandra.ks.name"
> = "test" ,
> "cassandra.cf.name" = "mytable" ,
> "cassandra.ks.repfactor" = "1" ,
> "cassandra.ks.strategy" =
> "org.apache.cassandra.locator.SimpleStrategy" );
>
>
--
Alexey Kuznetsov
Re: Configuring ignite distributed hibernate cache
Denis, Agree, I updated the doc. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Configuring-ignite-distributed-hibernate-cache-tp7975p8067.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Cluster Communication with SSH Tunnels
Hi Krzysztof, This issue was fixed some time ago, so in 1.8 Ignite will publish public addresses in the IP finder. You can with the nightly build [1] in the meantime. [1] https://cwiki.apache.org/confluence/display/IGNITE/Nightly+Builds -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cluster-Communication-with-SSH-Tunnels-tp273p8066.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Couchbase as persistent store
What is the exception now? -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Couchbase-as-persistent-store-tp7476p8065.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: [Fatal Error] :42:19: The element type "p" must be terminated by the matching end-tag "".
Well, from what I see in the logs, Ignite node is up and running without issues. Looks like there is Hibernate associated with Ignite, and I guess it's coming from there. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Fatal-Error-42-19-The-element-type-p-must-be-terminated-by-the-matching-end-tag-p-tp7911p8064.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Finding Key to Node Mapping
Alper, It has to go through discovery to know the topology. This is the heaviest part and there is no way to avoid this, at least within current architecture. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Finding-Key-to-Node-Mapping-tp7653p8063.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Data Streamer error
Hi Matt, You need to figure out why JMSException is serialized. From what I see in the trace, it's used as a value in a LinkedHashMap. I think you should use debugger to trace the whole serialization stack and see where this is coming from. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-Streamer-error-tp7725p8062.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Two network interface on os
Hi, You can set IgniteConfiguration.localHost property to 172.10.2.1. This will force Ignite to bind only to this interface. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Two-network-interface-on-os-tp8058p8061.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
spark SQL thriftserver over ignite and cassandra
Hi,
I am evaluating the possibility to use Spark SQL (and its scalability) over
an Ignite cache with Cassandra persistent store to increase read workloads
like OLAP style analytics.
Is there any way to configure Spark thriftserver to load an external table
in Ignite like we can do in Cassandra ?
Here is an example of config for spark backed by cassandra
CREATE EXTERNAL TABLE MyHiveTable
( id int, data string )
STORED BY 'org.apache.hadoop.hive.cassandra.cql.CqlStorageHandler'
TBLPROPERTIES ("cassandra.host" = "x.x.x.x", "cassandra.ks.name" =
"test" ,
"cassandra.cf.name" = "mytable" ,
"cassandra.ks.repfactor" = "1" ,
"cassandra.ks.strategy" =
"org.apache.cassandra.locator.SimpleStrategy" );
Re: Two network interface on os
you may be setting the ip static, as follows ... 1.2.3.4 1.2.3.5:47500..47509 reference http://apacheignite.gridgain.org/v1.1/docs/cluster-config -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Two-network-interface-on-os-tp8058p8059.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Two network interface on os
Hi everyone, When initializing topology how can i force ignite to use one of my network interface cards. Lets say i have two interfaces eth0 192.168.1.103 eth1 172.10.2.1 I want to use only eth1. From other node when i call node.addresses() i want to get only one ip. Thanks. -- *Tolga KAVUKÇU*
Re: Unexpected flag value
Dmitry,
Could you please provide reproduced example?
On Mon, Oct 3, 2016 at 6:31 PM, dmitry.parkhonin
wrote:
> Vladislav,
>
> The same exception is on the server side:
>
> 2016-10-03 16:01:14,301 DEBUG - Loaded class [cls=ru.depsy.TaskOutput,
> ldr=GridDeploymentClassLoader
> [id=982001b8751-37bc0767-6310-4473-964a-e329e88fed8b, singleNode=false,
> nodeLdrMap={2a43ed8f-e226-47dc-b693-c8451f9c0ff9=
> 8bf001b8751-2a43ed8f-e226-47dc-b693-c8451f9c0ff9},
> p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false]]
> [org.apache.ignite.internal.managers.deployment.
> GridDeploymentPerVersionStore]
> [pub-#7%pricingGridServer%] {}
> 2016-10-03 16:01:14,316 ERROR - Failed to execute job due to unexpected
> runtime exception [jobId=3cf001b8751-2a43ed8f-e226-47dc-b693-c8451f9c0ff9,
> ses=GridJobSessionImpl [ses=GridTaskSessionImpl
> [taskName=ru.depsy.ignite.IgniteSession$$Lambda$2/16415465,
> dep=SharedDeployment [rmv=false, super=GridDeployment [ts=1475506874176,
> depMode=SHARED, clsLdr=GridDeploymentClassLoader
> [id=982001b8751-37bc0767-6310-4473-964a-e329e88fed8b, singleNode=false,
> nodeLdrMap={2a43ed8f-e226-47dc-b693-c8451f9c0ff9=
> 8bf001b8751-2a43ed8f-e226-47dc-b693-c8451f9c0ff9},
> p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false],
> clsLdrId=982001b8751-37bc0767-6310-4473-964a-e329e88fed8b, userVer=0,
> loc=false, sampleClsName=ru.depsy.ignite.IgniteSession$$Lambda$2/16415465,
> pendingUndeploy=false, undeployed=false, usage=1]],
> taskClsName=ru.depsy.ignite.IgniteSession$$Lambda$2/16415465,
> sesId=2cf001b8751-2a43ed8f-e226-47dc-b693-c8451f9c0ff9,
> startTime=1475506873427, endTime=9223372036854775807,
> taskNodeId=2a43ed8f-e226-47dc-b693-c8451f9c0ff9,
> clsLdr=GridDeploymentClassLoader
> [id=982001b8751-37bc0767-6310-4473-964a-e329e88fed8b, singleNode=false,
> nodeLdrMap={2a43ed8f-e226-47dc-b693-c8451f9c0ff9=
> 8bf001b8751-2a43ed8f-e226-47dc-b693-c8451f9c0ff9},
> p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false], closed=false,
> cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false,
> subjId=2a43ed8f-e226-47dc-b693-c8451f9c0ff9, mapFut=IgniteFuture
> [orig=GridFutureAdapter [resFlag=0, res=null, startTime=1475506874207,
> endTime=0, ignoreInterrupts=false, state=INIT]]],
> jobId=3cf001b8751-2a43ed8f-e226-47dc-b693-c8451f9c0ff9]]
> [org.apache.ignite.internal.processors.job.GridJobWorker]
> [pub-#7%pricingGridServer%] {}
> class org.apache.ignite.binary.BinaryObjectException: Unexpected flag
> value
> [pos=1296, expected=3, actual=-2]
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.checkFlagNoHandles(
> BinaryReaderExImpl.java:1423)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.
> readInt(BinaryReaderExImpl.java:658)
> at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$
> IntPrimitiveAccessor.read(BinaryFieldAccessor.java:321)
> at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.
> read(BinaryClassDescriptor.java:776)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(
> BinaryReaderExImpl.java:1481)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.
> readField(BinaryReaderExImpl.java:1683)
> at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$
> DefaultFinalClassAccessor.read(BinaryFieldAccessor.java:639)
> at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.
> read(BinaryClassDescriptor.java:776)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(
> BinaryReaderExImpl.java:1481)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.
> readField(BinaryReaderExImpl.java:1683)
> at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$
> DefaultFinalClassAccessor.read(BinaryFieldAccessor.java:639)
> at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.
> read(BinaryClassDescriptor.java:776)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(
> BinaryReaderExImpl.java:1481)
> at
> org.apache.ignite.internal.binary.BinaryUtils.
> doReadObject(BinaryUtils.java:1608)
> at
> org.apache.ignite.internal.binary.BinaryUtils.deserializeOrUnmarshal(
> BinaryUtils.java:1955)
> at
> org.apache.ignite.internal.binary.BinaryUtils.
> doReadObjectArray(BinaryUtils.java:1819)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.readObjectArray(
> BinaryReaderExImpl.java:1149)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.readObjectArray(
> BinaryReaderExImpl.java:1142)
> at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$
> DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:792)
> at
> org.apache.ignite.internal.binary.BinaryFieldAccessor$
> DefaultFinalClassAccessor.read(BinaryFieldAccessor.java:639)
> at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.
> read(BinaryClassDescriptor.java:776)
> at
> org.apache.ignite.internal.
Re: CLUSTER GET HANGED SUDDENLY
Hi, Lockups may be caused that some locks may not be released. The discussion moved to chatter. Let community know about solution, if it you solved already... -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/CLUSTER-GET-HANGED-SUDDENLY-tp8027p8056.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Unexpected flag value
Vladislav,
The same exception is on the server side:
2016-10-03 16:01:14,301 DEBUG - Loaded class [cls=ru.depsy.TaskOutput,
ldr=GridDeploymentClassLoader
[id=982001b8751-37bc0767-6310-4473-964a-e329e88fed8b, singleNode=false,
nodeLdrMap={2a43ed8f-e226-47dc-b693-c8451f9c0ff9=8bf001b8751-2a43ed8f-e226-47dc-b693-c8451f9c0ff9},
p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false]]
[org.apache.ignite.internal.managers.deployment.GridDeploymentPerVersionStore]
[pub-#7%pricingGridServer%] {}
2016-10-03 16:01:14,316 ERROR - Failed to execute job due to unexpected
runtime exception [jobId=3cf001b8751-2a43ed8f-e226-47dc-b693-c8451f9c0ff9,
ses=GridJobSessionImpl [ses=GridTaskSessionImpl
[taskName=ru.depsy.ignite.IgniteSession$$Lambda$2/16415465,
dep=SharedDeployment [rmv=false, super=GridDeployment [ts=1475506874176,
depMode=SHARED, clsLdr=GridDeploymentClassLoader
[id=982001b8751-37bc0767-6310-4473-964a-e329e88fed8b, singleNode=false,
nodeLdrMap={2a43ed8f-e226-47dc-b693-c8451f9c0ff9=8bf001b8751-2a43ed8f-e226-47dc-b693-c8451f9c0ff9},
p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false],
clsLdrId=982001b8751-37bc0767-6310-4473-964a-e329e88fed8b, userVer=0,
loc=false, sampleClsName=ru.depsy.ignite.IgniteSession$$Lambda$2/16415465,
pendingUndeploy=false, undeployed=false, usage=1]],
taskClsName=ru.depsy.ignite.IgniteSession$$Lambda$2/16415465,
sesId=2cf001b8751-2a43ed8f-e226-47dc-b693-c8451f9c0ff9,
startTime=1475506873427, endTime=9223372036854775807,
taskNodeId=2a43ed8f-e226-47dc-b693-c8451f9c0ff9,
clsLdr=GridDeploymentClassLoader
[id=982001b8751-37bc0767-6310-4473-964a-e329e88fed8b, singleNode=false,
nodeLdrMap={2a43ed8f-e226-47dc-b693-c8451f9c0ff9=8bf001b8751-2a43ed8f-e226-47dc-b693-c8451f9c0ff9},
p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false], closed=false,
cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false,
subjId=2a43ed8f-e226-47dc-b693-c8451f9c0ff9, mapFut=IgniteFuture
[orig=GridFutureAdapter [resFlag=0, res=null, startTime=1475506874207,
endTime=0, ignoreInterrupts=false, state=INIT]]],
jobId=3cf001b8751-2a43ed8f-e226-47dc-b693-c8451f9c0ff9]]
[org.apache.ignite.internal.processors.job.GridJobWorker]
[pub-#7%pricingGridServer%] {}
class org.apache.ignite.binary.BinaryObjectException: Unexpected flag value
[pos=1296, expected=3, actual=-2]
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.checkFlagNoHandles(BinaryReaderExImpl.java:1423)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readInt(BinaryReaderExImpl.java:658)
at
org.apache.ignite.internal.binary.BinaryFieldAccessor$IntPrimitiveAccessor.read(BinaryFieldAccessor.java:321)
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:776)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1481)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readField(BinaryReaderExImpl.java:1683)
at
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read(BinaryFieldAccessor.java:639)
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:776)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1481)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readField(BinaryReaderExImpl.java:1683)
at
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read(BinaryFieldAccessor.java:639)
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:776)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1481)
at
org.apache.ignite.internal.binary.BinaryUtils.doReadObject(BinaryUtils.java:1608)
at
org.apache.ignite.internal.binary.BinaryUtils.deserializeOrUnmarshal(BinaryUtils.java:1955)
at
org.apache.ignite.internal.binary.BinaryUtils.doReadObjectArray(BinaryUtils.java:1819)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readObjectArray(BinaryReaderExImpl.java:1149)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readObjectArray(BinaryReaderExImpl.java:1142)
at
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:792)
at
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read(BinaryFieldAccessor.java:639)
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:776)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1481)
at
org.apache.ignite.internal.binary.BinaryUtils.doReadObject(BinaryUtils.java:1608)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readObject(BinaryReaderExImpl.java:1123)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2V2.readBin
Re: Unexpected flag value
Hi Vladislav, I have tried to 1) remove final modifiers, 2) then make the TaskOutput to be expliciltly Externalizable, The result is the same. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Unexpected-flag-value-tp8050p8054.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Unexpected flag value
Hi Dmitry,
Could you please check what will be if you implement Externalizable
explicitly?
public class TaskOutput implements Externalizable {...}
On Mon, Oct 3, 2016 at 4:45 PM, dmitry.parkhonin
wrote:
> It is not a response, it is a question.
>
> In addition to my original question:
> Just before the error there are the following lines in the log:
>
> 2016-10-03 13:28:08,788 DEBUG - Received peer class/resource loading
> request
> [node=7c7ae245-d3f2-40a5-a5fb-47fb18f97501, req=GridDeploymentRequest
> [rsrcName=ru/depsy/TaskOutput.class,
> ldrId=6d5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7, isUndeploy=false,
> nodeIds=null]]
> [org.apache.ignite.internal.managers.deployment.
> GridDeploymentCommunication]
> [p2p-#147%pricingGridServer%] {}
> 2016-10-03 13:28:08,788 DEBUG - Sent peer class loading response
> [node=7c7ae245-d3f2-40a5-a5fb-47fb18f97501, res=GridDeploymentResponse
> [success=true, errMsg=null, byteSrc=GridByteArrayList [size=1198]]]
> [org.apache.ignite.internal.managers.deployment.
> GridDeploymentCommunication]
> [p2p-#147%pricingGridServer%] {}
> 2016-10-03 13:28:08,819 DEBUG - Send recovery acknowledgement
> [rmtNode=7c7ae245-d3f2-40a5-a5fb-47fb18f97501, rcvCnt=80]
> [org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi]
> [grid-nio-worker-0-#105%pricingGridServer%] {}
> 2016-10-03 13:28:08,819 DEBUG - Received grid job response message
> [msg=GridJobExecuteResponse [nodeId=7c7ae245-d3f2-40a5-a5fb-47fb18f97501,
> sesId=0e5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7,
> jobId=1e5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7, gridEx=null,
> isCancelled=false], nodeId=7c7ae245-d3f2-40a5-a5fb-47fb18f97501]
> [org.apache.ignite.internal.processors.task.GridTaskProcessor]
> [sys-#40%pricingGridServer%] {}
> 2016-10-03 13:28:08,835 ERROR - Failed to obtain remote job result policy
> for result from ComputeTask.result(..) method ...
>
> It seems to me that the error appears just after the TaskOuput class is
> loaded by remote classloader.
>
> The TaskOutput class:
>
> public class TaskOutput {
>
> private final String taskId;
> private final Throwable exception;
> private final T output;
>
> public TaskOutput(String taskId, T output, Throwable exception) {
> this.taskId = taskId;
> this.output = output;
> this.exception = exception;
> }
>
> public String getTaskId() {
> return taskId;
> }
>
> public T getOutput() {
> return output;
> }
>
> public Throwable getException() {
> return exception;
> }
> }
>
> May the output field be the reason of the exception?
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Unexpected-flag-value-tp8050p8052.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
Re: Unexpected flag value
It is not a response, it is a question.
In addition to my original question:
Just before the error there are the following lines in the log:
2016-10-03 13:28:08,788 DEBUG - Received peer class/resource loading request
[node=7c7ae245-d3f2-40a5-a5fb-47fb18f97501, req=GridDeploymentRequest
[rsrcName=ru/depsy/TaskOutput.class,
ldrId=6d5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7, isUndeploy=false,
nodeIds=null]]
[org.apache.ignite.internal.managers.deployment.GridDeploymentCommunication]
[p2p-#147%pricingGridServer%] {}
2016-10-03 13:28:08,788 DEBUG - Sent peer class loading response
[node=7c7ae245-d3f2-40a5-a5fb-47fb18f97501, res=GridDeploymentResponse
[success=true, errMsg=null, byteSrc=GridByteArrayList [size=1198]]]
[org.apache.ignite.internal.managers.deployment.GridDeploymentCommunication]
[p2p-#147%pricingGridServer%] {}
2016-10-03 13:28:08,819 DEBUG - Send recovery acknowledgement
[rmtNode=7c7ae245-d3f2-40a5-a5fb-47fb18f97501, rcvCnt=80]
[org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi]
[grid-nio-worker-0-#105%pricingGridServer%] {}
2016-10-03 13:28:08,819 DEBUG - Received grid job response message
[msg=GridJobExecuteResponse [nodeId=7c7ae245-d3f2-40a5-a5fb-47fb18f97501,
sesId=0e5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7,
jobId=1e5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7, gridEx=null,
isCancelled=false], nodeId=7c7ae245-d3f2-40a5-a5fb-47fb18f97501]
[org.apache.ignite.internal.processors.task.GridTaskProcessor]
[sys-#40%pricingGridServer%] {}
2016-10-03 13:28:08,835 ERROR - Failed to obtain remote job result policy
for result from ComputeTask.result(..) method ...
It seems to me that the error appears just after the TaskOuput class is
loaded by remote classloader.
The TaskOutput class:
public class TaskOutput {
private final String taskId;
private final Throwable exception;
private final T output;
public TaskOutput(String taskId, T output, Throwable exception) {
this.taskId = taskId;
this.output = output;
this.exception = exception;
}
public String getTaskId() {
return taskId;
}
public T getOutput() {
return output;
}
public Throwable getException() {
return exception;
}
}
May the output field be the reason of the exception?
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Unexpected-flag-value-tp8050p8052.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Unexpected flag value
2016-10-03 13:28:08,819 DEBUG - Received grid job response message
[msg=GridJobExecuteResponse [nodeId=7c7ae245-d3f2-40a5-a5fb-47fb18f97501,
sesId=0e5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7,
jobId=1e5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7, gridEx=null,
isCancelled=false], nodeId=7c7ae245-d3f2-40a5-a5fb-47fb18f97501]
[org.apache.ignite.internal.processors.task.GridTaskProcessor]
[sys-#40%pricingGridServer%] {}
2016-10-03 13:28:08,835 ERROR - Failed to obtain remote job result policy
for result from ComputeTask.result(..) method (will fail the whole task):
GridJobResultImpl [job=C2V2
[c=ru.depsy.IgniteSession$$Lambda$2/22273532@f41c66], sib=GridJobSiblingImpl
[sesId=0e5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7,
jobId=1e5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7,
nodeId=7c7ae245-d3f2-40a5-a5fb-47fb18f97501, isJobDone=false],
jobCtx=GridJobContextImpl
[jobId=1e5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7, timeoutObj=null,
attrs={}], node=TcpDiscoveryNode [id=7c7ae245-d3f2-40a5-a5fb-47fb18f97501,
addrs=[...], sockAddrs=[...], discPort=47502, order=1, intOrder=1,
lastExchangeTime=1475497687197, loc=false, ver=1.7.0#20160801-sha1:383273e3,
isClient=false], ex=class o.a.i.compute.ComputeUserUndeclaredException:
Failed to execute job due to unexpected runtime exception
[jobId=1e5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7,
ses=GridJobSessionImpl [ses=GridTaskSessionImpl
[taskName=ru.depsy.IgniteSession$$Lambda$2/22273532, dep=SharedDeployment
[rmv=false, super=GridDeployment [ts=1475497688663, depMode=SHARED,
clsLdr=GridDeploymentClassLoader
[id=68dc38a8751-7c7ae245-d3f2-40a5-a5fb-47fb18f97501, singleNode=false,
nodeLdrMap={c03371e4-49be-4b34-947d-20022b45dee7=6d5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7},
p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false],
clsLdrId=68dc38a8751-7c7ae245-d3f2-40a5-a5fb-47fb18f97501, userVer=0,
loc=false, sampleClsName=ru.depsy.IgniteSession$$Lambda$2/22273532,
pendingUndeploy=false, undeployed=false, usage=1]],
taskClsName=ru.depsy.IgniteSession$$Lambda$2/22273532,
sesId=0e5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7,
startTime=1475497687977, endTime=9223372036854775807,
taskNodeId=c03371e4-49be-4b34-947d-20022b45dee7,
clsLdr=GridDeploymentClassLoader
[id=68dc38a8751-7c7ae245-d3f2-40a5-a5fb-47fb18f97501, singleNode=false,
nodeLdrMap={c03371e4-49be-4b34-947d-20022b45dee7=6d5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7},
p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false], closed=false,
cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false,
subjId=c03371e4-49be-4b34-947d-20022b45dee7, mapFut=IgniteFuture
[orig=GridFutureAdapter [resFlag=0, res=null, startTime=1475497688679,
endTime=0, ignoreInterrupts=false, state=INIT]]],
jobId=1e5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7]], hasRes=true,
isCancelled=false, isOccupied=true]
[org.apache.ignite.internal.processors.task.GridTaskWorker]
[sys-#40%pricingGridServer%] {}
class org.apache.ignite.IgniteException: Remote job threw user exception
(override or implement ComputeTask.result(..) method if you would like to
have automatic failover for this exception).
at
org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:101)
at
org.apache.ignite.internal.processors.task.GridTaskWorker$4.apply(GridTaskWorker.java:946)
at
org.apache.ignite.internal.processors.task.GridTaskWorker$4.apply(GridTaskWorker.java:939)
at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6553)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:939)
at
org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:810)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:995)
at
org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1220)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1238)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:866)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:106)
at
org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:829)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.compute.ComputeUserUndeclaredException:
Failed to execute job due to unexpected runtime exception
[jobId=1e5e38a8751-c03371e4-49be-4b34-947d-20022b45dee7,
ses=GridJobSessionImpl [ses=GridTaskSessionImpl
[taskName=ru.depsy.IgniteSession$$Lambda$2/22273532, dep=SharedDeployment
[rmv=false, supe
Re: Is Apache Tomcat 8 supported for Web Session Clustering yet?
Hi Yakov, I have not vigorously tested it, but so far it has been working perfectly fine in Tomcat 8. Cheers, Yuci -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Is-Apache-Tomcat-8-supported-for-Web-Session-Clustering-yet-tp7824p8049.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Delay in obtaining connection to Ignite server
Hi,
You can try to use only some server for establishing connection, also
reduce port range for search:
ipFinder.setAddresses(Arrays.asList("10.x.x.x:47500..47505"));
set local host for early determination of network interface
and decrease TcpDiscoverySpi.SocketTimeout to 0.3 seconds.
What the OS do you use?
The delay may be due to OS configuration.
On Fri, Sep 30, 2016 at 4:04 PM, Sri Ganesh V
wrote:
> Hi,
>
> We have 3 node (server) ignite cluster.
> Cache : Off-Heap Mode
> Heap Memory : 4GB (Each)
>
> We are trying to connect to the cluster.
> CODE :
>
> TcpDiscoverySpi spi = new TcpDiscoverySpi();
>
> TcpDiscoveryVmIpFinder ipFinder = new
> TcpDiscoveryVmIpFinder();
>
> ipFinder.setAddresses(Arrays.asList("10.x.x.x"));
> spi.setIpFinder(ipFinder);
> IgniteConfiguration cfg = new
> IgniteConfiguration();
> cfg.setFailureDetectionTimeout(30);
> System.out.println(cfg.
> getFailureDetectionTimeout());
> cfg.setDiscoverySpi(spi);
> cfg.setClientMode(true);
> cfg.setGridName(args[0]);
> Ignite ignite = Ignition.start(cfg);
>
> Everything is fine when we try out 10-15 concurrent connections.
>
> But when we try for 80-100 concurrent connection , there is a significant
> delay in obtaining connection but the fetching of data from cache post
> obtaining connection is really fast.
>
> Public and System thread pool size : 32
>
> It would be great if we get to know parameters with respect to connection
> getting queued up or any info in that direction that will reduce the delay
> to obtain connection.
>
> Thanks
> Sri Ganesh
>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Delay-in-obtaining-connection-to-
> Ignite-server-tp8035.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
--
Vladislav Pyatkov
Re: Out of memory
Sam, Could you please provide heap dump from this node and configuration of caches? javastuff@gmail.com wrote > Good to know about future change. I will try your suggestion regarding > system property. I do not think I will have scenario where insertion and > deletion of same key happens at the same time. > > Where can I get more details on this system property? or many such > internal tweaks. > > BTW all of my cache are marked TRANSACTIONAL, just for the sack of having > capability of explicit distributed lock. Hoping the system property > suggested is not only limited for ATOMIC. > > Thanks, > -Sam -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Out-of-memory-tp7995p8047.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
