Re: ThinClient issues
Finally found the issue. In the thin client configuration I had only one port provided to connect to the server nodes. As soon as I provided the default range (10800-10900) the connections seemed to behave better. This resolved the issue somehow. However I tried it out at the cluster on the campus. With 50 nodes as thin clients connecting to two server nodes the server nodes seem to be completely overwhelmed. Then I thought why not let every node have it's own Ignite instance. It seems that a lot of resources are consumed from the Ignite Servers on the nodes even why no load at all on requests. Then I realized that starting up all servers is a nightmare. It takes several minutes and just activating all nodes is more than 3 minutes without even reliably bringing up all of the nodes Does anybody know how to define the consistend-id's in the xml file? Regards, Wolfgang Am 02.12.20 um 10:58 PM schrieb akorensh: Hi, The connection parameters are set by the thin client connector. see: https://ignite.apache.org/docs/latest/thin-clients/getting-started-with-thin-clients#configuring-thin-client-connector all attributes are listed here: https://ignite.apache.org/releases/2.9.0/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html see: https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#getThreadPoolSize-- (this specifies the number of threads set to process client requests) Simple C++ put/get example: https://github.com/apache/ignite/blob/master/modules/platforms/cpp/examples/thin-client-put-get-example/src/thin_client_put_get_example.cpp connection resources should released when you destroy the IgniteClient object. monitor the Server logs to see what is happening. see if the app is still connected using netstat. try to connect via other thin clients to see if connection requests are honored or not. Thanks, Alex -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
ThinClient issues
Hi, today I tried to use the thin client to connect to two server nodes and test the performance. Basically I tried to push random data from nodes where the Ignite database is not running. I used the put_all method of the c++ thin client api to do that with concurrent threads. However after a short amount of time I get connection problems and there does not seem to be a disconnect or close connection method available by the API. Therefore I tried to create the thin client and the configuration as a pointer with new and deleted them in case of errors safely. Each thread has its own client and configuration instance. For some reason the client is however unable to reconnect. Can somebody explain me why? Is there a connection limit set by the default configuration so that the number of nodes and threads is limited in some way? What am I doing wrong? As there are no examples online available for the thinclient where I can further dig in this is very difficult. I tried to look at the thin client c++ code and there is indeed a close function but this is not visible to the outside world for usage Regards, Wolfgang
CAP Theorem
Hi, the CAP Theorem states that a database can only achieve two of the following design goals: - Availability - Consistency - Partition Tolerance Which two of the above design goals Apache Ignite has been developed for? Mongo DB and Redis for instance match Consitency and Partition tolerance but lack of the availability design goal. Regards, Wolfgang
SQLRowCount in unixodbc driver
Hi, another issue popped up. SQLRowCount gives me back 0 when I execute the following query: select * from mytable limit 10; The column count is correct. The rowcount not as the api is fetching 10 rows I honestly think what I've seen so far is that the current ODBC driver implementation in Apache Ignite is more than basic I will try to amend the things that I need if I'm able to dig myself through the code and see if I can get to some reasonable point to understand why Regards, Wolfgang
Unixodbc currently not working...
Hi, after spending several hours to get the unixodbc driver up and running I nearly gave up. However together with the author of unixodbc I was able to find out that the current odbc driver in Apache Ignite is not doeing what it's supposed to do. As soon as I execute the command: et = SQLConnect(dbc, (SQLCHAR*)DSN, SQL_NTS, (SQLCHAR*)"", SQL_NTS, (SQLCHAR*)"", SQL_NTS); I get a crash in my program stating that: isql: symbol lookup error: /usr/local/lib/libignite-odbc.so: undefined symbol: SQLGetPrivateProfileString According to the author of unixodbc this is a function which is called to find out where to connect to by opening the /etc/odbc.ini file an looking for the DSN provided by the parameter. I compiled the Apache Ignite odbc connector exactly as stated in the manual. However an ldd on the /usr/local/lib/libignite-odbc.so does not show me a dependency on the odbcinst.so as stated by the author. So it seems that the configure script for the compilation is broken somehow. I installed unixodbc-dev on my ubuntu box so that shouldn't be the problem. Digging down into the cmake script it seems that it also correctly detects the installed unixodbc-dev installation. But the dependency to the odbcinst.so is missing. Hopefully someone can help. In the meantime I'm using the SQLDriverConnect routine which is not dependent on the SQLGetPrivateProfileString. That works but it just a dirty workaround and shouldn't be the final solution. Which ODBC Version is implemented in the code? Version2 or Version3? Reagards, Wolfgang
cpp unixodbc connection
The code I'm using at the moment is here... https://filebin.ca/5ihkjcOx8nid Am 26.11.20 um 7:41 PM schrieb Wolfgang Meyerle: Hi, I'm now trying to connect to apache ignite via the odbc driver the whole afternoon and now I'm stuck. The compilation and the installation in the platform/cpp directory went fine. Unixodbc driver was successfully installed according to the readmes. I also double checked dependency libraries of the ignite odbc so files and everything seems to be ok so far. When I run isql on the command line I cannot connect to Apache Ignite. If I use iusql on the command line with the DSN configured in /etc/odbc.ini I can perform a connection and can access Ignites database. However the cpp code does not work and I have no clue neither the error message is providing useful hints. According to the manual the ignite driver on linux is using unicode and is not supporting ansii or am I wrong? Can anybody provide me a short cpp example how to setup a connection with cpp using the ODBC driver? Regards, Wolfgang
cpp unixodbc connection
Hi, I'm now trying to connect to apache ignite via the odbc driver the whole afternoon and now I'm stuck. The compilation and the installation in the platform/cpp directory went fine. Unixodbc driver was successfully installed according to the readmes. I also double checked dependency libraries of the ignite odbc so files and everything seems to be ok so far. When I run isql on the command line I cannot connect to Apache Ignite. If I use iusql on the command line with the DSN configured in /etc/odbc.ini I can perform a connection and can access Ignites database. However the cpp code does not work and I have no clue neither the error message is providing useful hints. According to the manual the ignite driver on linux is using unicode and is not supporting ansii or am I wrong? Can anybody provide me a short cpp example how to setup a connection with cpp using the ODBC driver? Regards, Wolfgang
Crashes when running Apache Ignite as a sever node together with cpp code
Hi, I tried to run apache ignite as given by example from the website as a server node attached to my cpp program. Basically it works fine but debugging is not possible anymore which is a show stopper. The application crashes everytime when the program is debugged in QT. Any suggestions? Regards, Wolfgang
Ignite Visor cmd cannot connect...
Hi,
I tried using
ignitevisorcmd.sh -cfg=
however I'm getting the following error message:
Local node's binary configuration is not equal to remote node's binary
configuration
It further states:
localBinaryCfg=null
rmtBinaryCfg={globIdMapper ...
I'm using the exact same xml configuration file.
However for some stupid reason the visorcmd cannot connect
I'm using cpp code which is instantiating the server node.
The node seems to be running as I can use a thin client to connect to it
and put and get values...
Why?
Regards,
Wolfgang
Configuration Files and storage location
Hi community, I have a question regarding the xml configuration which is used in clients / servers and thinclients. Is it really necessary that each node is using the same configuration? I'm just asking because I'm trying out to setup multiple servers in one machine which totally is nonsense in a production release but worth working in a debugging environment and to find out how Ignite acts in certain use cases. I started basically two server configurations. One from a cpp code environment and the other from the usual ./ignite.sh bash script which has been provided with the release. The ignite environment was started first and somehow seems to block the cpp code from inserting data into the persistent storage as I cannot see my log output but just that the node has started up. After I started the ignite.sh script I activated the server node with control.sh --activate Is that maybe the problem as new nodes are not accepted anymore to be joined to the topology? Why do I have to activate the nodes or deactivate them anyway in the first place. In my mind the servers should be shutdown when the last node leaves the topology Regards, Wolfgang
Java VM Parameters setting through cpp api
Hi, Is it possible somehow to define specific Java VM Parameters through the cpp interfacte or the xml configuration files. I'm facing serious sys-stripe timeouts and one recommendation of Ignite is to switch to IPv4 only. Unfortunately this is an Java VM Parameter which I do not seem to have influence as soon as I start the Ignite Cpp Server Process via Ignite::Ignition::Start Regards, Wolfgang
Re: Thin Client connection not working...
Hi,
sorry for the late reply but it was getting too late yesterday evening...
Below you can find the code from the Apache Ignite Website that I tried
without successfully getting a connection to the Ignite Server Node...
Just to mention. I had both (the thin client and the server) running on
the same machine.
According to the xml file (which I can also attach if asked) I
configured the port to 10800.
nc localhost 10800 works so I assume the server is listening and the
problem is the client code from the Apache Ignite Website or my
configuration...
#include
#include
using namespace ignite::thin;
void TestClient()
{
IgniteClientConfiguration cfg;
//Endpoints list format is "[port[..range]][,...]"
cfg.SetEndPoints("localhost:10800");
IgniteClient client = IgniteClient::Start(cfg);
cache::CacheClient cacheClient =
client.GetOrCreateCache("TestCache");
cacheClient.Put(42, "Hello Ignite Thin Client!");
}
int main(int argc, char** argv) {
TestClient();
}
terminate called after throwing an instance of 'ignite::IgniteError'
what(): Failed to establish connection with any host.
22:18:50: The program has unexpectedly finished.
Am 16.11.20 um 1:50 PM schrieb wolfgang.meye...@googlemail.com:
I can Post an detailed Error mag tonight
Sent from Nine <http://www.9folders.com/>
*Von:* Stephen Darlington
*Gesendet:* Montag, 16. November 2020 12:10
*An:* user
*Betreff:* Re: Thin Client connection not working...
Doesn’t work how? Doesn’t compile? Doesn’t connect? Doesn’t create the
cache? Is there an error?
> On 16 Nov 2020, at 09:57, Wolfgang Meyerle
wrote:
>
> Hi,
>
> I tried using the cpp thin client example from the Apache ignite site
to create a small thin client connection example to one of the running
Apache Ignite cluster nodes.
>
> However it doesn't work and I'm out of a clue.
>
> I added the following bean to my persistence configuration file:
>
> id="ignite.cfg">
>
> class="org.apache.ignite.configuration.ClientConnectorConfiguration">
>
>
>
>
>
>
>
> I restarted the cluster node without any issues.
> A netcat localhost 10800 is able to start a connection to the node so
I assume the problem is on the cpp code side.
>
> I used the code sample from the website, but modified the port:
>
> #include
> #include
>
> using namespace ignite::thin;
>
> int main(int argc, char**argv)
> {
> IgniteClientConfiguration cfg;
>
> //Endpoints list format is "[port[..range]][,...]"
> cfg.SetEndPoints("127.0.0.1:10800");
>
> IgniteClient client = IgniteClient::Start(cfg);
>
> cache::CacheClient cacheClient =
> client.GetOrCreateCache("TestCache");
>
> cacheClient.Put(42, "Hello Ignite Thin Client!");
>
> return 0;
> }
>
>
> So what's wrong here?
>
>
> Regards,
>
> Wolfgang
Re: Thin Client connection not working...
I can Post an detailed Error mag tonight
Sent from Nine
Von: Stephen Darlington
Gesendet: Montag, 16. November 2020 12:10
An: user
Betreff: Re: Thin Client connection not working...
Doesn’t work how? Doesn’t compile? Doesn’t connect? Doesn’t create the cache?
Is there an error?
> On 16 Nov 2020, at 09:57, Wolfgang Meyerle
> wrote:
>
> Hi,
>
> I tried using the cpp thin client example from the Apache ignite site to
> create a small thin client connection example to one of the running Apache
> Ignite cluster nodes.
>
> However it doesn't work and I'm out of a clue.
>
> I added the following bean to my persistence configuration file:
>
> id="ignite.cfg">
>
> class="org.apache.ignite.configuration.ClientConnectorConfiguration">
>
>
>
>
>
>
>
> I restarted the cluster node without any issues.
> A netcat localhost 10800 is able to start a connection to the node so I
> assume the problem is on the cpp code side.
>
> I used the code sample from the website, but modified the port:
>
> #include
> #include
>
> using namespace ignite::thin;
>
> int main(int argc, char**argv)
> {
> IgniteClientConfiguration cfg;
>
> //Endpoints list format is "[port[..range]][,...]"
> cfg.SetEndPoints("127.0.0.1:10800");
>
> IgniteClient client = IgniteClient::Start(cfg);
>
> cache::CacheClient cacheClient =
> client.GetOrCreateCache("TestCache");
>
> cacheClient.Put(42, "Hello Ignite Thin Client!");
>
> return 0;
> }
>
>
> So what's wrong here?
>
>
> Regards,
>
> Wolfgang
Re: Thin Client connection not working...
Unable to connect
Sent from Nine
Von: Stephen Darlington
Gesendet: Montag, 16. November 2020 12:10
An: user
Betreff: Re: Thin Client connection not working...
Doesn’t work how? Doesn’t compile? Doesn’t connect? Doesn’t create the cache?
Is there an error?
> On 16 Nov 2020, at 09:57, Wolfgang Meyerle
> wrote:
>
> Hi,
>
> I tried using the cpp thin client example from the Apache ignite site to
> create a small thin client connection example to one of the running Apache
> Ignite cluster nodes.
>
> However it doesn't work and I'm out of a clue.
>
> I added the following bean to my persistence configuration file:
>
> id="ignite.cfg">
>
> class="org.apache.ignite.configuration.ClientConnectorConfiguration">
>
>
>
>
>
>
>
> I restarted the cluster node without any issues.
> A netcat localhost 10800 is able to start a connection to the node so I
> assume the problem is on the cpp code side.
>
> I used the code sample from the website, but modified the port:
>
> #include
> #include
>
> using namespace ignite::thin;
>
> int main(int argc, char**argv)
> {
> IgniteClientConfiguration cfg;
>
> //Endpoints list format is "[port[..range]][,...]"
> cfg.SetEndPoints("127.0.0.1:10800");
>
> IgniteClient client = IgniteClient::Start(cfg);
>
> cache::CacheClient cacheClient =
> client.GetOrCreateCache("TestCache");
>
> cacheClient.Put(42, "Hello Ignite Thin Client!");
>
> return 0;
> }
>
>
> So what's wrong here?
>
>
> Regards,
>
> Wolfgang
Thin Client connection not working...
Hi,
I tried using the cpp thin client example from the Apache ignite site to
create a small thin client connection example to one of the running
Apache Ignite cluster nodes.
However it doesn't work and I'm out of a clue.
I added the following bean to my persistence configuration file:
id="ignite.cfg">
class="org.apache.ignite.configuration.ClientConnectorConfiguration">
I restarted the cluster node without any issues.
A netcat localhost 10800 is able to start a connection to the node so I
assume the problem is on the cpp code side.
I used the code sample from the website, but modified the port:
#include
#include
using namespace ignite::thin;
int main(int argc, char**argv)
{
IgniteClientConfiguration cfg;
//Endpoints list format is "[port[..range]][,...]"
cfg.SetEndPoints("127.0.0.1:10800");
IgniteClient client = IgniteClient::Start(cfg);
cache::CacheClient cacheClient =
client.GetOrCreateCache("TestCache");
cacheClient.Put(42, "Hello Ignite Thin Client!");
return 0;
}
So what's wrong here?
Regards,
Wolfgang
Ignite performance issues...
Hi, I have a question in regards to Apache Ignite performance tuning. This is my config so far: https://pastebin.com/NWDzY3RK I plan to store 2 billion entries in the database. The machine that I have is not great, 32gb ram nothing more. I wonder how I can tune the performance to speed up things. I know that if I leave out indexing that would speed things are but indices are necessary. Also an SQL access is necessary though not necessarily during inserts. As I'm using C++ I don't know if ignite supports the feature in creating an SQL access later on after the data has been inserted in a cache. Is this possible? According to my logging I'm able to insert 42108964 entries in 66 minutes with 610 Insertions per second:610 and an average of 1057 insertions per second (which is not great in my opinion) According to ignites performance guidelines I segregated already to Wal stores from the persistence storage drive. I'm also getting Apache ignite errors from time to time: [21:24:15] Possible failure suppressed accordingly to a configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=db-checkpoint-thread, igniteInstanceName=null, finished=false, heartbeatTs=1605385441327]]] Tried to find something online and in the Apache Ignite documentation but struggles so hopefully any geek out there can drop a comment... Regards, Wolfgang
XML Configuration issues
Hi, I'm trying to change the xml configuration in apache ignite to support SQL for data caches: see https://pastebin.com/4qtepp0A However this config file doesn't seem to work. If I comment out the cache configuration part it works. What's wrong with it? Can't I mix cache configuration and persistent data store configuration? Regards, Wolfgang
Proper shutdown from C++ environment
Another question according a proper shutdown of ignite. Do I have to catch the sigint signal to properly shutdown apache ignite in my environment and set the node to inactive or is the code in ignite catering for this itself? Is it safe to install signal handlers with the usage of the ignite fat client? Regards, Wolfgang
SQL and Key Value usage in C++
Hi, I have a question where I'm currently struggling to find the answer in the Ignite Documentation and hopefully somebody of you can guide me in the right direction. According to the Ignite DDL Documentation of SQL I'm able upon Table creation to provide a cache name and I have a keyname and valuename property which I can set with for example: CREATE TABLE Test (a double, b double, c double, res boolean, primary key (a,b,c)) with "CACHE_NAME=Test, Key_type=FOO, Value_type=bar"; Can somebody example me for what purpose key_type and value_Type are used for? My intention is to use in code the key value cache principle in c++ and later on for complex queries standard sql. My hope is that the cpp key value based approach looks on the one hand prettier in code than daft long sql statements and performs better. Is anybody having experience in this matter? How can I enable sql accessible table in Ignite by using the cpp api without SQL statements? How would I use this in code. Are there some samples available that I can study? Reagards, Wolfgang
Unixodbc and Apache Ignite on OSX compilation
Hi, I'm currently struggling a little bit getting Ignite up and running with UnixOdbc on my OSX machine. I'm compiling the platform cpp files atm with cmake -DWITH_ODBC=ON -DWITH_THIN_CLIENT=ON -DWITH_TESTS=ON -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/Users/user/ApacheIgnite .. and getting: Undefined symbols for architecture x86_64: "_SQLGetPrivateProfileString", referenced from: ignite::odbc::ReadDsnString(char const*, std::__1::basic_string, std::__1::allocator > const&, std::__1::basic_stringstd::__1::char_traits, std::__1::allocator > const&) in dsn_config.cpp.o "_SQLInstallerError", referenced from: ignite::odbc::ThrowLastSetupError() in dsn_config.cpp.o "_SQLWritePrivateProfileString", referenced from: ignite::odbc::WriteDsnString(char const*, char const*, char const*) in dsn_config.cpp.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [odbc/libignite-odbc.2.9.0.50002.dylib] Error 1 make[1]: *** [odbc/CMakeFiles/ignite-odbc.dir/all] Error 2 make: *** [all] Error 2 as an output. Any ideas? My rough guess is that the UnixOdbc interface must have changed. Currently I'm using version unixodbc 2.3.9 according to brew. Apache Ignite is in version apache-ignite-2.9.0 Regards, Wolfgang
