Re: ODBC Driver compile error
Hi Linus, I had the same error when I was trying to compile with openssl 1.1. Compiling with openssl1.0 solved the issue for me.. -Floris From: Linus Surguy Sent: Sunday, August 12, 2018 3:08 PM To: user@ignite.apache.org Subject: ODBC Driver compile error [External] Hi all, I've been trying to compile the ODBC Driver under a fairly standard Debian system. Ignite installed from the deb on the Ignite website, although I've also tried the latest GIT release with the same results. Google found another user with a similar issue back in March on this list, but no resolution was reported back. The make fails with the following errors listed below. Is there particular gcc/g++/openssl version requirement which isn't obvious? Any pointers would be appreciated, Linus Making all in include make[3]: Entering directory '/usr/share/apache-ignite/platforms/cpp/odbc/include' make[3]: Nothing to be done for 'all'. make[3]: Leaving directory '/usr/share/apache-ignite/platforms/cpp/odbc/include' make[3]: Entering directory '/usr/share/apache-ignite/platforms/cpp/odbc' CXX src/ssl/secure_socket_client.lo In file included from ./include/ignite/odbc/ssl/ssl_bindings.h:21:0, from src/ssl/secure_socket_client.cpp:25: ./include/ignite/odbc/ssl/ssl_bindings.h:133:24: error: ‘ignite::odbc::ssl::OPENSSL_init_ssl’ declared as an ‘inline’ variable inline int SSL_library_init() ^ ./include/ignite/odbc/ssl/ssl_bindings.h:133:24: error: expression list treated as compound expression in initializer [-fpermissive] inline int SSL_library_init() ^ In file included from src/ssl/secure_socket_client.cpp:25:0: ./include/ignite/odbc/ssl/ssl_bindings.h:134:13: error: expected ‘,’ or ‘;’ before ‘{’ token { ^ In file included from ./include/ignite/odbc/ssl/ssl_bindings.h:21:0, from src/ssl/secure_socket_client.cpp:25: ./include/ignite/odbc/ssl/ssl_bindings.h:142:25: error: variable or field ‘OPENSSL_init_ssl’ declared void inline void SSL_load_error_strings() ^ src/ssl/secure_socket_client.cpp: In static member function ‘static void* ignite::odbc::ssl::SecureSocketClient::MakeContext(const string&, const string&, const string&, ignite::odbc::diagnostic::Diagnosable&)’: src/ssl/secure_socket_client.cpp:206:31: error: ‘ignite::odbc::ssl::OPENSSL_init_ssl’ cannot be used as a function (void)SSL_library_init(); ^ src/ssl/secure_socket_client.cpp:208:25: error: ‘ignite::odbc::ssl::OPENSSL_init_ssl’ cannot be used as a function SSL_load_error_strings(); ^ src/ssl/secure_socket_client.cpp:237:40: error: ‘SSL_CTRL_OPTIONS’ was not declared in this scope ssl::SSL_CTX_ctrl(ctx, SSL_CTRL_OPTIONS, flags, NULL); -- Magrathea Telecommunications Ltd Tel: 0345 004 0040 +44 118 321 0321 5 Commerce Park, Theale, RG7 4AB ENGLAND. Registered Company 4260485
RE: affinity key field not recognized c++
Ah thank you very much! That indeed fixes the problem. All the examples I could find had the full name specified there and since the classNames property were also full names, I never thought of changing this to the simple name. Apparently putting the full name there, triggers this strange behavior that the C++ code wants to send an updated type metadata to the Java code, causing this null error. Thanks a lot again for the help! -Floris From: Ilya Kasnacheev [mailto:ilya.kasnach...@gmail.com] Sent: Thursday 09 August 2018 4:48 PM To: user@ignite.apache.org Subject: Re: affinity key field not recognized c++ [External] Hello! I am pretty confident that affinity key configuration is supported by C++. There is one error in your configation file: you are using Simple Mapper, but still specify package of class in question. This causes weird behavior on Ignite side, but is trivial to fix: After that, I am able to put both MicFc and MicFc-typed BinaryObjects into cache, read them in C++ code using the snippet that you have specified. There is no need of explicit support of Affinity from C++ code side in this case as it is handled purely from Java side. Hope this helps, -- Ilya Kasnacheev 2018-08-09 16:22 GMT+03:00 Floris Van Nee mailto:florisvan...@optiver.com>>: Just an update.. Affinity key is indeed *not* supported in C++ at the moment. By digging into the C++ source I found the following.. core/src/impl/binary/binary_type_updater_impl.cpp line 78: rawWriter.WriteString(0); // Affinity key is not supported for now. It just always passes in a null value for affinity key.. This obviously leads to the error I saw, because on the Java side it tries to merge its valid affinity key with the null value passed from the C++ code. Does anyone know if it is on the planning to fix this? It is quite a vital thing to be able to choose a different mapping for your keys.. -Floris From: Floris Van Nee Sent: Thursday 09 August 2018 11:26 AM To: user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: RE: affinity key field not recognized c++ [External] Thanks for your reply. It is indeed the case that both Java and C++ configuration files are the same, and the AffinityKeyMapped annotation is not used in the Java declaration. Still, it is throwing an error. I have attached here a minimal reproducing example. My Java key class is the following: public class MicFc implements Binarylizable, Comparable { public String market; public String feedcode; @Override public int compareTo(MicFc t) { int m = market.compareTo(t.market); return m == 0 ? feedcode.compareTo(t.feedcode) : m; } @Override public void readBinary(BinaryReader reader) throws BinaryObjectException { market = reader.readString("market"); feedcode = reader.readString("feedcode"); } @Override public void writeBinary(BinaryWriter writer) throws BinaryObjectException { writer.writeString("market", market); writer.writeString("feedcode", feedcode); } } My configuration file (the same everywhere): org.apache.ignite.examples.streaming.MicFc And my C++ key class: namespace { class MicFc { public: std::string market, feedcode; }; } namespace ignite { namespace binary { IGNITE_BINARY_TYPE_START(MicFc) IGNITE_BINARY_GET_TYPE_ID_AS_HASH(MicFc) IGNITE_BINARY_GET_TYPE_NAME_AS_IS(MicFc) IGNITE_BINARY_GET_FIELD_ID_AS_HASH IGNITE_BINARY_IS_NULL_FALSE(MicFc) IGNITE_BINARY_GET_NULL_DEFAULT_CTOR(MicFc) static void Write(BinaryWriter& writer, const MicFc& obj) { writer.WriteString("market", obj.market); writer.WriteString("feedcode", obj.feedcode); } static void Read(BinaryReader& reader, MicFc& dst) { dst.market = reader.ReadStri
RE: affinity key field not recognized c++
Just an update.. Affinity key is indeed *not* supported in C++ at the moment. By digging into the C++ source I found the following.. core/src/impl/binary/binary_type_updater_impl.cpp line 78: rawWriter.WriteString(0); // Affinity key is not supported for now. It just always passes in a null value for affinity key.. This obviously leads to the error I saw, because on the Java side it tries to merge its valid affinity key with the null value passed from the C++ code. Does anyone know if it is on the planning to fix this? It is quite a vital thing to be able to choose a different mapping for your keys.. -Floris From: Floris Van Nee Sent: Thursday 09 August 2018 11:26 AM To: user@ignite.apache.org Subject: RE: affinity key field not recognized c++ [External] Thanks for your reply. It is indeed the case that both Java and C++ configuration files are the same, and the AffinityKeyMapped annotation is not used in the Java declaration. Still, it is throwing an error. I have attached here a minimal reproducing example. My Java key class is the following: public class MicFc implements Binarylizable, Comparable { public String market; public String feedcode; @Override public int compareTo(MicFc t) { int m = market.compareTo(t.market); return m == 0 ? feedcode.compareTo(t.feedcode) : m; } @Override public void readBinary(BinaryReader reader) throws BinaryObjectException { market = reader.readString("market"); feedcode = reader.readString("feedcode"); } @Override public void writeBinary(BinaryWriter writer) throws BinaryObjectException { writer.writeString("market", market); writer.writeString("feedcode", feedcode); } } My configuration file (the same everywhere): org.apache.ignite.examples.streaming.MicFc And my C++ key class: namespace { class MicFc { public: std::string market, feedcode; }; } namespace ignite { namespace binary { IGNITE_BINARY_TYPE_START(MicFc) IGNITE_BINARY_GET_TYPE_ID_AS_HASH(MicFc) IGNITE_BINARY_GET_TYPE_NAME_AS_IS(MicFc) IGNITE_BINARY_GET_FIELD_ID_AS_HASH IGNITE_BINARY_IS_NULL_FALSE(MicFc) IGNITE_BINARY_GET_NULL_DEFAULT_CTOR(MicFc) static void Write(BinaryWriter& writer, const MicFc& obj) { writer.WriteString("market", obj.market); writer.WriteString("feedcode", obj.feedcode); } static void Read(BinaryReader& reader, MicFc& dst) { dst.market = reader.ReadString("market"); dst.feedcode = reader.ReadString("feedcode"); } IGNITE_BINARY_TYPE_END } } To reproduce the error, first start a server with the config mentioned above and then run a C++ client with the same config (except for setting clientMode=true) and then run the following code: MicFc mfc; mfc.market = “TEST”; mfc.feedcode=”TEST”; auto c = ignite.GetOrCreateCache("test"); auto contains = c.ContainsKey(mfc); // this line throws the error Java exception occurred [cls=org.apache.ignite.binary.BinaryObjectException, msg=Binary type has different affinity key fields [typeName=MicFc, affKeyFieldName1=market, affKeyFieldName2=null]] -Floris From: Ilya Kasnacheev [mailto:ilya.kasnach...@gmail.com] Sent: Thursday 09 August 2018 11:09 AM To: user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: Re: affinity key field not recognized c++ [External] Hello! As far as my understanding goes, you have to supply cacheKeyConfiguration in both Java and C++ configuration files, and remove @AffinityKeyMapped from Java CustomKey class (or other ways of specifying it where applicable). Regards, -- Ilya Kasnacheev 2018-08-09 10:50 GMT+03:00 Floris Van Nee mailto:florisvan...@optiver.com>>: Hi all, I’m experiencing exactly the same issue as is described in a pre
RE: affinity key field not recognized c++
Thanks for your reply. It is indeed the case that both Java and C++ configuration files are the same, and the AffinityKeyMapped annotation is not used in the Java declaration. Still, it is throwing an error. I have attached here a minimal reproducing example. My Java key class is the following: public class MicFc implements Binarylizable, Comparable { public String market; public String feedcode; @Override public int compareTo(MicFc t) { int m = market.compareTo(t.market); return m == 0 ? feedcode.compareTo(t.feedcode) : m; } @Override public void readBinary(BinaryReader reader) throws BinaryObjectException { market = reader.readString("market"); feedcode = reader.readString("feedcode"); } @Override public void writeBinary(BinaryWriter writer) throws BinaryObjectException { writer.writeString("market", market); writer.writeString("feedcode", feedcode); } } My configuration file (the same everywhere): org.apache.ignite.examples.streaming.MicFc And my C++ key class: namespace { class MicFc { public: std::string market, feedcode; }; } namespace ignite { namespace binary { IGNITE_BINARY_TYPE_START(MicFc) IGNITE_BINARY_GET_TYPE_ID_AS_HASH(MicFc) IGNITE_BINARY_GET_TYPE_NAME_AS_IS(MicFc) IGNITE_BINARY_GET_FIELD_ID_AS_HASH IGNITE_BINARY_IS_NULL_FALSE(MicFc) IGNITE_BINARY_GET_NULL_DEFAULT_CTOR(MicFc) static void Write(BinaryWriter& writer, const MicFc& obj) { writer.WriteString("market", obj.market); writer.WriteString("feedcode", obj.feedcode); } static void Read(BinaryReader& reader, MicFc& dst) { dst.market = reader.ReadString("market"); dst.feedcode = reader.ReadString("feedcode"); } IGNITE_BINARY_TYPE_END } } To reproduce the error, first start a server with the config mentioned above and then run a C++ client with the same config (except for setting clientMode=true) and then run the following code: MicFc mfc; mfc.market = “TEST”; mfc.feedcode=”TEST”; auto c = ignite.GetOrCreateCache("test"); auto contains = c.ContainsKey(mfc); // this line throws the error Java exception occurred [cls=org.apache.ignite.binary.BinaryObjectException, msg=Binary type has different affinity key fields [typeName=MicFc, affKeyFieldName1=market, affKeyFieldName2=null]] -Floris From: Ilya Kasnacheev [mailto:ilya.kasnach...@gmail.com] Sent: Thursday 09 August 2018 11:09 AM To: user@ignite.apache.org Subject: Re: affinity key field not recognized c++ [External] Hello! As far as my understanding goes, you have to supply cacheKeyConfiguration in both Java and C++ configuration files, and remove @AffinityKeyMapped from Java CustomKey class (or other ways of specifying it where applicable). Regards, -- Ilya Kasnacheev 2018-08-09 10:50 GMT+03:00 Floris Van Nee mailto:florisvan...@optiver.com>>: Hi all, I’m experiencing exactly the same issue as is described in a previous post on this mailing list: http://apache-ignite-users.70518.x6.nabble.com/Affinity-Key-field-is-not-identified-if-binary-configuration-is-used-on-cache-key-object-td15959.html In short – defining an XML config with the appropriate binaryConfiguration (for Java/C++ interopability) and cacheKeyConfiguration (to define an affinityKeyFieldName for a certain key type) will fail when running from C++. Unfortunately, the earlier item on the mailing list didn’t find/post a solution to the problem. My custom key type is a class with two String members. I get the following error when I try to retrieve something from the cache: An error occurred: Java exception occurred [cls=org.apache.ignite.binary.BinaryObjectException, msg=Binary type has different affinity key fields [typeName=CustomKey,
affinity key field not recognized c++
Hi all, I'm experiencing exactly the same issue as is described in a previous post on this mailing list: http://apache-ignite-users.70518.x6.nabble.com/Affinity-Key-field-is-not-identified-if-binary-configuration-is-used-on-cache-key-object-td15959.html In short - defining an XML config with the appropriate binaryConfiguration (for Java/C++ interopability) and cacheKeyConfiguration (to define an affinityKeyFieldName for a certain key type) will fail when running from C++. Unfortunately, the earlier item on the mailing list didn't find/post a solution to the problem. My custom key type is a class with two String members. I get the following error when I try to retrieve something from the cache: An error occurred: Java exception occurred [cls=org.apache.ignite.binary.BinaryObjectException, msg=Binary type has different affinity key fields [typeName=CustomKey, affKeyFieldName1=string_1, affKeyFieldName2=null]] Running exactly the same from Java works fine. Also, when I remove the cacheKeyConfiguration part from the XML, it runs fine in both Java and C++ (but then this runs without the proper affinity key field of course). It seems like this is a bug, or am I missing something? -Floris
c++ build from source
Hi, I'm trying to build Apache Ignite C++ from source on Ubuntu. First, I downloaded the Ignite 2.6 source code and built the Java part. This was successful. Then, I went to the modules/platforms/cpp directory and ran: libtoolize && aclocal && autoheader && automake --add-missing && autoreconf This completed successfully as well. After that, ./configure exited with the following error: config.status: executing depfiles commands config.status: executing libtool commands rm: cannot remove 'core': Is a directory (This cannot remove 'core' message is displayed more often in the output). It seems it is trying to remove the directory 'core', but I don't know why. Has anyone seen this problem before? -Floris
Re: SQL SELECT with AffinityKeyMapped - no results
I tried your suggestion but unfortunately to no effect yet. I restarted the cluster every time I tried something new. It seems it is now the following that causes a problem: cfg.setKeyConfiguration(new CacheKeyConfiguration(TestKey.class.getName(), "b")); // using this line, i get the incorrect behavior cfg.setKeyConfiguration(new CacheKeyConfiguration(TestKey.class.getName(), "some_field_that_does_not_exist")); // this results in ok query behavior (but obviously not the affinity I want as the field does not exist. i think it just takes the full key as affinity here because they field does not exist The lines that configure my cache are now: CacheConfiguration cfg = new CacheConfiguration<>(TEST); cfg.setSqlSchema("PUBLIC"); cfg.setName("Test"); cfg.setKeyConfiguration(new CacheKeyConfiguration(TestKey.class.getName(), "b")); cfg.setIndexedTypes(TestKey.class, TestValue.class); -Floris From: Denis Mekhanikov Sent: Friday, August 3, 2018 5:35 PM To: user@ignite.apache.org Subject: Re: SQL SELECT with AffinityKeyMapped - no results [External] Floris, Binary metadata may be saved in work/binary_meta directory. Try cleaning this directory and see if it helps. You will also need to restart the whole cluster. Note, that it may lead to impossibility to read persisted data, if you have any. Denis пт, 3 авг. 2018 г. в 18:15, Floris Van Nee mailto:florisvan...@optiver.com>>: Thank you for your quick reply. That does look a lot like what I’m experiencing. However, I did some testing but so far I did not get the workaround to work. I put the following in the XML config file: org.apache.ignite.examples.streaming.TestKey Furthermore, when I create the cache, I pass a config with: cfg.setKeyConfiguration(new CacheKeyConfiguration(TestKey.class.getName(), "b")); According to the ticket this should be sufficient. Could I still be missing something here? -Floris From: Denis Mekhanikov [mailto:dmekhani...@gmail.com<mailto:dmekhani...@gmail.com>] Sent: Friday 03 August 2018 4:05 PM To: user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: Re: SQL SELECT with AffinityKeyMapped - no results [External] Floris, Most probably, you hit a bug, that was introduced in Ignite 2.0: https://issues.apache.org/jira/browse/IGNITE-5795<https://webvpn.optiver.com/jira/browse/,DanaInfo=.aituxixFhxjmsqM26w,SSL+IGNITE-5795> Because of this bug @AffinityKeyMapped annotation is ignored in classes, that are used in query entity configuration. As far as I can see, this is exactly your case. It's going to be fixed in Ignite 2.7. There is a workaround for this problem: you can list the problematic classes in BinaryConfiguration#classNames<https://webvpn.optiver.com/releases/latest/javadoc/org/apache/ignite/configuration/,DanaInfo=.aihplxjFhxjmsqM26w,SSL+BinaryConfiguration.html#setClassNames-java.util.Collection-> configuration property. Binary configuration should be specified as IgniteConfiguration#binaryConfiguration<https://webvpn.optiver.com/releases/latest/javadoc/org/apache/ignite/configuration/,DanaInfo=.aihplxjFhxjmsqM26w,SSL+IgniteConfiguration.html#setBinaryConfiguration-org.apache.ignite.configuration.BinaryConfiguration->. This configuration should be the same on all nodes. You may also need to configure CacheConfiguration#keyConfiguration<https://webvpn.optiver.com/releases/latest/javadoc/org/apache/ignite/configuration/,DanaInfo=.aihplxjFhxjmsqM26w,SSL+CacheConfiguration.html#setKeyConfiguration-org.apache.ignite.cache.CacheKeyConfiguration...-> for your cache. Denis пт, 3 авг. 2018 г. в 16:58, Floris Van Nee mailto:florisvan...@optiver.com>>: Hi all, I have defined two classes in Java – one for a key and one for a value. Suppose they look like this: public static class Key implements Serializable { public String a; @QuerySqlField @AffinityKeyMapped public String b; } public static class Value implements Serializable { @QuerySqlField public int c; } I then define a distributed cache (in Java) for this key/value pair and fill it with values (in Java). Now, I run the following in SQL: SELECT * FROM kv_table; I indeed see results of all the stuff that I inserted into the cache. However, when I try to select a certain value, it returns no results: SELECT * FROM kv_table WHERE b = ‘test’; I get expected results when filtering on any column that is not defined as AffinityKeyMapped. The only case where I get wrong results is for the AffinityKeyMapped column - I always get a wrong result (most of the time zero rows, for some values I do get one row, but I expect to see more r
RE: SQL SELECT with AffinityKeyMapped - no results
Thank you for your quick reply. That does look a lot like what I’m experiencing. However, I did some testing but so far I did not get the workaround to work. I put the following in the XML config file: org.apache.ignite.examples.streaming.TestKey Furthermore, when I create the cache, I pass a config with: cfg.setKeyConfiguration(new CacheKeyConfiguration(TestKey.class.getName(), "b")); According to the ticket this should be sufficient. Could I still be missing something here? -Floris From: Denis Mekhanikov [mailto:dmekhani...@gmail.com] Sent: Friday 03 August 2018 4:05 PM To: user@ignite.apache.org Subject: Re: SQL SELECT with AffinityKeyMapped - no results [External] Floris, Most probably, you hit a bug, that was introduced in Ignite 2.0: https://issues.apache.org/jira/browse/IGNITE-5795 Because of this bug @AffinityKeyMapped annotation is ignored in classes, that are used in query entity configuration. As far as I can see, this is exactly your case. It's going to be fixed in Ignite 2.7. There is a workaround for this problem: you can list the problematic classes in BinaryConfiguration#classNames<https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/BinaryConfiguration.html#setClassNames-java.util.Collection-> configuration property. Binary configuration should be specified as IgniteConfiguration#binaryConfiguration<https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setBinaryConfiguration-org.apache.ignite.configuration.BinaryConfiguration->. This configuration should be the same on all nodes. You may also need to configure CacheConfiguration#keyConfiguration<https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/CacheConfiguration.html#setKeyConfiguration-org.apache.ignite.cache.CacheKeyConfiguration...-> for your cache. Denis пт, 3 авг. 2018 г. в 16:58, Floris Van Nee mailto:florisvan...@optiver.com>>: Hi all, I have defined two classes in Java – one for a key and one for a value. Suppose they look like this: public static class Key implements Serializable { public String a; @QuerySqlField @AffinityKeyMapped public String b; } public static class Value implements Serializable { @QuerySqlField public int c; } I then define a distributed cache (in Java) for this key/value pair and fill it with values (in Java). Now, I run the following in SQL: SELECT * FROM kv_table; I indeed see results of all the stuff that I inserted into the cache. However, when I try to select a certain value, it returns no results: SELECT * FROM kv_table WHERE b = ‘test’; I get expected results when filtering on any column that is not defined as AffinityKeyMapped. The only case where I get wrong results is for the AffinityKeyMapped column - I always get a wrong result (most of the time zero rows, for some values I do get one row, but I expect to see more rows). Also when I remove the AffinityKeyMapped annotation and run my test again, everything works as expected. I run just a single server node and execute my query using the SQLLine tool that Ignite ships with. Is it possible that there is a bug in the code that handles this affinity key mapping for SQL queries? Or am I doing something wrong here? -Floris
SQL SELECT with AffinityKeyMapped - no results
Hi all, I have defined two classes in Java - one for a key and one for a value. Suppose they look like this: public static class Key implements Serializable { public String a; @QuerySqlField @AffinityKeyMapped public String b; } public static class Value implements Serializable { @QuerySqlField public int c; } I then define a distributed cache (in Java) for this key/value pair and fill it with values (in Java). Now, I run the following in SQL: SELECT * FROM kv_table; I indeed see results of all the stuff that I inserted into the cache. However, when I try to select a certain value, it returns no results: SELECT * FROM kv_table WHERE b = 'test'; I get expected results when filtering on any column that is not defined as AffinityKeyMapped. The only case where I get wrong results is for the AffinityKeyMapped column - I always get a wrong result (most of the time zero rows, for some values I do get one row, but I expect to see more rows). Also when I remove the AffinityKeyMapped annotation and run my test again, everything works as expected. I run just a single server node and execute my query using the SQLLine tool that Ignite ships with. Is it possible that there is a bug in the code that handles this affinity key mapping for SQL queries? Or am I doing something wrong here? -Floris
c++ data streamer api
Hi all, I'm looking into using Ignite and noticed the C++ API seems to be missing functionality for data streaming (Java IgniteDataStreamer together with the StreamReceiver/StreamVisitor/StreamTransformer classes). Without these, I assume it is not going to be easy to stream large amounts of data with an acceptable data rate (because it doesn't batch inserts etc.) 1) Are they indeed not available or am I missing something in the docs? 2) If they are missing - are there plans to add these and how much effort is estimated to implement such API? Is it just a matter of "we haven't got around to it yet, but it's similar to what's already there in the C++ API" or is there some limitation that is currently blocking implementation of the data streamer in C++? -Floris