RE: key value cache - value size limits or concerns

Hi SCott access will be driven by data loads on the other end. I am looking at odbc and rest access which will be used to load data into SQL server here is a base of how it will look class data { timespan loaddate string item string value } class alldata { list records } then when we

key value cache - value size limits or concerns

Is the following design appropriate or does it need reworking i want to build a key value store where the key is a company and the value is a list of data that will grow over time. The value data can contain thousands of items and these can change over time. My question is having a value for a

quick Q- date type in cache is converted to ticks. is this based on time since from 1970?

-- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Can ODBC driver be used to query a key value cache?

Thanks for the quick response Ilya. This is exactly what i am looking for -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Can ODBC driver be used to query a key value cache?

I have had a rather late change to a project that makes a db engine i am using not adequate. I wanted to see if Users can access data in a key value cache with the odbc driver or if i have to write a tool that uses the api or use rest -- Sent from:

ODBC Specified attribute is not supported

Is there a reason the following error is returned by the odbc driver OLE DB provider "MSDASQL" for linked server "IGNITE" returned message "Specified attribute is not supported." I am trying to query the odbc driver from SQL server with 2.7. I can see the table, the query returns empty columns

Re: where can i find 2.7 nightly release build

Thanks Igor -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

where can i find 2.7 nightly release build

i need to access the odbc driver and have had problems building on windows at work. Thanks in advanced -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Can Custom SQL functions get loaded after server start?

The problem is it will work in java using the api but all my code is in .Net and unfortunately the .net api doesn't have any of the capability in it. I guess we could have a separate java app that does this. I tried to fudge it by compiling the same class with an additional function but it looks

Can Custom SQL functions get loaded after server start?

I am looking to build a capability to add new functions in without restarting the service. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignite.net custom sql functions

That works - super happy with the result you are always so helpful thank you Ilya -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignite.net custom sql functions

I don't know why i didn't think of that - superb thanks Ilya. Are you certain the function needs to be in libs\home - It can't seem to find the function and i would have expected to at least see something in the log about loading it. ENV setup system variable IGNITE_HOME = c:\Ignite_2.7 test

Re: ignite.net custom sql functions

I have just realized an issue with this process. I have built my own version of the web console that generates caches for tables in databases. This is an automated process in the UI so i do not know what additional caches will be created beforehand. This leaves me with the dilemma of this custom

Re: ignite.net custom sql functions

one question regarding this configuration. I noticed it is cache specific and to me that implies every cache will need to have this configuration section set. Is there perhaps a way to configure this globally once instead of replicating it for every cache? -- Sent from:

Re: ignite.net custom sql functions

Thank you both for your assistance here, it is greatly appreciated - Ignite Rocks -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignite.net custom sql functions

Thanks Ilya So i need a little guidance here if you don't mind i am just throwing together a quick test as a precursor to a new project. I have a .net ignite service wrapper var cfg = new IgniteConfiguration() { SpringConfigUrl = "/ignitecfg.xml",

Re: What is the username of ignite web console

click register and enter your details and it will let you in -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Unable to load more than 5g data through sqlline

a different machine is fine all you need to do is build a client using either java, c++,.net, scala, and probably powershell as well i have something like this in .Net and can parallel load files (one per thread in a pool of 10 threads) using (var ldr =

Re: Unable to load more than 5g data through sqlline

a better option would be to drop sqlline and write your own client that reads the csv files and loads into the database. This way you can have multiple threads loading multiple files concurrently and each load you setup the parameters for the streamer including batch sizes and flush frequency.

Re: how to handle dataregion out of memmory gracefully

Thanks for this but what concerns me is if i setup several data regions and one of them happens to be a play area i guess for data scientists and they hit the memory limit of that region then the whole server goes down. in the case for a cluster all the nodes hosting that region will go down. I

destroy cache holding residual metadata in memory (2.7)

i am testing code and part of my tests is adding\removing tables. In one of the tests i add a table then destroy it and add it again but with an additional column. When i try load the table i am getting a data type mismatch and it is referring to the previous version of the table in the work

how to handle dataregion out of memmory gracefully

in testing i managed to exceed a data regions space and this region is memory only. When this happens an unhandled exception is thrown from the underlying ignite dlls and the process crashes. How can i handle this gracefully without losing the server? -- Sent from:

Re: odbc 2.7 columns issue

Thanks Igor I didn't think about the underscores in the names. Will try it in the morning. Thanks for the tip, hopefully it will get me over the line for the demo to the COO. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: odbc 2.7 columns issue

hi here is the odbc log file odbc.log My server configuration http://ignite.apache.org/schema/dotnet/IgniteConfigurationSection;> 127.0.0.1:47500..47509

Re: 2.7 (.net) JavaException: class org.apache.ignite.binary.BinaryObjectException: Invalid flag value: 0

hi please ignore the issue has been resolved by getting the latest from the nightly build -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

odbc 2.7 columns issue

Hi Thank you for fixing the bug where in Excel i was not able to see the tables. I can now however there are no columns visible and it gives an error that the data source did not return any columns. So it is a bit better that we can see the tables but we still cannot browse them and have to run a

Re: ignite odbc build error

hi Thank you it is all resolved now using the nightly build including the odbc and i can now see the odbc tables from Excel. fabulous -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: where to download 2.7 source

hi Thank you it is all resolved now using the nightly build including the odbc and i can now see the odbc tables from Excel. fabulous -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: where to download 2.7 source

i am giving the nightly builds a go and will use those in the .net proj reference. Fingers crossed - been struggling on this all day so far and feel very demotivated. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignite odbc build error

omg you are a life saver I CAN'T THANK YOU ENOUGH :) -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: where to download 2.7 source

I believe i have done this. I have downloaded 2.7 from github. packaged it with maven and then copied the entire dir to my local windows pc. I then updated the IGNITE_HOME variable to point to this new location. I started the server using the bat file (i couldnt start it in a .net proj as it was

Re: ignite odbc build error

maybe you could provide me with the 2.7 ignite.odbc.dll and i can skip the build part in visual studio and run light.exe -ext WixUIExtension ignite-odbc-x86.wixobj -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignite odbc build error

ok i didnt realise that. Could you please help me understand why i am getting that error when following the exact instructions on https://apacheignite.readme.io/docs/odbc-driver *Building installers on Windows* /Once you have built driver binaries you may want to build installers for easier

Re: ignite odbc build error

i do. this is a major blocker for me for my demo to the COO. All i am looking for is the 32bit odbc msi. is there anywhere i can get it or if you know if i should be using a different version of linux -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

2.7 (.net) JavaException: class org.apache.ignite.binary.BinaryObjectException: Invalid flag value: 0

Not sure if there is something missing in the .net ignite side for 2.7 I have built 2.7 on a linux machine, copied it to my windows machine and can launch it from the ignite.bat file no problem. I built the dotnet source as well and referenced those in my current project (a winforms app). When

Re: ignite odbc build error

those errors are gone but it looks like some ubuntu issue is now a problem. Have you seen this before? ubuntu@ip-10-112-47-3:/ignite2.7/ignite/modules/platforms/cpp$ libtoolize && aclocal && autoheader && automake --add-missing && autoreconf ./configure --enable-odbc --disable-node --disable-core

Re: ignite odbc build error

Hi Igor *I tried as a standard user and i get this* ubuntu@ip-10-112-47-3:/ignite2.7/ignite/modules/platforms/cpp$ libtoolize && aclocal && autoheader && automake --add-missing && autoreconf ./configure.ac --enable-odbc --disable-node --disable-core aclocal:configure.ac:32: warning: macro

Re: where to download 2.7 source

Hi Igor Not sure if there is something missing in the .net ignite side for 2.7 I have built 2.7 on a linux machine, copied it to my windows machine and can launch it from the ignite.bat file no problem. I built the dotnet source as well and referenced those in my current project (a winforms

Re: where to download 2.7 source

thanks Igor Unfortunately i have a demo next week and i need the odbc bug that was fixed in 2.7 to be ready for the demo so i have no choice but to try use the current status of 2.7. Thanks for the info -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

ignite odbc build error

i am trying to build the 2.7 odbc driver and i am getting an error. I am using ubuntu 14 root@ip-10-112-47-3:/ignite2.7/ignite/modules/platforms/cpp# ls aclocal.m4 config.sub cpp.dxg jni missing thin-client autom4te.cache configuredepcomp ltmain.sh

Re: OVER() clause in ignite

i noticed H2 roadmap for the current version has windowing functions. http://www.h2database.com/html/roadmap.html i guess this will mean ignite will have it available in the near future? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

where to download 2.7 source

I am doing a demo to our COO next week and part of this demo is excel ignite odbc which in 2.6 doesn't allow tables to be browsed see the following jira https://issues.apache.org/jira/browse/IGNITE-9891 anyway I am struggling to find if there is an available source with the latest builds to

Re: GetDataStreamer issue - ignite stops writing when underlying db reader closes

looks like you are right, there is dirty data in the source. Thanks Ilya for the pointer -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: GetDataStreamer issue - ignite stops writing when underlying db reader closes

could it have something to do with a Guid being the key? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

GetDataStreamer issue - ignite stops writing when underlying db reader closes

I have the following load code for ignite (.net) 1) setup a data streamer ldr = igniteclient.GetDataStreamer(TableName); 2) open connection to database and read data 3) create class objects and pass to streamer 4) close db connection I am observing the following behaviour the row count for this

Re: odbc caches - cannot browse

Thank you Igor -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: .net Datetime loads without error but table empty

this is sorted now thanks Stan - >net has several ways to do it and it didn't seem to like the other ways i was trying. simply setting it to datetime.touniversaltime resolved it. I do however think it odd that ignite was not throwing any errors -- Sent from:

.net Datetime loads without error but table empty

Running into an issue where i have an underlying sql server with a DateTime field. That is loaded into a class that has DateTime field in .net. I set the class field to its value like this from the data reader value = dataReader.GetDateTime(ordinal); the load code uses GetDataStreamer and

RE: data streamer - failed to update keys (GUID)

please ignore this thread - i have found the problem and it was some old residual test code that was being called instead. Correcting that has resolved the issue. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: odbc caches - cannot browse

here is the log file generated when opening a new odbc connection to ignite in excel. odbc3.log -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: client probe cache metadata

thanks for the help on this Stan -i will look into that method. thanks -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: client probe cache metadata

i need to to be able to discover what each cache configuration is. It this system is to have a large number of tables we need a way to identify what the original configuration for the given cache is. All our configs are happening at runtime (no xml file) because we are moving in a direction of

RE: data streamer - failed to update keys (GUID)

There is an error “Failed to update index, incorrect key class”. Any chance you’ve changed an integer field to a string one, or something like that? using (var ldr = igniteclient.GetDataStreamer(TableName)) = works using (var ldr = igniteclient.GetDataStreamer(TableName)) = fails shown error

RE: data streamer - failed to update keys (GUID)

not sure if it is related but the error isn;t pointing to it being an issue. interop requires uct date and from sql server we are parsing in a datetimeoffset and ignite sees it as an object and not timestamp -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: data streamer - failed to update keys (GUID)

I get the same error when trying the key as a string. here are some screen shots of the code, error, and table structure table_stucture.png code.png

Re: Query 3x slower with index

having worked with databases for 20 years i can see your indexes are not fully scoped. i see this as your issue having SUM(product_views_app) > 2 OR SUM(product_clicks_app) > 1 add those columns to the composite index so that it doesn't need to access the underlying table and can just use the

data streamer - failed to update keys (GUID)

hi I just wanted to check something. I have a table that has a guid key. When i try insert into this table at the end of the insert when the data reader connection closes i get this error (i have included a select that shows there is just 1 record with that key). Can Ignite handle guid keys?

RE: 2 questions about eviction and transactions

this is brilliant. thanks for this information. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

client probe cache metadata

Hi The rest service has a meta method that returns fields and index info. I can't see another rest method that would show what a cache config is like (backup, atomic or transactional, if metrics tracking is enabled, and whatever other relevant cache config there might be. Can i get this from the

2 questions about eviction and transactions

hi I can't find additional information on evictions and transactions that i need to document. 1) does the eviction policies apply at a node level or cluster level. I understand with replicated it probably is a node level but for a partitioned cache how does it work 2) if i have 4 tables and 1

Re: odbc caches - cannot browse

i would love some guidance on this. i have the 2.6 odbc setup and i can query it in excel 2016 but only if i manually type in the query. Excel won't show any of the tables like you would typically expect odbc connection is not configured to use a specific cache and no security is configured

Re: BUG - .net decimal type creating ignite table produces double

okay i think i need to close this- not sure why but running it on my local machine produces a double but on a server it is coming as a decimal. Must be an issue of some sort with my local pc so i think we can ignore this. thanks though for getting back to me -- Sent from:

Re: BUG - .net decimal type creating ignite table produces double

hi i am referring to c# classes that have properties that are decimal types. If i create caches with those classes the data type in ignite is double. Now admittedly i am interrogating schema information using the rest meta but i would have though that the data types returned are not schema bound

odbc caches - cannot browse

is there a reason why the odbc driver can't show all the caches and browse their schemas' like regular odbc drivers? this is a regular odbc driver allowing the user to navigate tables. odbc.png -- Sent from:

BUG - .net decimal type creating ignite table produces double

version 2.6 i have a tool that creates ignite tables and it is passing in a class with the type as decimal but it is coming out as double in when i query the metadata using rest in ignite? The documentation explicitly states c# decimal converts to java.math.bigdecimal which is not a floating

Re: ignite .net index group definition

So are you saying that i have to revert to the spring xml to get this done? I am using query entities when trying to do this but the order field doesn't seem to apply in the .net API. foreach (var field in item.Columns) { properties.Add(new Property()

Re: ignite .net programatically access cluster config (not working)

thanks ill try the compute tasks route -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Fetch column names in sql query results

hi I am on 2.6 using the .net api and there are is no metadata object on the result set even though this was said to have been added in 2.1. I could use the rest service but would prefer to use the api. can anyone advise if this is possible with SqlFieldsQuery? -- Sent from:

Re: Ignite Web Agent

just an fyi - the data loader for the web agent is very slow. I was getting 10 rows per second throughput and nobody on here could really provide any assistance so we opted to write our own version of the tool that uses the data streamer which is loading 15k rows per second with persisted storage

Re: ignite .net programatically access cluster config (not working)

i am building a management tool that creates and manages caches and for this i wanted to be able to dynamically detect the data regions on the cluster -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignite .net index group definition

anyone? i have tried applying it like this but it just creates 2 indexes [QuerySqlField(IsIndexed = true, IndexGroups = new[] { "name=aa,order=1" })] My understanding is under the hood it is essentially btree index and these index structures require an ordinal column configuration to build

Re: ignite .net programatically access cluster config (not working)

I don't but looking to query what data regions exist on the cluster. Is there a way to do this from the client? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

ignite .net index group definition

Hi In java a the index groups are defined in a way that makes sense e.g. @QuerySqlField(orderedGroups={@QuerySqlField.Group( name = "age_salary_idx", order = 0, descending = true)}) in .net however it looks like this [QuerySqlField(IsIndexed =true, IndexGroups = new[] {"someIndexGroup"})]

determine when client is connected

the Isactive method only checks if the grid is active. is there a way to check when the client has connected to resume processing? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

ignite .net programatically access cluster config (not working)

hi I have a single node cluster started with the following config when i launch a client and do Iignite.GetCluster().Configuration here is the screenshot of the config the client gets - the data storage config is null 2018-09-25_18-22-15.png

Re: .net decimal being stored as Other in ignite.

thanks, just another FYI , the values in that column cut off anything after the decimal point while it is other so 3.456 becomes 3 for now we are storing as a string and casting to decimal when we query -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: .net decimal being stored as Other in ignite.

anybody have any clue as to how Ignite is not mapping .Net decimal to Java.math.Bigdecimal? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

.net decimal being stored as Other in ignite.

i have the following class [QuerySqlField] public int vd { get; set; } [QuerySqlField] public long sharesinindex { get; set; } [QuerySqlField] public string name { get; set; } [QuerySqlField] public string isin { get; set; }

Re: webconsole generated project - cache load performance

I think I know what I need to do. I need to override the loadcache method and implement my own based on the example on the site. I think if I use generics I could parse in any class and it will handle the load. What's your thoughts on that? -- Sent from:

rmdbs integration readthrough - Ownership flag not set for binary property. Have you set 'keyFields' property of QueryEntity in programmatic or XML configuration?

i have setup rmdbs integration with read-through if i query an empty cache it should read-through from the underlying system but that isn't happening i get the following error [16:43:35,351][SEVERE][client-connector-#56%StoxxDataLake%][JdbcRequestHandler] Failed to get parameters metadata

Re: webconsole generated project - cache load performance

hi Initially this will be a single node demo but will be deployed as a 10 node cluster. I am running 2.6 and it is loading data from SQL server. There are 2 issues i am finding at the moment 1) it takes an hour to load but only loads 2.5k out of 4.6 million records (this seems serious) 2) it

multiple tables in same cache - is it possible

i have read that the preferred way is to have a cache per table. Is it however possible to have multiple tables in a single cache to make the administration of that cache simpler. what are the sort of problems that would surface if this were to be done? thanks -- Sent from:

webconsole generated project - cache load performance

Hi Is there a way to make the loading run faster for a project that has been generated by the web console. I am loading 4 million records and it is taking 30+ minutes -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Looking for information on these methods

hows it going with this Luqman I am still exploring an approach to single signon using kerberos without needing config files. I think i have made progress but i need to register a service principal name first. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: .net entity based configuration / datetime field error

not sure if it helps but the odbc driver doesnt play nice with date fields and deals with them in binary format class org.apache.ignite.binary.BinaryInvalidTypeException your error looks to be related - have you tried the jdbc driver as a test? -- Sent from:

error after activating cluster - Failed to wait for completion of partition map exchange

a Single node cluster starts fine. If i set it to active i get the following error popup (log file) [2018-09-03 13:54:45,187][INFO ][main][IgniteKernal%POC5] >>>__ >>> / _/ ___/ |/ / _/_ __/ __/ >>> _/ // (7 7// / / / / _/ >>>

error starting Ignite -Failed to activate node components

Ignition.start("C:igniteexamplesconfigpersistentstoreexample-persistent-store.xml"); "C:\Program Files\Java\jdk1.8.0_151\bin\java" "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2017.3\lib\idea_rt.jar=51729:C:\Program Files\JetBrains\IntelliJ IDEA

Looking for information on these methods

okay i have a white list plugin class that is now working - ThankYou Ilya (GridGain) I can now control nodes joining the cluster be it clients or servers. My next task is to manage user requests to specific caches and what they can do on those. I already have the GridSecurityProcessor class

Looking for information on these methods

okay i have a white list plugin class that is now working - ThankYou Ilya (GridGain) I can now control nodes joining the cluster be it clients or servers. My next task is to manage user requests to specific caches and what they can do on those. I already have the GridSecurityProcessor class as

Re: ignite.net config (applying spring and .net config)

Its working - Thanks a million Ilya Regards Wayne -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignite.net config (applying spring and .net config)

Super, that looks like the solution. Thank you so much for this. It's been so difficult navigating without complete documentation. Will give it a whirl. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignite.net config (applying spring and .net config)

by the way i am doing all this in Java -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignite.net config (applying spring and .net config)

I need to use the web console for rmdbs integration so i am using the config file that is generated and the project. What i am busy working on is writing a custom plugin that authenticates users. I need to setDiscoverySpi with the new authenticator but there doesn't seem to be a method in the

how to convert org.apache.ignite.lang.IgniteBiTuple to IgniteConfiguration

need to be able to read in the config file and then apply an authenticator to it before starting ignite. I have managed to call IgnitionEx.loadConfigurations("server.xml"); which returns org.apache.ignite.lang.IgniteBiTuple How can i convert this to an IgniteConfiguration object? -- Sent

Re: ignite.net config (applying spring and .net config)

hi Ilya Is there a way to do this in java? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: TcpDiscoverySpi.setAuthenticator

I am using a config file and not setting config in code. How can i register setDiscoverySpi with my custom class in the config file or can i set it in code and also apply the config file? thanks -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

TcpDiscoverySpi.setAuthenticator

Where does this call for a custom plugin need to be done? i.e TcpDiscoverySpi discoSpi = new TcpDiscoverySpi(); discoSpi.setAuthenticator(new TokenAuthenticator()); igniteConfig.setDiscoverySpi(discoSpi); -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: custom plugin question - jdbc client

Hi Taras I am slightly confused. You are mentioning sslcontext and looking at those related classes it doesn't appear to offer authentication and authorization. Here is where i currently am with this project. I have a white list plugin that i will apply to server nodes when they join. I have

custom plugin question - jdbc client

i have finally managed to get a plugin working for a white list on ignite 2.6. I am now going to start working on an authorization for users connecting to the cluster. How can i get clients pass through a kerberos ticket to the cluster? I think i need to override the authorization context class

has the following been resolved since 2.3? (jdbc thin client security)

https://issues.apache.org/jira/browse/IGNITE-6856 https://issues.apache.org/jira/browse/IGNITE-6941 -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

  1   2   >