This is my SQL table schema
ID (Long), URL (Varchar), SCORE (Double), APPNAME_ID (Long) We have a composite index on Score, Appname_Id. Based on your answer I've two questions. 1. How can I insert SQL rows using JCache data streamer API (if possible, with example)? Currently, I'm using jdbc thin with STREAMING ON. But the issue is mentioned above. 2. Each row data is -> ID (Long), URL (Varchar), SCORE (Double), APPNAME_ID (Long). How this data is stored as Key-Value? I mean what will be the key and what will be the value? Can you please answer these two questions? ---- On Mon, 15 Jun 2020 21:44:38 +0530 Stephen Darlington <[email protected]> wrote ---- Do you need the sorting as part of the loading process? If not, the best route would be to use the data streamer to load the data. You can still use the SQL engine and access your sorted data afterwards — remember that SQL and key-value are two different ways of accessing the same underlying data. > On 15 Jun 2020, at 15:46, adipro <mailto:[email protected]> wrote: > > We have an SQL table which we need because for normal JCache K-V we cannot > sort on some column's data. We need that sort feature. That's why we chose > SQL table representation. > > Our application is heavily multi-threaded. > > Now when trying to insert rows in that table, each thread simultaneously > sends 5000-10000 rows in bulk. Now if we use, SqlFieldsQuery, it's taking so > much of time as we cannot do it in bulk and have to do it in loop one by > one. > > For this case, we are using JDBC thin driver. > > But since it's multi-threaded we can't use single connection to execute in > parallel as it is not thread safe. > > So, what we did is, we added a synchronisation block which contains the > insertion of those rows in bulk using thin driver. The query performance is > good, but so many threads are in wait state as this is happening. > > Can someone please suggest any idea on how to insert those many rows in bulk > efficiently without threads waiting for so much time to use JDBC connection. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
