Re: Distributed scheduling

[Dmitriy Setrakyan]

I am still not clear how it can be used or useful. Can you provide a simple
example of API calls that will make this possible?

On Thu, Jun 29, 2017 at 7:57 PM, Alexey Kuznetsov 
wrote:

> Hi,
>
> >> Alexey, why do you think it will be useful?
>
> I need to execute some tasks periodically on cluster. I think it is a
> common task.
> I could aggregate data once a day, I could generate reports and so on...
>
> Nodes can fail, cluster could be restarted. And with new persistence
> feature distributed scheduling
>  that survives cluster restart could be implemented.
>
> >>A similar topic was raised and discussed some time ago:
> >>http://apache-ignite-developers.2346864.n4.nabble.
> com/Tasks-Scheduling-and-Chaining-td14293.html
>
> I read that topic it is a bit different from my point of view.
> I'm talking only about periodical or one-time planned jobs on cluster that
> will be executed with some guaranties.
>
> But we also can take into account that use-case.
>
>
> On Fri, Jun 30, 2017 at 5:53 AM, Dmitriy Setrakyan 
> wrote:
>
> > Alexey, why do you think it will be useful?
> >
> > On Thu, Jun 29, 2017 at 12:22 PM, Alexey Kuznetsov <
> akuznet...@apache.org>
> > wrote:
> >
> > > Hi, All!
> > >
> > > I would like to start discussion about distributed scheduling.
> > >
> > > So, Ignite already has a module "ignite-schedule" that provide API for
> > > LOCAL scheduling on node.
> > > And if node failed - schedule will be lost.
> > >
> > > So, it will be very useful feature to have distributed scheduling.
> > >
> > > Lets discuss how it could be implemented.
> > >
> > > I see two options:
> > >   1) Extend "ignite-schedule" module to have API for distributed
> > > scheduling.
> > >   2) Extend compute API with methods that will allow scheduling of
> tasks
> > on
> > > cluster.
> > >   3) Implement both of 1) and 2) ?
> > >
> > > Any ideas and thought are welcomed!
> > >
> > > --
> > > Alexey Kuznetsov
> > >
> >
>
>
>
> --
> Alexey Kuznetsov
>

Re: Distributed scheduling

[Alexey Kuznetsov]

Hi,

>> Alexey, why do you think it will be useful?

I need to execute some tasks periodically on cluster. I think it is a
common task.
I could aggregate data once a day, I could generate reports and so on...

Nodes can fail, cluster could be restarted. And with new persistence
feature distributed scheduling
 that survives cluster restart could be implemented.

>>A similar topic was raised and discussed some time ago:
>>http://apache-ignite-developers.2346864.n4.nabble.
com/Tasks-Scheduling-and-Chaining-td14293.html

I read that topic it is a bit different from my point of view.
I'm talking only about periodical or one-time planned jobs on cluster that
will be executed with some guaranties.

But we also can take into account that use-case.


On Fri, Jun 30, 2017 at 5:53 AM, Dmitriy Setrakyan 
wrote:

> Alexey, why do you think it will be useful?
>
> On Thu, Jun 29, 2017 at 12:22 PM, Alexey Kuznetsov 
> wrote:
>
> > Hi, All!
> >
> > I would like to start discussion about distributed scheduling.
> >
> > So, Ignite already has a module "ignite-schedule" that provide API for
> > LOCAL scheduling on node.
> > And if node failed - schedule will be lost.
> >
> > So, it will be very useful feature to have distributed scheduling.
> >
> > Lets discuss how it could be implemented.
> >
> > I see two options:
> >   1) Extend "ignite-schedule" module to have API for distributed
> > scheduling.
> >   2) Extend compute API with methods that will allow scheduling of tasks
> on
> > cluster.
> >   3) Implement both of 1) and 2) ?
> >
> > Any ideas and thought are welcomed!
> >
> > --
> > Alexey Kuznetsov
> >
>



-- 
Alexey Kuznetsov

[jira] [Created] (IGNITE-5627) Web Console: refactor grid columns menu

[Ilya Borisov (JIRA)]

Ilya Borisov created IGNITE-5627:


 Summary: Web Console: refactor grid columns menu
 Key: IGNITE-5627
 URL: https://issues.apache.org/jira/browse/IGNITE-5627
 Project: Ignite
  Issue Type: Sub-task
  Components: UI, wizards
Reporter: Ilya Borisov
Assignee: Ilya Borisov
Priority: Minor


Currently, every menu that allows to toggle individual columns/categories has 
it's own implementation.

What to do:
1. Create a global reusable component that implements that should:
* Support cases when grid has regular columnDefs only and columnDefs with 
categories.
* Not display items that are always visible/hidden.
* Have it's own default button markup which can be replaced by transclusion 
slot.
* Automatically enable groups during previous groupping when user groups by 
such item (old implementation had such an issue, see admin users list for live 
example). For example, if first grouping was by column User with column Email 
hidden, and then user switches to Email grouping, Email column should become 
visible in grid and should be hidden from menu.
2. Use new component where appropriate and remove old copypasted code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Ignite Teamcity email notifications

[Denis Magda]

Trigged the alias creation. Will let you know once it’s ready for usage.

—
Denis

> On Jun 29, 2017, at 3:51 PM, Dmitriy Setrakyan  wrote:
> 
> I like this alias.
> 
> On Thu, Jun 29, 2017 at 3:43 PM, Denis Magda  wrote:
> 
>> What’s about c...@ignite.apache.org ? If there
>> are no objections I’ll create the alias.
>> 
>> —
>> Denis
>> 
>>> On Jun 29, 2017, at 11:53 AM, Dmitry Pavlov 
>> wrote:
>>> 
>>> Hi Dmitriy,
>>> 
>>> At the first step we need some email address to send notifications from
>>> behalf of TeamCity. We need to set up SMTP server, username and password.
>>> Later users may set up personal notification rules ( see the link
>>> http://ci.ignite.apache.org/profile.html?item=userNotifications).
>>> 
>>> Teamcity takes into account last commits in branch and sends
>> notifications
>>> separately and only to users which may break the build (option name
>> 'builds
>>> containing my changes').
>>> 
>>> Best Regards,
>>> Dmitriy Pavlov
>>> 
>>> 
>>> чт, 29 июн. 2017 г. в 21:21, Dmitriy Setrakyan :
>>> 
 Dmitry, I don't think ign...@apache.org is a valid email address. Why
 creating a mailing list for TC is not a good option?
 
 On Thu, Jun 29, 2017 at 8:17 AM, Dmitry Pavlov 
 wrote:
 
> Hi Igniters,
> 
> I want to set up a email notifications from the public Teamcity about
> broken builds. But there is no configured address and email account. To
 set
> up notifications we need some mail box (account) to send messages from.
> 
> I asked the apache infrastructure team in
> https://issues.apache.org/jira/browse/INFRA-14455
> However, they recommended through PMC to create a mailing list. I doubt
 it
> can solve the problem.
> 
> I also found out that there are e-mails from the address Ignite
>> Teamcity
 <
> ign...@apache.org>.
> 
> Can we use it? Do we know the password for this account?
> Any thought how to solve it?
> 
> Best Regards,
> Dmitriy Pavlov
> 
 
>> 
>> 

Re: Distributed scheduling

[Dmitriy Setrakyan]

Alexey, why do you think it will be useful?

On Thu, Jun 29, 2017 at 12:22 PM, Alexey Kuznetsov 
wrote:

> Hi, All!
>
> I would like to start discussion about distributed scheduling.
>
> So, Ignite already has a module "ignite-schedule" that provide API for
> LOCAL scheduling on node.
> And if node failed - schedule will be lost.
>
> So, it will be very useful feature to have distributed scheduling.
>
> Lets discuss how it could be implemented.
>
> I see two options:
>   1) Extend "ignite-schedule" module to have API for distributed
> scheduling.
>   2) Extend compute API with methods that will allow scheduling of tasks on
> cluster.
>   3) Implement both of 1) and 2) ?
>
> Any ideas and thought are welcomed!
>
> --
> Alexey Kuznetsov
>

Re: Ignite Teamcity email notifications

[Dmitriy Setrakyan]

I like this alias.

On Thu, Jun 29, 2017 at 3:43 PM, Denis Magda  wrote:

> What’s about c...@ignite.apache.org ? If there
> are no objections I’ll create the alias.
>
> —
> Denis
>
> > On Jun 29, 2017, at 11:53 AM, Dmitry Pavlov 
> wrote:
> >
> > Hi Dmitriy,
> >
> > At the first step we need some email address to send notifications from
> > behalf of TeamCity. We need to set up SMTP server, username and password.
> > Later users may set up personal notification rules ( see the link
> > http://ci.ignite.apache.org/profile.html?item=userNotifications).
> >
> > Teamcity takes into account last commits in branch and sends
> notifications
> > separately and only to users which may break the build (option name
> 'builds
> > containing my changes').
> >
> > Best Regards,
> > Dmitriy Pavlov
> >
> >
> > чт, 29 июн. 2017 г. в 21:21, Dmitriy Setrakyan :
> >
> >> Dmitry, I don't think ign...@apache.org is a valid email address. Why
> >> creating a mailing list for TC is not a good option?
> >>
> >> On Thu, Jun 29, 2017 at 8:17 AM, Dmitry Pavlov 
> >> wrote:
> >>
> >>> Hi Igniters,
> >>>
> >>> I want to set up a email notifications from the public Teamcity about
> >>> broken builds. But there is no configured address and email account. To
> >> set
> >>> up notifications we need some mail box (account) to send messages from.
> >>>
> >>> I asked the apache infrastructure team in
> >>> https://issues.apache.org/jira/browse/INFRA-14455
> >>> However, they recommended through PMC to create a mailing list. I doubt
> >> it
> >>> can solve the problem.
> >>>
> >>> I also found out that there are e-mails from the address Ignite
> Teamcity
> >> <
> >>> ign...@apache.org>.
> >>>
> >>> Can we use it? Do we know the password for this account?
> >>> Any thought how to solve it?
> >>>
> >>> Best Regards,
> >>> Dmitriy Pavlov
> >>>
> >>
>
>

Re: Distributed scheduling

[Denis Magda]

Hi Alex,

A similar topic was raised and discussed some time ago:
http://apache-ignite-developers.2346864.n4.nabble.com/Tasks-Scheduling-and-Chaining-td14293.html
 


Probably, we need to reincarnate that thread with the inputs from your side.

—
Denis

> On Jun 29, 2017, at 12:22 PM, Alexey Kuznetsov  wrote:
> 
> Hi, All!
> 
> I would like to start discussion about distributed scheduling.
> 
> So, Ignite already has a module "ignite-schedule" that provide API for
> LOCAL scheduling on node.
> And if node failed - schedule will be lost.
> 
> So, it will be very useful feature to have distributed scheduling.
> 
> Lets discuss how it could be implemented.
> 
> I see two options:
>  1) Extend "ignite-schedule" module to have API for distributed scheduling.
>  2) Extend compute API with methods that will allow scheduling of tasks on
> cluster.
>  3) Implement both of 1) and 2) ?
> 
> Any ideas and thought are welcomed!
> 
> -- 
> Alexey Kuznetsov

Re: Ignite Teamcity email notifications

[Denis Magda]

What’s about c...@ignite.apache.org ? If there 
are no objections I’ll create the alias.

—
Denis

> On Jun 29, 2017, at 11:53 AM, Dmitry Pavlov  wrote:
> 
> Hi Dmitriy,
> 
> At the first step we need some email address to send notifications from
> behalf of TeamCity. We need to set up SMTP server, username and password.
> Later users may set up personal notification rules ( see the link
> http://ci.ignite.apache.org/profile.html?item=userNotifications).
> 
> Teamcity takes into account last commits in branch and sends notifications
> separately and only to users which may break the build (option name 'builds
> containing my changes').
> 
> Best Regards,
> Dmitriy Pavlov
> 
> 
> чт, 29 июн. 2017 г. в 21:21, Dmitriy Setrakyan :
> 
>> Dmitry, I don't think ign...@apache.org is a valid email address. Why
>> creating a mailing list for TC is not a good option?
>> 
>> On Thu, Jun 29, 2017 at 8:17 AM, Dmitry Pavlov 
>> wrote:
>> 
>>> Hi Igniters,
>>> 
>>> I want to set up a email notifications from the public Teamcity about
>>> broken builds. But there is no configured address and email account. To
>> set
>>> up notifications we need some mail box (account) to send messages from.
>>> 
>>> I asked the apache infrastructure team in
>>> https://issues.apache.org/jira/browse/INFRA-14455
>>> However, they recommended through PMC to create a mailing list. I doubt
>> it
>>> can solve the problem.
>>> 
>>> I also found out that there are e-mails from the address Ignite Teamcity
>> <
>>> ign...@apache.org>.
>>> 
>>> Can we use it? Do we know the password for this account?
>>> Any thought how to solve it?
>>> 
>>> Best Regards,
>>> Dmitriy Pavlov
>>> 
>> 

Re: Get a ticket

[Denis Magda]

Hello Andrei,

Done. Please go ahead and pick a ticket of interest. 

—
Denis

> On Jun 29, 2017, at 10:46 AM, Elen Code  wrote:
> 
> Dear Ignite team,
> Could you please add me to the contributor list. My username is: Litanontaru.
> 
> Kind regards,Andrei 
> 
>On Thursday, June 29, 2017 3:11 PM, Dmitry Pavlov  
> wrote:
> 
> 
> Hi Andrei,
> 
> You need to sign in to apache ignite Jira and then send an email to the dev
> list sharing your JIRA id, so we can add you as a contributor in Jira.
> 
> Here is useful information to know.
> 
> Get familiar with Ignite development process described here:
> https://cwiki.apache.org/confluence/display/IGNITE/Development+Process
> 
> Instructions on how to contribute can be found here:
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
> 
> Project setup in Intellij IDEA:
> https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup
> 
> Best Regards,
> Dmitry Pavlov
> 
> чт, 29 июн. 2017 г. в 13:59, Elen Code :
> 
>> Hi Dmitry,
>> How can it get permissions to assign to me?I guess I need contributor
>> status or something like that.
>> Could you please help me with that?
>> Regards,
>> 
>> On Thursday, June 29, 2017 2:06 PM, Dmitry Pavlov <
>> dpavlov@gmail.com> wrote:
>> 
>> 
>>   Hi Andrei ,
>> 
>> IGNITE-5597  is
>> unassigned.
>> Usually such issue can be picked up by any community member.
>> So please feel free to assign issue to yourself.
>> 
>> Best Regards,
>> Dmitry Pavlov
>> 
>> чт, 29 июн. 2017 г. в 9:25, Elen Code :
>> 
>>> Dear Ignite team,
>>> 
>>> I would like to start my first contribution with ticket
>>> https://issues.apache.org/jira/browse/IGNITE-5597.
>>> If it is ok, I'll assign it to myself.
>>> Please confirm,
>>> Kind regards,
>>> Andrei
>>> 
>>> 
>> 
>> 
> 

[jira] [Created] (IGNITE-5626) SQL TRIM function is not working properly

[Valentin Kulichenko (JIRA)]

Valentin Kulichenko created IGNITE-5626:
---

 Summary: SQL TRIM function is not working properly
 Key: IGNITE-5626
 URL: https://issues.apache.org/jira/browse/IGNITE-5626
 Project: Ignite
  Issue Type: Bug
  Components: sql
Affects Versions: 2.0
Reporter: Valentin Kulichenko
 Fix For: 2.1


{{TRIM}} function trims only one char instead of the whole provided string. 
Reproducer attached.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: golang client for Ignite

[Vladimir Ozerov]

Aleksandr,

Ignite is distributed system. Typical call to get/put/sql is of micro- and
millisecond magnitude. Java->CPP call takes several dozen nanoseconds.
CPP->Java (so-called "reverse JNI") takes several hundred nanoseconds.
Typical marshalling overhead is of nano- and micro-second scale as well.
That is, total overhead of platform->Java->platform interaction is several
orders of magnitude *smaller* than any call to remote node. So if Golang
users are afraid of JNI, than any distributed system would scare them to
death.

We have a plan to implement platform-independent thin client protocol which
could be re-used from many languages. But you should understand, that in
most cases it will be *slower* than current implementation, because Java
core contains essential logic for efficient request routing. Native thin
client could be faster only if you port many and many thousands of relevant
non-trivial Java code to native platform, which can be estimated not to
man-months, but to man-years to complete.

Having said that, I would recommend you to look closer to current JNI-based
implementation :-)


On Thu, Jun 29, 2017 at 9:11 AM, Aleksandr Sokolovskii 
wrote:

> Guys,
>
> Thanks for your answers.
>
> So If users want to use Ignite DataGrid from “unsupported” language they
> have two choice for now:
>
> 1) REST API
> It supports: cache management, put-get, sql exec queries, fetch
> It does not support: sql transactions
> REST API is easy to use.
> REST API has a lot of overhead (HTTP, json marshalling/unmarshalling,..
> you know)
>
> 2) develop pure SQL driver for Ignite ODBC endpoint
> It supports: sql exec queries, fetch
> It does not support: cache management, put-get, sql transactions
> Roman is right – using cgo is not very good idea.
> Honestly It’s not trivial task to develop pure SQL driver for Ignite ODBC
> endpoint.
> I spent some hours to remember how STL serializes std::string to
> unmarshall it in golang. )))
> My last C++ experience was in 2004. )))
> Another my concern is C++ wrapper over Ignite Java process.
> I try to explain:
> The main reason to develop something using golang is performance.
> People how use golang are enslaved by performance.
> They think how to avoid unnecessary memory allocation, avoid reflection,
> etc.
> Go provides goroutine instead of system threads to avoid memory allocation
> and cross-threads switching.
> Cross-process switching (cpp->jvm->cpp) on server side of in-memory
> database looks like “shot yourself in foot” for such people. )))
>
> There is another way: to implement native client for H2 DB protocol.
> (If I’m not mistaken Ignite use H2 DB protocol for JDBC endpoint).
> Again it’s not trivial task to implement pure native H2 DB client for each
> necessary programming language.
>
> What do you think about to implement gRPC (or Apache Thrift, or…)
> platform-language-neutral protocol endpoint in Java core of Ignite?
> I resolves “unsupported” language client problem.
>
> Thanks,
> Aleksandr
>
> From: Roman Shtykh
> Sent: 29 июня 2017 г. 6:36
> To: dev@ignite.apache.org
> Subject: Re: golang client for Ignite
>
> Guys,
>
> Just an observation, but when I introduce Ignite to other developers, they
> get very interested. But when the question goes to the support of the
> language they use for development, often they look disappointed because a
> chunk of functionalities I introduce becomes inaccessible. I think from
> their NoSQL experience they expect a client protocol.
> As for Golang, from what I know, you have to write bindings in C for C++
> and use them. There are performance concerns with cgo.
> -- Roman
>
>
> On Thursday, June 29, 2017 9:34 AM, Dmitriy Setrakyan <
> dsetrak...@apache.org> wrote:
>
>
>  Denis,
>
> Perhaps adding the same level of support for Golang as we have for .NET or
> C++ would be asking too much. The reason we start a JVM in .NET and C++ is
> because we implemented the full Ignite API support, even including
> transactions and executing .NET closures on remote Java servers.
>
> Perhaps, from Golang client standpoint, it is enough to implement it
> directly over the REST protocol initially.
>
> D.
>
> On Wed, Jun 28, 2017 at 3:59 PM, Denis Magda  wrote:
>
> > Aleksandr,
> >
> > > I take a look into ignite cpp core.
> > > Looks like it attaches to running jvm and calls java methods.
> > > Please tell me that I’m wrong! )))
> >
> > That’s a correct observation. Both C++ and .NET clients spawn a JVM
> > process underneath and redirect almost all the requests to it. That
> > approach allowed us to support these languages easily. Otherwise, it
> would
> > have taken ages to develop true C++ and .NET libs.
> >
> > > It will not work for golang.
> >
> > Why?
> >
> > > Is there language-neutral protocol to communicate with Ignite server?
> >
> > REST, ODBC and JDBC only.
> >
> > —
> > Denis
> >
> >
> > > On Jun 28, 2017, at 12:43 PM, Aleksandr Sokolovskii  >
> > wrote:
> > >
> 

Re: By bytes access to binary format

[Vladimir Ozerov]

Hi Vlad,

I am not quite sure I understand the problem. Can you show how the API you
propose would look like? Remember that "field" method can return anything
from primitive, String or byte array, to another BinaryObject. And returned
BinaryObject can have references outside of itself, so it cannot be
serialized easily without full rebuild. .

On Thu, Jun 29, 2017 at 10:16 AM, Vladislav Pyatkov 
wrote:

> Val,
>
> I proposal, access as a stream to binary object, because we have doubled
> copy on touch a field (first at copy from cache and second - on getting a
> field).
>
> For the stream in/out to cache I will be used IGFS.
> Main idea to avoid GC pressure when make a massive read from key-value
> storage.
>
> On Wed, Jun 28, 2017 at 9:36 PM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Vladislav,
> >
> > Are you suggesting to stream directly from cache. or from a binary object
> > that is already copied from cache?
> >
> > -Val
> >
> > On Wed, Jun 28, 2017 at 2:52 AM, Vladislav Pyatkov <
> vpyat...@gridgain.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Recently, from one of Ignite user, I listened interest idea.
> > > What if I want to pass some date to java stream from cache.
> > >
> > > With binary I do it like this:
> > >
> > > BinaryObject get = (BinaryObject) cache.get(key);
> > > byte[] dataFromCache = get.field("data");
> > > System.out.write(dataFromCache, 0, dataFromCache.length);
> > >
> > > But in this case we got garbage a lot, due to each time new bytes array
> > is
> > > creating.
> > >
> > > This will lead to many GC events in case we load a some of million
> > entries.
> > > Could we offer additional API for working with java stream:
> > >
> > > BinaryObject.writeBytesToBuf("data", ByteBuffer.allocate(1024));
> > >
> > > or with buffer
> > >
> > > BinaryObject.writeBytesToBuf("data", new byte[1000], 100);
> > >
> > > I already created a Jira ticket.
> > > https://issues.apache.org/jira/browse/IGNITE-5602
> > >
> > > --
> > > Vladislav Pyatkov
> > > Architect-Consultant "GridGain Rus" Llc.
> > > +7 963 716 68 99
> > >
> >
>
>
>
> --
> Vladislav Pyatkov
> Architect-Consultant "GridGain Rus" Llc.
> +7 963 716 68 99
>

[jira] [Created] (IGNITE-5625) SQL: auto-increment of fields

[Denis Magda (JIRA)]

Denis Magda created IGNITE-5625:
---

 Summary: SQL: auto-increment of fields
 Key: IGNITE-5625
 URL: https://issues.apache.org/jira/browse/IGNITE-5625
 Project: Ignite
  Issue Type: Task
Affects Versions: 2.0
Reporter: Denis Magda
 Fix For: 2.3


SQL engine should be able to auto-increment fields. At least this has to be 
supported for primary keys.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5624) SQL: LTRIM/RTRIM not working with multiple character

[Denis Magda (JIRA)]

Denis Magda created IGNITE-5624:
---

 Summary: SQL: LTRIM/RTRIM not working with multiple character
 Key: IGNITE-5624
 URL: https://issues.apache.org/jira/browse/IGNITE-5624
 Project: Ignite
  Issue Type: Task
Reporter: Denis Magda


 LTRIM/RTRIM is working only for one char and not for a word (Multiple 
Character). For instance, the example below doesn't work for Ignite:

{code}
SELECT product_name, LTRIM(product_name, 'Monitor ') "Short Name"
   FROM products
   WHERE product_name LIKE 'Monitor%';
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5623) DDL needs to support DEFAULT operator

[Denis Magda (JIRA)]

Denis Magda created IGNITE-5623:
---

 Summary: DDL needs to support DEFAULT operator 
 Key: IGNITE-5623
 URL: https://issues.apache.org/jira/browse/IGNITE-5623
 Project: Ignite
  Issue Type: Task
Reporter: Denis Magda
Assignee: Alexander Paschenko


There should be a way to set a default value for a column/field if the one is 
not specified during an insert operation. In general, we need to support 
{{ DEFAULT }} in a way it's show below:

{code}
CREATE TABLE Persons (
  ID int,
  FirstName varchar(255),
  Age int,
  City varchar(255) DEFAULT 'Sandnes'
);
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5622) Streaming and batching for ODBC driver

[Denis Magda (JIRA)]

Denis Magda created IGNITE-5622:
---

 Summary: Streaming and batching for ODBC driver
 Key: IGNITE-5622
 URL: https://issues.apache.org/jira/browse/IGNITE-5622
 Project: Ignite
  Issue Type: Task
Affects Versions: 2.0
Reporter: Denis Magda
Assignee: Igor Sapego
 Fix For: 2.2


Ignite ODBC driver has to support both streaming and batching mode is a similar 
way it's done for JDBC: 
http://apache-ignite-developers.2346864.n4.nabble.com/Batch-DML-queries-design-discussion-td12859.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2220: Ignite 3935

[1vanan]

GitHub user 1vanan opened a pull request:

https://github.com/apache/ignite/pull/2220

Ignite 3935

Issue https://issues.apache.org/jira/browse/IGNITE-3935

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/1vanan/ignite ignite-3935

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2220.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2220


commit 0c0ebb844de5f392e10522d1f6abe70b82dcfb04
Author: ivanan 
Date:   2017-06-29T19:31:16Z

initial commit

commit dab3fc1a702ec125b1a26dfc5c1a155cd245ed60
Author: ivanan 
Date:   2017-06-29T19:51:27Z

StreamingExample & StreamingExampleSelfTest was added




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Distributed scheduling

[Alexey Kuznetsov]

Hi, All!

I would like to start discussion about distributed scheduling.

So, Ignite already has a module "ignite-schedule" that provide API for
LOCAL scheduling on node.
And if node failed - schedule will be lost.

So, it will be very useful feature to have distributed scheduling.

Lets discuss how it could be implemented.

I see two options:
  1) Extend "ignite-schedule" module to have API for distributed scheduling.
  2) Extend compute API with methods that will allow scheduling of tasks on
cluster.
  3) Implement both of 1) and 2) ?

Any ideas and thought are welcomed!

-- 
Alexey Kuznetsov

Re: Ignite Teamcity email notifications

[Dmitry Pavlov]

Hi Dmitriy,

At the first step we need some email address to send notifications from
behalf of TeamCity. We need to set up SMTP server, username and password.
Later users may set up personal notification rules ( see the link
http://ci.ignite.apache.org/profile.html?item=userNotifications).

Teamcity takes into account last commits in branch and sends notifications
separately and only to users which may break the build (option name 'builds
containing my changes').

Best Regards,
Dmitriy Pavlov


чт, 29 июн. 2017 г. в 21:21, Dmitriy Setrakyan :

> Dmitry, I don't think ign...@apache.org is a valid email address. Why
> creating a mailing list for TC is not a good option?
>
> On Thu, Jun 29, 2017 at 8:17 AM, Dmitry Pavlov 
> wrote:
>
> > Hi Igniters,
> >
> > I want to set up a email notifications from the public Teamcity about
> > broken builds. But there is no configured address and email account. To
> set
> > up notifications we need some mail box (account) to send messages from.
> >
> > I asked the apache infrastructure team in
> > https://issues.apache.org/jira/browse/INFRA-14455
> >  However, they recommended through PMC to create a mailing list. I doubt
> it
> > can solve the problem.
> >
> > I also found out that there are e-mails from the address Ignite Teamcity
> <
> > ign...@apache.org>.
> >
> > Can we use it? Do we know the password for this account?
> > Any thought how to solve it?
> >
> > Best Regards,
> > Dmitriy Pavlov
> >
>

[jira] [Created] (IGNITE-5621) Support BINARY and VARBINARY types for C++ and ODBC

[Denis Magda (JIRA)]

Denis Magda created IGNITE-5621:
---

 Summary: Support BINARY and VARBINARY types for C++ and ODBC
 Key: IGNITE-5621
 URL: https://issues.apache.org/jira/browse/IGNITE-5621
 Project: Ignite
  Issue Type: Task
Affects Versions: 2.0
Reporter: Denis Magda
Assignee: Igor Sapego
 Fix For: 2.2


We need to support BINARY and VARBINARY types at C++ and ODBC levels.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5620) Meaningful error codes and types of exceptions for SQL operations

[Denis Magda (JIRA)]

Denis Magda created IGNITE-5620:
---

 Summary: Meaningful error codes and types of exceptions for SQL 
operations 
 Key: IGNITE-5620
 URL: https://issues.apache.org/jira/browse/IGNITE-5620
 Project: Ignite
  Issue Type: Task
Reporter: Denis Magda
Assignee: Alexander Paschenko


Presently, SQL engine throws a generic type of exception with custom text in 
case of an operation failure. In result, Ignite ODBC driver returns a similar 
error code (2000) for different kind of failures.

For example, error code 2000 is returned for the following

{code}
Duplicate key during INSERT [key=CorpcontactcountKey [idHash=1412656257, 
hash=2004096461, mdn=9192]] 
{code}

{code}
Failed to parse query: INSERT INTO "DG".Corpcontactcount 
(mdn,contactcount,lastupdatetime)
values(?,?,?,?) 
{code}

{code}
Wrong value has been set [typeName=Pocsubscrinfo, fieldName=vocoderid, 
fieldType=short, assignedValueType=byte] Error Code: 2000
{code}

The following has to be done:
* Create unique types of exceptions for Java whenever applicable.
* Add {{errorCode}} parameter and method to a generic SQL exception.
* ODBC and JDBC drivers have to return unique codes based on the exception code 
or type.
* All the codes have to be documented on readme.io. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Ignite Teamcity email notifications

[Dmitriy Setrakyan]

Dmitry, I don't think ign...@apache.org is a valid email address. Why
creating a mailing list for TC is not a good option?

On Thu, Jun 29, 2017 at 8:17 AM, Dmitry Pavlov 
wrote:

> Hi Igniters,
>
> I want to set up a email notifications from the public Teamcity about
> broken builds. But there is no configured address and email account. To set
> up notifications we need some mail box (account) to send messages from.
>
> I asked the apache infrastructure team in
> https://issues.apache.org/jira/browse/INFRA-14455
>  However, they recommended through PMC to create a mailing list. I doubt it
> can solve the problem.
>
> I also found out that there are e-mails from the address Ignite Teamcity <
> ign...@apache.org>.
>
> Can we use it? Do we know the password for this account?
> Any thought how to solve it?
>
> Best Regards,
> Dmitriy Pavlov
>

IGNITE-4901 (Decrease logging level for DataStremer retry)

[Alexey Kukushkin]

Hi,
Presently when DataStreamer is flushing data and cluster topology changes, the 
DataStreamer would output an ERROR message to the console and retry flushing 
the data with the latest topology. 
We have this IGNITE-4901 issue asking to decrease logging level since a 
topology change is normal and DataStreamer reliably handles it.
In addition to setting the log level to INFO, I suggest we change Ignite to 
fail to update the cache only if MAJOR topology version changed (another node 
joined/left) since minor version change (registering/unregistering another 
cache) does not impact updating the cache unless we delete the cache-in-use but 
the latter error is handled differently anyway.
Please let me know if anyone has objections or comments. Otherwise I will 
submit this solution.
Thank you!

Best regards, Alexey

Re: Get a ticket

[Elen Code]

Dear Ignite team,
Could you please add me to the contributor list. My username is: Litanontaru.

Kind regards,Andrei 

On Thursday, June 29, 2017 3:11 PM, Dmitry Pavlov  
wrote:
 

 Hi Andrei,

You need to sign in to apache ignite Jira and then send an email to the dev
list sharing your JIRA id, so we can add you as a contributor in Jira.

Here is useful information to know.

Get familiar with Ignite development process described here:
https://cwiki.apache.org/confluence/display/IGNITE/Development+Process

Instructions on how to contribute can be found here:
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute

Project setup in Intellij IDEA:
https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup

Best Regards,
Dmitry Pavlov

чт, 29 июн. 2017 г. в 13:59, Elen Code :

> Hi Dmitry,
> How can it get permissions to assign to me?I guess I need contributor
> status or something like that.
> Could you please help me with that?
> Regards,
>
>    On Thursday, June 29, 2017 2:06 PM, Dmitry Pavlov <
> dpavlov@gmail.com> wrote:
>
>
>  Hi Andrei ,
>
> IGNITE-5597  is
> unassigned.
> Usually such issue can be picked up by any community member.
> So please feel free to assign issue to yourself.
>
> Best Regards,
> Dmitry Pavlov
>
> чт, 29 июн. 2017 г. в 9:25, Elen Code :
>
> > Dear Ignite team,
> >
> > I would like to start my first contribution with ticket
> > https://issues.apache.org/jira/browse/IGNITE-5597.
> > If it is ok, I'll assign it to myself.
> > Please confirm,
> > Kind regards,
> > Andrei
> >
> >
>
>

   

[GitHub] ignite pull request #2219: IGNITE-5187 Made test more reliable.

[alexpaschenko]

GitHub user alexpaschenko opened a pull request:

https://github.com/apache/ignite/pull/2219

IGNITE-5187 Made test more reliable.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5187

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2219.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2219


commit 2c3d443ac6039f83f735fe1ec594412d93bf25a3
Author: Alexander Paschenko 
Date:   2017-06-29T16:12:10Z

IGNITE-5187 Made test more reliable.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Ignite Teamcity email notifications

[Dmitry Pavlov]

Hi Igniters,

I want to set up a email notifications from the public Teamcity about
broken builds. But there is no configured address and email account. To set
up notifications we need some mail box (account) to send messages from.

I asked the apache infrastructure team in
https://issues.apache.org/jira/browse/INFRA-14455
 However, they recommended through PMC to create a mailing list. I doubt it
can solve the problem.

I also found out that there are e-mails from the address Ignite Teamcity <
ign...@apache.org>.

Can we use it? Do we know the password for this account?
Any thought how to solve it?

Best Regards,
Dmitriy Pavlov

[jira] [Created] (IGNITE-5619) CPP: Implement Compute::Execute() for Ignite C++

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-5619:
---

 Summary: CPP: Implement Compute::Execute() for Ignite C++
 Key: IGNITE-5619
 URL: https://issues.apache.org/jira/browse/IGNITE-5619
 Project: Ignite
  Issue Type: New Feature
  Components: platforms
Affects Versions: 2.0
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.1


Need to implement method {{Compute::Execute}} and {{Compute::ExecuteAsync}} for 
Ignite C++



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

I've attached HangTest. I suppose it should not hang, am i right ?

чт, 29 июн. 2017 г. в 14:54, ALEKSEY KUZNETSOV :

> Igntrs.
> Im rewieving all usages of threadId of
> transaction.(IgniteTxAdapter#threadID). What is the point of usage threadId
> in mvcc entry ?
>
> пн, 3 апр. 2017 г. в 9:47, ALEKSEY KUZNETSOV :
>
>> so what do u think on my idea?
>>
>> пт, 31 Мар 2017 г., 11:05 ALEKSEY KUZNETSOV :
>>
>>> sorry for misleading you. We planned to support multi-node transactions,
>>> but failed.
>>>
>>> пт, 31 мар. 2017 г. в 10:51, Alexey Goncharuk <
>>> alexey.goncha...@gmail.com>:
>>>
>>> Well, now the scenario is more clear, but it has nothing to do with
>>> multiple coordinators :) Let me think a little bit about it.
>>>
>>> 2017-03-31 9:53 GMT+03:00 ALEKSEY KUZNETSOV :
>>>
>>> > so what do u think on the issue ?
>>> >
>>> > чт, 30 Мар 2017 г., 17:49 ALEKSEY KUZNETSOV >> >:
>>> >
>>> > > Hi ! Thanks for help. I've created ticket :
>>> > > https://issues.apache.org/jira/browse/IGNITE-4887
>>> > > and a commit :
>>> > >
>>> https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06
>>> > 436b638e5c
>>> > > We really need this feature
>>> > >
>>> > > чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk <
>>> > alexey.goncha...@gmail.com
>>> > > >:
>>> > >
>>> > > Aleksey,
>>> > >
>>> > > I doubt your approach works as expected. Current transaction recovery
>>> > > protocol heavily relies on the originating node ID in its internal
>>> logic.
>>> > > For example, currently a transaction will be rolled back if you want
>>> to
>>> > > transfer a transaction ownership to another node and original tx
>>> owner
>>> > > fails. An attempt to commit such a transaction on another node may
>>> fail
>>> > > with all sorts of assertions. After transaction ownership changed,
>>> you
>>> > need
>>> > > to notify all current transaction participants about this change,
>>> and it
>>> > > should also be done failover-safe, let alone that you did not add any
>>> > tests
>>> > > for these cases.
>>> > >
>>> > > I back Denis here. Please create a ticket first and come up with
>>> clear
>>> > > use-cases, API and protocol changes design. It is hard to reason
>>> about
>>> > the
>>> > > changes you've made when we do not even understand why you are making
>>> > these
>>> > > changes and how they are supposed to work.
>>> > >
>>> > > --AG
>>> > >
>>> > > 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV <
>>> alkuznetsov...@gmail.com>:
>>> > >
>>> > > > So, what do u think on my idea ?
>>> > > >
>>> > > > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV <
>>> > alkuznetsov...@gmail.com
>>> > > >:
>>> > > >
>>> > > > > Hi! No, i dont have ticket for this.
>>> > > > > In the ticket i have implemented methods that change transaction
>>> > status
>>> > > > to
>>> > > > > STOP, thus letting it to commit transaction in another thread. In
>>> > > another
>>> > > > > thread you r going to restart transaction in order to commit it.
>>> > > > > The mechanism behind it is obvious : we change thread id to
>>> newer one
>>> > > in
>>> > > > > ThreadMap, and make use of serialization of txState, transactions
>>> > > itself
>>> > > > to
>>> > > > > transfer them into another thread.
>>> > > > >
>>> > > > >
>>> > > > > вт, 28 мар. 2017 г. в 20:15, Denis Magda :
>>> > > > >
>>> > > > > Aleksey,
>>> > > > >
>>> > > > > Do you have a ticket for this? Could you briefly list what
>>> exactly
>>> > was
>>> > > > > done and how the things work.
>>> > > > >
>>> > > > > —
>>> > > > > Denis
>>> > > > >
>>> > > > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV <
>>> > > > alkuznetsov...@gmail.com>
>>> > > > > wrote:
>>> > > > > >
>>> > > > > > Hi, Igniters! I 've made implementation of transactions of
>>> > non-single
>>> > > > > > coordinator. Here you can start transaction in one thread and
>>> > commit
>>> > > it
>>> > > > > in
>>> > > > > > another thread.
>>> > > > > > Take a look on it. Give your thoughts on it.
>>> > > > > >
>>> > > > > >
>>> > > > > https://github.com/voipp/ignite/pull/10/commits/
>>> > > > 3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
>>> > > > > >
>>> > > > > > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin <
>>> > > sergi.vlady...@gmail.com
>>> > > > >:
>>> > > > > >
>>> > > > > >> You know better, go ahead! :)
>>> > > > > >>
>>> > > > > >> Sergi
>>> > > > > >>
>>> > > > > >> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV <
>>> > > > alkuznetsov...@gmail.com
>>> > > > > >:
>>> > > > > >>
>>> > > > > >>> we've discovered several problems regarding your
>>> "accumulation"
>>> > > > > >>> approach.These are
>>> > > > > >>>
>>> > > > > >>>   1. perfomance issues when transfering data from temporary
>>> cache
>>> > > to
>>> > > > > >>>   permanent one. Keep in mind big deal of concurent
>>> transactions
>>> > in
>>> > > > > >>> Service
>>> > > > > >>>   commiter
>>> > > > > >>>   2. extreme 

[GitHub] ignite pull request #2218: IGNITE-5576: Implemented Compute::Broadcast for C...

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/2218

IGNITE-5576: Implemented Compute::Broadcast for C++



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5582

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2218.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2218


commit e7311199eebbe9133f7d3cebe6f6f2a38bd67762
Author: Igor Sapego 
Date:   2017-06-22T16:13:19Z

IGNITE-5576: Added Compute::Run()

commit b6c16a5170e0f50c49096f8ba6b0e7cc33312eaa
Author: Igor Sapego 
Date:   2017-06-22T16:57:12Z

IGNITE-5576: Added tests. Fixed bug

commit 2618d312008d3961323b4b788667548c0f2b1223
Author: Igor Sapego 
Date:   2017-06-23T17:14:03Z

IGNITE-5576: Fixed config

commit efb53f9105154a32c287c02ba39c3a270ee3d2c2
Author: Igor Sapego 
Date:   2017-06-23T17:28:15Z

IGNITE-5576: Edited documentation

commit 561519af001b93ae195330fa8e2c2ab4db3046b3
Author: Igor Sapego 
Date:   2017-06-27T12:02:02Z

IGNITE-5582: Refactoring

commit b16fb1c3e522835e4286cdb5b0378f9329276df0
Author: Igor Sapego 
Date:   2017-06-28T16:31:23Z

IGNITE-5582: Added MultipleJobComputeTaskHolder

commit d505d10206dcc2ce752b72179efce6e6794270d3
Author: Igor Sapego 
Date:   2017-06-29T12:55:33Z

IGNITE-5582: Added tests and fixed issues.

commit ab2a6aae554a1be0a7feade1916fea16091daccd
Author: Igor Sapego 
Date:   2017-06-29T13:13:46Z

IGNITE-5582: Added Error tests

commit e9c298085374611f40cfe5aa92a470f34d04ddc3
Author: Igor Sapego 
Date:   2017-06-29T13:20:11Z

IGNITE-5582: Added tests




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: IGNITE-4365: Data grid in deadlock on stop by DataStreamerImpl

[Александр Меньшиков]

I don't have it. I got all information from thread dump which you added to
the ticket: one thread stuck in the DataStreamerImpl#doFlush() (which was
called by GridDistributedCacheAdapter.GlobalRemoveAllJob#localExecute()),
and the other in the GridCacheGateway#onStopped() (which was called by
GridCacheProcessor#onExchangeDone()).

I read about a problem with reproducing (Alexey Kuznetsov's first comment
in JIRA) and made the decision to look at the different view.
Code still looks dangerous, so I don't think the problem has resolved
itself.

In thread dump there are 2 tests:
1) GridCacheNearTxForceKeyTest
2) CrossCacheTxRandomOperationsTest

They all passed in a single running.

2017-06-29 15:31 GMT+03:00 Yakov Zhdanov :

> Alex, can you please share a test that demonstrates the hang?
>
> --Yakov
>
> 2017-06-29 14:27 GMT+03:00 Александр Меньшиков :
>
>> Hello,
>>
>> I want to make ticket IGNITE-4365
>> . The problem came
>> from DataStreamerImpl.
>> There are methods which use DataStreamerImpl under the lock
>> (GridCacheGateway), but the method DataStreamerImpl#doFlush() has a
>> "while(true)" loop. And in case when someone is calling the
>> GridCacheGateway#onStopped(), application can get stuck in the loop in
>> DataStreamerImpl#doFlush(), and in trying get a lock in
>> GridCacheGateway#onStopped().
>>
>> So I need an expert opinion about DataStreamerImpl#doFlush().
>> 1) Can I just drop unfinished futures in DataStreamerImpl#doFlush() when
>> someone is calling GridCacheGateway#onStopped()? I can track it by adding a
>> volatile boolean flag in the GridCacheGateway.
>> 2) Or better to modify a futures execution DataStreamerImpl#load0() to
>> use onDone with an exception or something like that?
>>
>> Methods which use or might use DataStreamerImpl under the lock:
>>
>> 1) GridCacheAdapter#localLoad()
>> 2) GridCacheAdapter#localLoadAndUpdate()
>> 3) GridCacheAdapter#localLoadCache()
>> 4) GridDistributedCacheAdapter.GlobalRemoveAllJob#localExecute() (it
>> exectly happen in thread dump in ticket)
>>
>
>

Re: IGNITE-4365: Data grid in deadlock on stop by DataStreamerImpl

[Yakov Zhdanov]

Alex, can you please share a test that demonstrates the hang?

--Yakov

2017-06-29 14:27 GMT+03:00 Александр Меньшиков :

> Hello,
>
> I want to make ticket IGNITE-4365
> . The problem came
> from DataStreamerImpl.
> There are methods which use DataStreamerImpl under the lock
> (GridCacheGateway), but the method DataStreamerImpl#doFlush() has a
> "while(true)" loop. And in case when someone is calling the
> GridCacheGateway#onStopped(), application can get stuck in the loop in
> DataStreamerImpl#doFlush(), and in trying get a lock in
> GridCacheGateway#onStopped().
>
> So I need an expert opinion about DataStreamerImpl#doFlush().
> 1) Can I just drop unfinished futures in DataStreamerImpl#doFlush() when
> someone is calling GridCacheGateway#onStopped()? I can track it by adding a
> volatile boolean flag in the GridCacheGateway.
> 2) Or better to modify a futures execution DataStreamerImpl#load0() to use
> onDone with an exception or something like that?
>
> Methods which use or might use DataStreamerImpl under the lock:
>
> 1) GridCacheAdapter#localLoad()
> 2) GridCacheAdapter#localLoadAndUpdate()
> 3) GridCacheAdapter#localLoadCache()
> 4) GridDistributedCacheAdapter.GlobalRemoveAllJob#localExecute() (it
> exectly happen in thread dump in ticket)
>

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Igntrs.
Im rewieving all usages of threadId of
transaction.(IgniteTxAdapter#threadID). What is the point of usage threadId
in mvcc entry ?

пн, 3 апр. 2017 г. в 9:47, ALEKSEY KUZNETSOV :

> so what do u think on my idea?
>
> пт, 31 Мар 2017 г., 11:05 ALEKSEY KUZNETSOV :
>
>> sorry for misleading you. We planned to support multi-node transactions,
>> but failed.
>>
>> пт, 31 мар. 2017 г. в 10:51, Alexey Goncharuk > >:
>>
>> Well, now the scenario is more clear, but it has nothing to do with
>> multiple coordinators :) Let me think a little bit about it.
>>
>> 2017-03-31 9:53 GMT+03:00 ALEKSEY KUZNETSOV :
>>
>> > so what do u think on the issue ?
>> >
>> > чт, 30 Мар 2017 г., 17:49 ALEKSEY KUZNETSOV :
>> >
>> > > Hi ! Thanks for help. I've created ticket :
>> > > https://issues.apache.org/jira/browse/IGNITE-4887
>> > > and a commit :
>> > > https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06
>> > 436b638e5c
>> > > We really need this feature
>> > >
>> > > чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk <
>> > alexey.goncha...@gmail.com
>> > > >:
>> > >
>> > > Aleksey,
>> > >
>> > > I doubt your approach works as expected. Current transaction recovery
>> > > protocol heavily relies on the originating node ID in its internal
>> logic.
>> > > For example, currently a transaction will be rolled back if you want
>> to
>> > > transfer a transaction ownership to another node and original tx owner
>> > > fails. An attempt to commit such a transaction on another node may
>> fail
>> > > with all sorts of assertions. After transaction ownership changed, you
>> > need
>> > > to notify all current transaction participants about this change, and
>> it
>> > > should also be done failover-safe, let alone that you did not add any
>> > tests
>> > > for these cases.
>> > >
>> > > I back Denis here. Please create a ticket first and come up with clear
>> > > use-cases, API and protocol changes design. It is hard to reason about
>> > the
>> > > changes you've made when we do not even understand why you are making
>> > these
>> > > changes and how they are supposed to work.
>> > >
>> > > --AG
>> > >
>> > > 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV <
>> alkuznetsov...@gmail.com>:
>> > >
>> > > > So, what do u think on my idea ?
>> > > >
>> > > > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV <
>> > alkuznetsov...@gmail.com
>> > > >:
>> > > >
>> > > > > Hi! No, i dont have ticket for this.
>> > > > > In the ticket i have implemented methods that change transaction
>> > status
>> > > > to
>> > > > > STOP, thus letting it to commit transaction in another thread. In
>> > > another
>> > > > > thread you r going to restart transaction in order to commit it.
>> > > > > The mechanism behind it is obvious : we change thread id to newer
>> one
>> > > in
>> > > > > ThreadMap, and make use of serialization of txState, transactions
>> > > itself
>> > > > to
>> > > > > transfer them into another thread.
>> > > > >
>> > > > >
>> > > > > вт, 28 мар. 2017 г. в 20:15, Denis Magda :
>> > > > >
>> > > > > Aleksey,
>> > > > >
>> > > > > Do you have a ticket for this? Could you briefly list what exactly
>> > was
>> > > > > done and how the things work.
>> > > > >
>> > > > > —
>> > > > > Denis
>> > > > >
>> > > > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV <
>> > > > alkuznetsov...@gmail.com>
>> > > > > wrote:
>> > > > > >
>> > > > > > Hi, Igniters! I 've made implementation of transactions of
>> > non-single
>> > > > > > coordinator. Here you can start transaction in one thread and
>> > commit
>> > > it
>> > > > > in
>> > > > > > another thread.
>> > > > > > Take a look on it. Give your thoughts on it.
>> > > > > >
>> > > > > >
>> > > > > https://github.com/voipp/ignite/pull/10/commits/
>> > > > 3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
>> > > > > >
>> > > > > > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin <
>> > > sergi.vlady...@gmail.com
>> > > > >:
>> > > > > >
>> > > > > >> You know better, go ahead! :)
>> > > > > >>
>> > > > > >> Sergi
>> > > > > >>
>> > > > > >> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV <
>> > > > alkuznetsov...@gmail.com
>> > > > > >:
>> > > > > >>
>> > > > > >>> we've discovered several problems regarding your
>> "accumulation"
>> > > > > >>> approach.These are
>> > > > > >>>
>> > > > > >>>   1. perfomance issues when transfering data from temporary
>> cache
>> > > to
>> > > > > >>>   permanent one. Keep in mind big deal of concurent
>> transactions
>> > in
>> > > > > >>> Service
>> > > > > >>>   commiter
>> > > > > >>>   2. extreme memory load when keeping temporary cache in
>> memory
>> > > > > >>>   3. As long as user is not acquainted with ignite, working
>> with
>> > > > cache
>> > > > > >>>   must be transparent for him. Keep this in mind. User's node
>> can
>> > > > > >> evaluate
>> > > > > >>>   logic with no transaction at all, so we should 

[GitHub] ignite pull request #2217: IGNITE-5444 Fix serialization of SingletonList to...

[WilliamDo]

GitHub user WilliamDo opened a pull request:

https://github.com/apache/ignite/pull/2217

IGNITE-5444 Fix serialization of SingletonList to binary format



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/WilliamDo/ignite IGNITE-5444

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2217.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2217


commit 2805fb89feedfe5c66e61161f4876a76b999e28b
Author: William Do 
Date:   2017-06-29T11:45:38Z

IGNITE-5444 Fix serialization of SingletonList to binary format




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Optimize transaction prepare step when store is enabled IGNITE-1553

[ALEKSEY KUZNETSOV]

Sorry for delay. I still need to work on this ticket. Let's leave it In
Progress

чт, 29 июн. 2017 г. в 13:08, Dmitry Pavlov :

> Hi Aleksey,
>
> I am not able to change status for this issue. Could you please change to
> "Patch Available"?
>
> Best Regards,
> Dmitry Pavlov
>
> пт, 23 июн. 2017 г. в 22:17, ALEKSEY KUZNETSOV :
>
> > Yeah, it can be set "Patch available" because I need smbd to review it
> >
> > пт, 23 июн. 2017 г., 21:06 Dmitry Pavlov :
> >
> > > Hi Alexey,
> > >
> > > This ticket status is 'in progress'.
> > > Is it still in development or 'patch available' can be set?
> > >
> > > Sincerely,
> > > Dmitry Pavlov
> > >
> > >
> > >
> > > ср, 7 июн. 2017 г. в 17:03, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > >
> > > > Hi, Igntrs!
> > > > Feel free to reviewing the ticket
> > > > https://issues.apache.org/jira/browse/IGNITE-1553
> > > > https://github.com/apache/ignite/pull/2091
> > > > --
> > > >
> > > > *Best Regards,*
> > > >
> > > > *Kuznetsov Aleksey*
> > > >
> > >
> > --
> >
> > *Best Regards,*
> >
> > *Kuznetsov Aleksey*
> >
>
-- 

*Best Regards,*

*Kuznetsov Aleksey*

IGNITE-4365: Data grid in deadlock on stop by DataStreamerImpl

[Александр Меньшиков]

Hello,

I want to make ticket IGNITE-4365
. The problem came from
DataStreamerImpl.
There are methods which use DataStreamerImpl under the lock
(GridCacheGateway), but the method DataStreamerImpl#doFlush() has a
"while(true)" loop. And in case when someone is calling the
GridCacheGateway#onStopped(), application can get stuck in the loop in
DataStreamerImpl#doFlush(), and in trying get a lock in
GridCacheGateway#onStopped().

So I need an expert opinion about DataStreamerImpl#doFlush().
1) Can I just drop unfinished futures in DataStreamerImpl#doFlush() when
someone is calling GridCacheGateway#onStopped()? I can track it by adding a
volatile boolean flag in the GridCacheGateway.
2) Or better to modify a futures execution DataStreamerImpl#load0() to use
onDone with an exception or something like that?

Methods which use or might use DataStreamerImpl under the lock:

1) GridCacheAdapter#localLoad()
2) GridCacheAdapter#localLoadAndUpdate()
3) GridCacheAdapter#localLoadCache()
4) GridDistributedCacheAdapter.GlobalRemoveAllJob#localExecute() (it
exectly happen in thread dump in ticket)

[jira] [Created] (IGNITE-5618) .NET: Ignite fails with OOM on startup under x86 (32 bit) with default configuration

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-5618:
--

 Summary: .NET: Ignite fails with OOM on startup under x86 (32 bit) 
with default configuration
 Key: IGNITE-5618
 URL: https://issues.apache.org/jira/browse/IGNITE-5618
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 2.0
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.1


Default MemoryPolicyConfiguration settings do not take process bitness into 
account. We can't use 80% of RAM under 32-bit mode.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Get a ticket

[Elen Code]

Hi Dmitry,
How can it get permissions to assign to me?I guess I need contributor status or 
something like that.
Could you please help me with that?
Regards, 

On Thursday, June 29, 2017 2:06 PM, Dmitry Pavlov  
wrote:
 

 Hi Andrei ,

IGNITE-5597  is
unassigned.
Usually such issue can be picked up by any community member.
So please feel free to assign issue to yourself.

Best Regards,
Dmitry Pavlov

чт, 29 июн. 2017 г. в 9:25, Elen Code :

> Dear Ignite team,
>
> I would like to start my first contribution with ticket
> https://issues.apache.org/jira/browse/IGNITE-5597.
> If it is ok, I'll assign it to myself.
> Please confirm,
> Kind regards,
> Andrei
>
>

   

[GitHub] ignite pull request #2208: for test purposes

[AMashenkov]

Github user AMashenkov closed the pull request at:

https://github.com/apache/ignite/pull/2208


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Optimize transaction prepare step when store is enabled IGNITE-1553

[Dmitry Pavlov]

Hi Aleksey,

I am not able to change status for this issue. Could you please change to
"Patch Available"?

Best Regards,
Dmitry Pavlov

пт, 23 июн. 2017 г. в 22:17, ALEKSEY KUZNETSOV :

> Yeah, it can be set "Patch available" because I need smbd to review it
>
> пт, 23 июн. 2017 г., 21:06 Dmitry Pavlov :
>
> > Hi Alexey,
> >
> > This ticket status is 'in progress'.
> > Is it still in development or 'patch available' can be set?
> >
> > Sincerely,
> > Dmitry Pavlov
> >
> >
> >
> > ср, 7 июн. 2017 г. в 17:03, ALEKSEY KUZNETSOV  >:
> >
> > > Hi, Igntrs!
> > > Feel free to reviewing the ticket
> > > https://issues.apache.org/jira/browse/IGNITE-1553
> > > https://github.com/apache/ignite/pull/2091
> > > --
> > >
> > > *Best Regards,*
> > >
> > > *Kuznetsov Aleksey*
> > >
> >
> --
>
> *Best Regards,*
>
> *Kuznetsov Aleksey*
>

Re: Get a ticket

[Dmitry Pavlov]

Hi Andrei ,

IGNITE-5597  is
unassigned.
Usually such issue can be picked up by any community member.
So please feel free to assign issue to yourself.

Best Regards,
Dmitry Pavlov

чт, 29 июн. 2017 г. в 9:25, Elen Code :

> Dear Ignite team,
>
> I would like to start my first contribution with ticket
> https://issues.apache.org/jira/browse/IGNITE-5597.
> If it is ok, I'll assign it to myself.
> Please confirm,
> Kind regards,
> Andrei
>
>

[GitHub] ignite pull request #2214: For testing.

[mcherkasov]

Github user mcherkasov closed the pull request at:

https://github.com/apache/ignite/pull/2214


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2216: For testing

[mcherkasov]

GitHub user mcherkasov opened a pull request:

https://github.com/apache/ignite/pull/2216

For testing



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.1.2-5521

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2216.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2216


commit b3e0753e78d21549ee9c1b834d682cb5eed1edc1
Author: Pavel Kovalenko 
Date:   2017-06-14T17:53:22Z

Fail CacheLateAffinityAssignmentTest#testAffinitySimpleStopRandomNode with 
known issue

commit 162ad951c9f61a2fbe10e1fd26490560113393b8
Author: Pavel Kovalenko 
Date:   2017-06-14T18:01:30Z

Fail GridCachePartitionedNodeRestartTest with known issue

commit 666aab9f345743f5faff3751bd8bb14296bceaca
Author: devozerov 
Date:   2017-06-14T18:06:11Z

IGNITE-5487: Fixed.

commit abf3ef9e154af86dad0d691dacfa1197a9d34aed
Author: Pavel Kovalenko 
Date:   2017-06-14T18:06:37Z

Fail IgniteClientReconnectFailoverTest with known issue

commit c50c1da4d3d70c2de3c5ce2c3944977f770ffb62
Author: devozerov 
Date:   2017-06-14T18:06:55Z

Merge remote-tracking branch 'upstream/ignite-2.1.1' into ignite-2.1.1

commit 1a2a0ed560013bb8e3e0d64fc665efe527ae886e
Author: devozerov 
Date:   2017-06-14T18:07:46Z

Merge remote-tracking branch 'upstream/ignite-2.1.1' into ignite-2.1.1

commit c828e0cfb1bd4700883cc6809f4ebe8ef1f935cf
Author: devozerov 
Date:   2017-06-14T18:17:08Z

Fixed JettyRestProcessorAbstractSelfTest.testExe.

commit 1a52eb5291090e43a8e5da21fd931cc3cae37e50
Author: devozerov 
Date:   2017-06-14T18:45:21Z

Fixed JettyRestProcessorAbstractSelfTest.testVisorGateway.

commit 4c5d9b2163f7f233d06bf714c01d4c07097ce120
Author: devozerov 
Date:   2017-06-14T18:59:59Z

Fixed JavaDoc.

commit 3fd9e041942f7faa547f2a152ce84fc4b1641028
Author: shroman 
Date:   2017-06-15T00:33:57Z

IGNITE-5229: Specify caches when using Redis protocol. Implemented via 
SELECT command. - Fixes #2098.

Signed-off-by: shroman 

commit b0565727dd696ce89cd124b56c2715f1234ce9b8
Author: Ilya Lantukh 
Date:   2017-06-15T05:19:13Z

Improved cacheName / groupName validation not to allow names reserved for 
data structures.

commit 83e82c12bc93a0347b135dcc535d2a7255c05fc9
Author: Ilya Lantukh 
Date:   2017-06-15T05:34:49Z

Replaced grid UUID with gridStartTime to check that volatile DS is outdated.

commit ca9b068f557f26f466278c4b0344ded952185d28
Author: Ilya Lantukh 
Date:   2017-06-15T05:35:23Z

Renaming.

commit 4026ddcc2161ca4bf5ad3dc04cabd284c608738a
Author: sboikov 
Date:   2017-06-15T06:23:36Z

ignite-5272 Do not use synchronous custom discovery event for client cache 
start/close

commit d15e2bde13d9a01484ea357e46f9a10a29fee227
Author: sboikov 
Date:   2017-06-15T07:26:27Z

Fixed exception conversion in DataStreamerImpl.

commit 05fd6804e769c98a3d08556cbd2fe6c7efcaa258
Author: Ilya Lantukh 
Date:   2017-06-15T07:44:54Z

Fixed marshalling of GridDhtPartitionMap.

commit 037ee6f4555a0de403bb829e0983a715251191d9
Author: Ilya Lantukh 
Date:   2017-06-15T08:05:45Z

Fixed tests.

commit 7f55b582b5181a6a1093260664da8e3677b843c4
Author: sboikov 
Date:   2017-06-15T08:11:51Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-2.1.1

commit 92f284f3f303d6877f97b718755c2bc0c242ad4e
Author: sboikov 
Date:   2017-06-15T08:13:05Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-2.1.1

commit f661fc5ee457f753f388c8f19a16d30277287d41
Author: sboikov 
Date:   2017-06-15T08:13:37Z

Merge remote-tracking branch 'community/ignite-2.1.1' into ignite-2.1.1

commit b48f4dea994fee47c8da4ddd00d9e9dc6d412e5a
Author: Dmitriy Govorukhin 
Date:   2017-06-15T08:21:26Z

ignite-2.1.1 remove unnecessary condition

commit 00a024c6563e694acc3ff08557548451a6c4012e
Author: Ilya Lantukh 
Date:   2017-06-15T08:24:23Z

Merge branches 'ignite-2.1.1' and 'ignite-5267-ds' of 
https://github.com/gridgain/apache-ignite into ignite-5267-ds

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/DataStructuresProcessor.java

commit 973cc1caf3986895b5161498091d9821a8711978
Author: Andrey Novikov 
Date:   2017-06-15T09:01:50Z

IGNITE-5494 Web console: Improved admin panel 

[GitHub] ignite pull request #2201: IGNITE-5521: Large near caches lead to cluster in...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2201


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5616) Add ability to change Amazon S3 URL (=support for on premise Minio)

[Bernhard Rausch (JIRA)]

Bernhard Rausch created IGNITE-5616:
---

 Summary: Add ability to change Amazon S3 URL (=support for on 
premise Minio)
 Key: IGNITE-5616
 URL: https://issues.apache.org/jira/browse/IGNITE-5616
 Project: Ignite
  Issue Type: Improvement
  Components: aws
Affects Versions: 2.0
Reporter: Bernhard Rausch


It would be nice if it would be possible to change the Amazon S3 URL to support 
an on premise S3 storage like Minio.
For now there is no easy ability to do that via the XML configuration because 
as we saw fixed S3 URL's in the apache ignite code...

For example something like that:

  
  
  




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5615) .NET: IgniteConfiguration.LocalEventListeners

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-5615:
--

 Summary: .NET: IgniteConfiguration.LocalEventListeners
 Key: IGNITE-5615
 URL: https://issues.apache.org/jira/browse/IGNITE-5615
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.1


Propagate {{IgniteConfiguration.LocalEventListeners}} to .NET. This allows 
catching all events right from the node start.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5614) .NET: IDataStreamer.AddData should take IEnumerable

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-5614:
--

 Summary: .NET: IDataStreamer.AddData should take IEnumerable
 Key: IGNITE-5614
 URL: https://issues.apache.org/jira/browse/IGNITE-5614
 Project: Ignite
  Issue Type: Improvement
  Components: platforms
Affects Versions: 2.0
Reporter: Pavel Tupitsyn
 Fix For: 2.2


There is {{IDataStreamer.AddData(ICollection> entries)}}, 
which is not consistent with other APIs like 
{{ICache.PutAll(IEnumerable> vals)}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5613) AtomicSequence usage inside transactions may cause deadlock

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-5613:


 Summary: AtomicSequence usage inside transactions may cause 
deadlock
 Key: IGNITE-5613
 URL: https://issues.apache.org/jira/browse/IGNITE-5613
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 1.5.0.final
Reporter: Alexey Goncharuk
 Fix For: 2.1


Consider the following update scenario:
{code}
Thread 1:
Transaction tx = txStart() {
get(key); // Acquires lock;
seq.incrementAndGet();
}

Thread 2:
seq.incrementAndGet();
{code}

Let's now assume that:
 * Sequence is exhausted and needs a non-local update
 * Thread 1 acquired lock on topology version N
 * Topology version changes
 * Thread 2 now calls incrementAndGet(), updates the sequence local guard and 
starts transaction which waits for topology version N+1 to become available
 * Thread 1 attemts to incrementAndGet().

Since guard is already changed, it waits for the concurrent update to complete



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: By bytes access to binary format

[Vladislav Pyatkov]

Val,

I proposal, access as a stream to binary object, because we have doubled
copy on touch a field (first at copy from cache and second - on getting a
field).

For the stream in/out to cache I will be used IGFS.
Main idea to avoid GC pressure when make a massive read from key-value
storage.

On Wed, Jun 28, 2017 at 9:36 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Vladislav,
>
> Are you suggesting to stream directly from cache. or from a binary object
> that is already copied from cache?
>
> -Val
>
> On Wed, Jun 28, 2017 at 2:52 AM, Vladislav Pyatkov 
> wrote:
>
> > Hi,
> >
> > Recently, from one of Ignite user, I listened interest idea.
> > What if I want to pass some date to java stream from cache.
> >
> > With binary I do it like this:
> >
> > BinaryObject get = (BinaryObject) cache.get(key);
> > byte[] dataFromCache = get.field("data");
> > System.out.write(dataFromCache, 0, dataFromCache.length);
> >
> > But in this case we got garbage a lot, due to each time new bytes array
> is
> > creating.
> >
> > This will lead to many GC events in case we load a some of million
> entries.
> > Could we offer additional API for working with java stream:
> >
> > BinaryObject.writeBytesToBuf("data", ByteBuffer.allocate(1024));
> >
> > or with buffer
> >
> > BinaryObject.writeBytesToBuf("data", new byte[1000], 100);
> >
> > I already created a Jira ticket.
> > https://issues.apache.org/jira/browse/IGNITE-5602
> >
> > --
> > Vladislav Pyatkov
> > Architect-Consultant "GridGain Rus" Llc.
> > +7 963 716 68 99
> >
>



-- 
Vladislav Pyatkov
Architect-Consultant "GridGain Rus" Llc.
+7 963 716 68 99

[jira] [Created] (IGNITE-5612) JdbcResultSet#next is fetching all the data into memory and fetch size is used only to data-column mapping

[Anil Dasari (JIRA)]

Anil Dasari created IGNITE-5612:
---

 Summary: JdbcResultSet#next is fetching all the data into memory 
and fetch size is used only to data-column mapping
 Key: IGNITE-5612
 URL: https://issues.apache.org/jira/browse/IGNITE-5612
 Project: Ignite
  Issue Type: Bug
Affects Versions: 1.9
Reporter: Anil Dasari


 GridReduceQueryExecutor is fetching all the records and then returns the 
results as per fetch results. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Get a ticket

[Elen Code]

Dear Ignite team,

I would like to start my first contribution with ticket 
https://issues.apache.org/jira/browse/IGNITE-5597.
If it is ok, I'll assign it to myself.
Please confirm,
Kind regards,
Andrei

RE: golang client for Ignite

[Aleksandr Sokolovskii]

Guys,

Thanks for your answers.

So If users want to use Ignite DataGrid from “unsupported” language they have 
two choice for now:

1) REST API
It supports: cache management, put-get, sql exec queries, fetch
It does not support: sql transactions
REST API is easy to use.
REST API has a lot of overhead (HTTP, json marshalling/unmarshalling,.. you 
know)

2) develop pure SQL driver for Ignite ODBC endpoint
It supports: sql exec queries, fetch
It does not support: cache management, put-get, sql transactions
Roman is right – using cgo is not very good idea.
Honestly It’s not trivial task to develop pure SQL driver for Ignite ODBC 
endpoint.
I spent some hours to remember how STL serializes std::string to unmarshall it 
in golang. )))
My last C++ experience was in 2004. )))
Another my concern is C++ wrapper over Ignite Java process.
I try to explain:
The main reason to develop something using golang is performance.
People how use golang are enslaved by performance.
They think how to avoid unnecessary memory allocation, avoid reflection, etc.
Go provides goroutine instead of system threads to avoid memory allocation and 
cross-threads switching.
Cross-process switching (cpp->jvm->cpp) on server side of in-memory database 
looks like “shot yourself in foot” for such people. )))

There is another way: to implement native client for H2 DB protocol.
(If I’m not mistaken Ignite use H2 DB protocol for JDBC endpoint).
Again it’s not trivial task to implement pure native H2 DB client for each 
necessary programming language.

What do you think about to implement gRPC (or Apache Thrift, or…) 
platform-language-neutral protocol endpoint in Java core of Ignite?
I resolves “unsupported” language client problem.

Thanks,
Aleksandr

From: Roman Shtykh
Sent: 29 июня 2017 г. 6:36
To: dev@ignite.apache.org
Subject: Re: golang client for Ignite

Guys, 

Just an observation, but when I introduce Ignite to other developers, they get 
very interested. But when the question goes to the support of the language they 
use for development, often they look disappointed because a chunk of 
functionalities I introduce becomes inaccessible. I think from their NoSQL 
experience they expect a client protocol.
As for Golang, from what I know, you have to write bindings in C for C++ and 
use them. There are performance concerns with cgo.
-- Roman


On Thursday, June 29, 2017 9:34 AM, Dmitriy Setrakyan 
 wrote:
 

 Denis,

Perhaps adding the same level of support for Golang as we have for .NET or
C++ would be asking too much. The reason we start a JVM in .NET and C++ is
because we implemented the full Ignite API support, even including
transactions and executing .NET closures on remote Java servers.

Perhaps, from Golang client standpoint, it is enough to implement it
directly over the REST protocol initially.

D.

On Wed, Jun 28, 2017 at 3:59 PM, Denis Magda  wrote:

> Aleksandr,
>
> > I take a look into ignite cpp core.
> > Looks like it attaches to running jvm and calls java methods.
> > Please tell me that I’m wrong! )))
>
> That’s a correct observation. Both C++ and .NET clients spawn a JVM
> process underneath and redirect almost all the requests to it. That
> approach allowed us to support these languages easily. Otherwise, it would
> have taken ages to develop true C++ and .NET libs.
>
> > It will not work for golang.
>
> Why?
>
> > Is there language-neutral protocol to communicate with Ignite server?
>
> REST, ODBC and JDBC only.
>
> —
> Denis
>
>
> > On Jun 28, 2017, at 12:43 PM, Aleksandr Sokolovskii 
> wrote:
> >
> > Denis,
> >
> > I take a look into ignite cpp core.
> > Looks like it attaches to running jvm and calls java methods.
> > Please tell me that I’m wrong! )))
> >
> > It will not work for golang.
> > Is there language-neutral protocol to communicate with Ignite server?
> >
> > Thanks,
> > Aleksandr
> >
> > From: Aleksandr Sokolovskii
> > Sent: 28 июня 2017 г. 22:26
> > To: dev@ignite.apache.org
> > Subject: RE: golang client for Ignite
> >
> > Hi Deins,
> >
> > Thanks for answer.
> >
> > Link helps.
> >
> > It’s not good practice to call “c” functions from golang.
> > That’s why I develop client by pure golang without Ignite “cpp” client
> API.
> >
> > I already tested golang “handshake” using ODBC endpoint server:10800.
> > It works great.
> > But I don’t see support of transactions via ODBC...
> >
> > Yes, I would like to  develop true native client.
> > How can I get protocol specification to develop client BinaryMarshaller
> with pure golang?
> >
> > Thanks,
> > Aleksandr
> >
> > From: Denis Magda
> > Sent: 28 июня 2017 г. 2:00
> > To: dev@ignite.apache.org
> > Subject: Re: golang client for Ignite
> >
> > Hi Aleksandr,
> >
> > That looks really interesting to me. Personally, I would like to see a
> dedicated Go module in Ignite.
> >
> > Do you support SQL API right now? If it’s so then you might want to
> switch to Ignite JDBC driver instead that should