date:20170123

Re: Abandon of the non-throwing version of C++ client API

2017-01-23 Thread Vladimir Ozerov

Originally I tried to cover situation when user disabled exceptions. In
this case throwing something will lead to application crash. However, for
now I believe that such use case is not very likely as Ignite is relatively
high-level product. For this reason I would prefer to keep clean and
compact API which will throw exceptions. But still have a workaround for
users which do not want that. For instance, we may consult to some
environment variable in runtime, or to some preprocessor flag in compile
time, and expose additional static "GetLastError" method.

On Mon, Jan 23, 2017 at 11:35 PM, Denis Magda  wrote:

> Guys,
>
> I found the initial discussion from the early times of our C++ client:
> http://apache-ignite-developers.2346864.n4.nabble.
> com/C-exception-handling-strategy-td778.html
>
> Vovan, that time you were on the side of error-code methods. Why have you
> changed you mind proposing to make the throwing version default one? Any
> new tendency in C++ community?
>
> —
> Denis
>
> On Jan 23, 2017, at 2:56 AM, Vladimir Ozerov  wrote:
>
> +1 to Igor's idea. Ignite is relatively high-level product and we do not
> expect ultra-optimized users who cannot allow exceptions to be enabled.
> Macros should be a good workaround for them, though.
>
> On Sat, Jan 21, 2017 at 6:47 PM, Denis Magda  wrote:
>
>> Hi Igor,
>>
>> My C++ experience is based only on error code methods. This is why I
>> thought that exceptions based approach is unrelated to C++ at all.
>>
>> I do remember we discussed all the pros and cons of these ways before.
>> Could you find that old discussion and share it here? I'm on a mobile now,
>> not easy to do on my own.
>>
>> Denis
>>
>>
>> On Friday, January 20, 2017, Igor Sapego  wrote:
>>
>>> Hi Igniters,
>>>
>>> I'm the guy who mostly contribute in C++ Ignite client and I
>>> need your advice. Mostly I'd like to hear from our users and
>>> those who are experienced in C++. Currently we have two
>>> versions of most API methods - the throwing one and the
>>> one that returns error through output argument. This was initially
>>> done because we were not sure which way of error-reporting
>>> is going to be preferred by our users.
>>>
>>> Now this approach bloats C++ API a lot and makes it harder to
>>> maintain and optimize code. I propose like to abandon and deprecate
>>> non-throwing version of API and only leave throwing version,
>>> but first I want to hear from you guys - what do you think? Does
>>> anyone use non-throwing version of the API? Maybe your toolchain
>>> does not support exceptions or are you disabling them on purpose?
>>>
>>> For those who prefer disabling exceptions I propose to introduce
>>> some macros like IGNITE_DISABLE_EXCEPTIONS and add
>>> some thread-local error-storing mechanism like ignite::GetLastError().
>>>
>>> What do you guys think?
>>>
>>> Best Regards,
>>> Igor
>>>
>>
>
>

Ignite cluster will be jammed for long-running queries

2017-01-23 Thread ght230

My ingnite cluster will be jammed after running for server days, I found many
warnning such as "Query execution is too long" from log file.

I have noticed there is a jira which is similar to my problem, the link is
https://issues.apache.org/jira/browse/IGNITE-4105.

In this jira, the Ignite version is 1.7. My ignite version is 1.6.8, I think
perhaps version 1.6.8 also has the same problem.

I still want to know when will this jira be solved, is there a plan?
The solution to this problem is very important to us.

Are there any temporary solutions before this jira resolved?
At present, increase the system thread pool will be useful?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-cluster-will-be-jammed-for-long-running-queries-tp10210.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Timed out waiting for message delivery receipt

2017-01-23 Thread hitendrapratap

I has been resolved when increased "ignite.failure.detection.timeout" to
60sec from 10sec.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Timed-out-waiting-for-message-delivery-receipt-tp10171p10209.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Timed out waiting for message delivery receipt

2017-01-23 Thread hitendrapratap

Please find the server and client configuration and let me know if you find
anything wrong.

*Server Configuration*:
IgniteConfiguration igniteConfiguration = new IgniteConfiguration();

igniteConfiguration.setConnectorConfiguration(null);

TcpCommunicationSpi comm = new TcpCommunicationSpi();
comm.setLocalPort(47100); 
igniteConfiguration.setCommunicationSpi(comm);

TcpDiscoverySpi spi = new TcpDiscoverySpi();
spi.setLocalPort(47500); 

TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
   
ipFinder.setAddresses("SERVER1_IP,SERVER2_IP,SERVER3_IP,SERVER4_IP");
spi.setIpFinder(ipFinder);

igniteConfiguration.setDiscoverySpi(spi);

igniteConfiguration.setTimeServerPortBase(31100); 
igniteConfiguration.setFailureDetectionTimeout(6);

*Client Configuration*:
IgniteConfiguration igniteConfiguration = new IgniteConfiguration();
igniteConfiguration.setClientMode(true);

TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi();
discoverySpi.setLocalPort(47500);

TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
   
ipFinder.setAddresses("SERVER1_IP,SERVER2_IP,SERVER3_IP,SERVER4_IP");
discoverySpi.setIpFinder(ipFinder);

igniteConfiguration.setDiscoverySpi(discoverySpi);

igniteConfiguration.setFailureDetectionTimeout(6);



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Timed-out-waiting-for-message-delivery-receipt-tp10171p10208.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: REST servive command LOG

2017-01-23 Thread Denis Magda

I see, sorry for confusion. I though this is a new PR :)

—
Denis

> On Jan 23, 2017, at 1:13 PM, Saikat Maitra  wrote:
> 
> Hi Denis,
> 
> The changes are already merged long time back and I wanted to share the old 
> jira ticket and PR for reference.
> 
> Regards
> Saikat
> 
> On Mon, Jan 23, 2017 at 1:33 PM, Denis Magda  > wrote:
> Saikat, 
> 
> Could you create a JIRA ticket providing more info why the modifications are 
> required and paste a link to the pull-request there?
> 
> Otherwise the dev community might miss your changes.
> 
> —
> Denis
> 
>> On Jan 22, 2017, at 10:16 AM, Saikat Maitra > > wrote:
>> 
>> Hello Sam,
>> 
>> Due to security reason the log file need to be chrooted to IGNITE_HOME/log.
>> 
>> Please review this PR for the fix.
>> 
>> https://github.com/apache/ignite/pull/420 
>> 
>> 
>> Regards,
>> Saikat
>> 
>> 
>> On Sun, Jan 22, 2017 at 12:03 PM, Saikat Maitra > > wrote:
>> Hi Sam,
>> 
>> You can pass path as query param to get the log details. Please see this 
>> jira ticket below for reference.
>> 
>> https://issues.apache.org/jira/browse/IGNITE-944 
>> 
>> 
>> Regards
>> Saikat
>> 
>> On Thu, Jan 19, 2017 at 2:21 PM, javastuff@gmail.com 
>>  > > wrote:
>> Hi,
>> 
>> I find LOG command very useful. Periodic metrics and topology change
>> information helps monitoring.
>> 
>> However it does not work with my application, It fails with below message -
>> "Request parameter 'path' must contain a path to valid log file."
>> 
>> My application already using Log4j, so not using Ignite configuration XML to
>> configure Ignite logging. Instead application log4j configured to push all
>> Ignite INFO logs to a separate rolling file. Logging works fine. But
>> 
>> 1. Ignite configuration does not have logging configuration details.
>> 2. Log files can be anywhere, not under IGNITE_HOME
>> 
>> And because of above 2 points LOG command fails.
>> 
>> How can I make it work?
>> 
>> Thanks,
>> -Sam
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> --
>> View this message in context: 
>> http://apache-ignite-users.70518.x6.nabble.com/REST-servive-command-LOG-tp10148.html
>>  
>> 
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com 
>> .
>> 
>> 
> 
>

答复: ignite client memory issue

2017-01-23 Thread Shawn Du

Hi Denis,

 

This is the code, more things: 

#1 We store a *table* in each cache entry. 

#2 Table has several columns, column may be a big object like more than 10k 
bytes. Column implement interface Binarylizable.

 

I think you can try to produce this issue by design a class like above table. 
Also if you need heap dump, I can share it also.

 

Thanks

Shawn

 

private void save(Context context)
{
BinaryObjectBuilder builder = 
IgniteManager.getIgnite().binary().builder(context.cacheName);
builder.setField(IgniteConstants.COLUMN_TIMESTAMP, 
context.table.getTimestamp() / 1000);
builder.setField(IgniteConstants.COLUMN_SITE, context.table.getSite());
builder.setField(IgniteConstants.COLUMN_PRODUCT, 
context.table.getProduct());
context.table.getDimColumns().forEach(c -> builder.setField(c.getName(), 
c));
builder.setField(context.table.getMeasColumn().getName(), 
context.table.getMeasColumn());
IgniteCache cache = getCache(context);
cache.put(generateKey(context.table), builder.build());
}

private CacheConfiguration 
createCacheConfiguration(Context context)
{
CacheConfiguration config = new 
CacheConfiguration<>();
List columns = new ArrayList<>();
columns.add(new ColumnScheme(IgniteConstants.COLUMN_TIMESTAMP, 
Long.class.getTypeName(),
commonConfig.indexingTimestampEnable, false));
columns.add(new ColumnScheme(IgniteConstants.COLUMN_SITE, 
commonConfig.indexingSiteEnable));
columns.add(new ColumnScheme(IgniteConstants.COLUMN_PRODUCT, 
commonConfig.indexingProductEnable));
for (Column column : context.table.getDimColumns())
{
columns.add(new ColumnScheme(column.getName(), 
Column.class.getTypeName(), false));
}
String measName = context.table.getMeasColumn().getName();
columns.add(new ColumnScheme(measName, Column.class.getTypeName(), false));
config.setQueryEntities(Collections
.singleton(IgniteManager.createEntity(String.class.getTypeName(), 
context.cacheName, columns)));
config.setName(context.cacheName);
config.setMemoryMode(commonConfig.cacheMemoryMode);
config.setBackups(commonConfig.backups);
config.setStartSize(10_000);
config.setCopyOnRead(commonConfig.copyOnRead);
return config;
}

private IgniteCache getCache(Context context)
{
IgniteCache cache = 
IgniteManager.getIgnite().cache(context.cacheName);
if (cache == null)
{
cache = 
IgniteManager.getIgnite().getOrCreateCache(createCacheConfiguration(context));
   }
if (context.ttl > 0)
{
cache = cache.withExpiryPolicy(new ModifiedExpiryPolicy(new 
Duration(TimeUnit.SECONDS, context.ttl)));
}
cache = cache.withKeepBinary();
return cache;
}

static class Context
{
final long ttl;
final Table table;
final String cacheName;

Context(long ttl, Table table)
{
this.ttl = ttl;
this.table = table;
this.cacheName = table.getSchema().generateCacheName();
}
}

 

 

发件人: Denis Magda [mailto:dma...@apache.org] 
发送时间: 2017年1月24日 3:30
收件人: user@ignite.apache.org
主题: Re: ignite client memory issue

 

Hi,

 

Share piece of the code that produces the leak.

 

—

Denis

 

On Jan 22, 2017, at 5:56 PM, Shawn Du  > wrote:

 

Hi,

 

My application run overnight  and crash again after I set max heap as 2G. For I 
saw there were many Future Objects,

I guess it may be caused by Async API. Now I am using Sync API. It seems that 
the memory issue disappeared. 

My application memory kept at 60M.

 

cache = cache.withAsync().withKeepBinary(); -->

cache = cache.withKeepBinary();

 

I think this will not happens always, but real happened on some condition and 
worthy further investigate.

 

Thanks

Shawn

 

发件人: Shawn Du [mailto:shawn...@neulion.com.cn] 
发送时间: 2017年1月22日 17:36
收件人: user@ignite.apache.org  
主题: RE: ignite client memory issue

 

Hi,

 

I am sure there are memory leaks.   See below.  

 

Class Name  
 | Objects 
| Shallow Heap |  Retained Heap

-

java.lang.Thread
   | 128 |  
 15,360 | >= 879,574,832

java.lang.ThreadLocal$ThreadLocalMap
   | 100 |2,400 | >= 
870,478,984

java.lang.ThreadLocal$ThreadLocalMap$Entry[]
| 100 |  271,168 | >=

RE: ignite client memory issue

2017-01-23 Thread Shawn Du

Thanks val, you are right, I am using sync API. It is fast enough for me. 

Thanks
Shawn

-邮件原件-
发件人: vkulichenko [mailto:valentin.kuliche...@gmail.com] 
发送时间: 2017年1月24日 3:39
收件人: user@ignite.apache.org
主题: Re: 答复: ignite client memory issue

Shawn,

It looks like you had too many asynchronous operations executed at the same
time and therefore too many futures created in memory. Async approach
requires more accuracy, and if everything works for you with sync
operations, I would use them.

-Val



--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/ignite-client-memory-issue-tp
10174p10201.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Timed out waiting for message delivery receipt

2017-01-23 Thread hitendrapratap

Openstack enviornment. I am using Ignite 1.8 version and as an embedded
within the application. I did validated iptables and ports connectivity but
everything is good. 

Below are some more messages:

1.Failed to send message to next node
2. Local node has detected failed nodes and started cluster-wide procedure.
To speed up failure detection please see 'Failure Detection' section under
javadoc for 'TcpDiscoverySpi'
-> I tried increasing this too. But didn't helped.
3."Node FAILED: TcpDiscoveryNode



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Timed-out-waiting-for-message-delivery-receipt-tp10171p10204.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: REST servive command LOG

2017-01-23 Thread Saikat Maitra

Hi Denis,

The changes are already merged long time back and I wanted to share the old
jira ticket and PR for reference.

Regards
Saikat

On Mon, Jan 23, 2017 at 1:33 PM, Denis Magda  wrote:

> Saikat,
>
> Could you create a JIRA ticket providing more info why the modifications
> are required and paste a link to the pull-request there?
>
> Otherwise the dev community might miss your changes.
>
> —
> Denis
>
> On Jan 22, 2017, at 10:16 AM, Saikat Maitra 
> wrote:
>
> Hello Sam,
>
> Due to security reason the log file need to be chrooted to
> IGNITE_HOME/log.
>
> Please review this PR for the fix.
>
> https://github.com/apache/ignite/pull/420
>
> Regards,
> Saikat
>
>
> On Sun, Jan 22, 2017 at 12:03 PM, Saikat Maitra 
> wrote:
>
>> Hi Sam,
>>
>> You can pass path as query param to get the log details. Please see this
>> jira ticket below for reference.
>>
>> https://issues.apache.org/jira/browse/IGNITE-944
>>
>> Regards
>> Saikat
>>
>> On Thu, Jan 19, 2017 at 2:21 PM, javastuff@gmail.com <
>> javastuff@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I find LOG command very useful. Periodic metrics and topology change
>>> information helps monitoring.
>>>
>>> However it does not work with my application, It fails with below
>>> message -
>>> "Request parameter 'path' must contain a path to valid log file."
>>>
>>> My application already using Log4j, so not using Ignite configuration
>>> XML to
>>> configure Ignite logging. Instead application log4j configured to push
>>> all
>>> Ignite INFO logs to a separate rolling file. Logging works fine. But
>>>
>>> 1. Ignite configuration does not have logging configuration details.
>>> 2. Log files can be anywhere, not under IGNITE_HOME
>>>
>>> And because of above 2 points LOG command fails.
>>>
>>> How can I make it work?
>>>
>>> Thanks,
>>> -Sam
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context: http://apache-ignite-users.705
>>> 18.x6.nabble.com/REST-servive-command-LOG-tp10148.html
>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>
>>
>>
>
>

Re: Abandon of the non-throwing version of C++ client API

2017-01-23 Thread Denis Magda

Guys,

I found the initial discussion from the early times of our C++ client:
http://apache-ignite-developers.2346864.n4.nabble.com/C-exception-handling-strategy-td778.html
 


Vovan, that time you were on the side of error-code methods. Why have you 
changed you mind proposing to make the throwing version default one? Any new 
tendency in C++ community? 

—
Denis

> On Jan 23, 2017, at 2:56 AM, Vladimir Ozerov  wrote:
> 
> +1 to Igor's idea. Ignite is relatively high-level product and we do not 
> expect ultra-optimized users who cannot allow exceptions to be enabled. 
> Macros should be a good workaround for them, though.
> 
> On Sat, Jan 21, 2017 at 6:47 PM, Denis Magda  > wrote:
> Hi Igor,
> 
> My C++ experience is based only on error code methods. This is why I thought 
> that exceptions based approach is unrelated to C++ at all.
> 
> I do remember we discussed all the pros and cons of these ways before. Could 
> you find that old discussion and share it here? I'm on a mobile now, not easy 
> to do on my own.
> 
> Denis
> 
> 
> On Friday, January 20, 2017, Igor Sapego  > wrote:
> Hi Igniters,
> 
> I'm the guy who mostly contribute in C++ Ignite client and I 
> need your advice. Mostly I'd like to hear from our users and 
> those who are experienced in C++. Currently we have two 
> versions of most API methods - the throwing one and the 
> one that returns error through output argument. This was initially
> done because we were not sure which way of error-reporting 
> is going to be preferred by our users.
> 
> Now this approach bloats C++ API a lot and makes it harder to
> maintain and optimize code. I propose like to abandon and deprecate
> non-throwing version of API and only leave throwing version,
> but first I want to hear from you guys - what do you think? Does
> anyone use non-throwing version of the API? Maybe your toolchain
> does not support exceptions or are you disabling them on purpose?
> 
> For those who prefer disabling exceptions I propose to introduce
> some macros like IGNITE_DISABLE_EXCEPTIONS and add
> some thread-local error-storing mechanism like ignite::GetLastError().
> 
> What do you guys think?
> 
> Best Regards,
> Igor
>

Re: 答复: ignite client memory issue

2017-01-23 Thread vkulichenko

Shawn,

It looks like you had too many asynchronous operations executed at the same
time and therefore too many futures created in memory. Async approach
requires more accuracy, and if everything works for you with sync
operations, I would use them.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/ignite-client-memory-issue-tp10174p10201.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: REST servive command LOG

2017-01-23 Thread Denis Magda

Saikat, 

Could you create a JIRA ticket providing more info why the modifications are 
required and paste a link to the pull-request there?

Otherwise the dev community might miss your changes.

—
Denis

> On Jan 22, 2017, at 10:16 AM, Saikat Maitra  wrote:
> 
> Hello Sam,
> 
> Due to security reason the log file need to be chrooted to IGNITE_HOME/log.
> 
> Please review this PR for the fix.
> 
> https://github.com/apache/ignite/pull/420 
> 
> 
> Regards,
> Saikat
> 
> 
> On Sun, Jan 22, 2017 at 12:03 PM, Saikat Maitra  > wrote:
> Hi Sam,
> 
> You can pass path as query param to get the log details. Please see this jira 
> ticket below for reference.
> 
> https://issues.apache.org/jira/browse/IGNITE-944 
> 
> 
> Regards
> Saikat
> 
> On Thu, Jan 19, 2017 at 2:21 PM, javastuff@gmail.com 
>   > wrote:
> Hi,
> 
> I find LOG command very useful. Periodic metrics and topology change
> information helps monitoring.
> 
> However it does not work with my application, It fails with below message -
> "Request parameter 'path' must contain a path to valid log file."
> 
> My application already using Log4j, so not using Ignite configuration XML to
> configure Ignite logging. Instead application log4j configured to push all
> Ignite INFO logs to a separate rolling file. Logging works fine. But
> 
> 1. Ignite configuration does not have logging configuration details.
> 2. Log files can be anywhere, not under IGNITE_HOME
> 
> And because of above 2 points LOG command fails.
> 
> How can I make it work?
> 
> Thanks,
> -Sam
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> --
> View this message in context: 
> http://apache-ignite-users.70518.x6.nabble.com/REST-servive-command-LOG-tp10148.html
>  
> 
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
> 
>

Re: Running Spark app in cluster!

2017-01-23 Thread vkulichenko

I would suggest to create IgniteConfiguration programmatically (use
IgniteContext constructor that accepts closure instead of XML file path).

However, it looks like there is a room to improve, I created a ticket:
https://issues.apache.org/jira/browse/IGNITE-4593

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Running-Spark-app-in-cluster-tp10073p10198.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Jetty configuration for adding new Handler

2017-01-23 Thread GB

Hi Andry/Saurabh,

I also have same issue as in original post.
I tried exact steps as mentioned by Andry but still facing same issue.
Header "Access-Control-Allow-Origin" is still not set in the response. Does
anything else needed in Jetty Configuration other than what Andry mentioned?

Thanks a lot.





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Jetty-configuration-for-adding-new-Handler-tp5841p10197.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Abandon of the non-throwing version of C++ client API

Ignite cluster will be jammed for long-running queries

Re: Timed out waiting for message delivery receipt

Re: Timed out waiting for message delivery receipt

Re: REST servive command LOG

答复: ignite client memory issue

RE: ignite client memory issue

Re: Timed out waiting for message delivery receipt

Re: REST servive command LOG

Re: Abandon of the non-throwing version of C++ client API

Re: 答复: ignite client memory issue

Re: REST servive command LOG

Re: Running Spark app in cluster!

Re: Jetty configuration for adding new Handler

14 matches

Site Navigation

Mail list logo

Footer information