Re: Data Streamer

[vkulichenko]

Streamer already maintains buffers internally and sends data in batches, so
giving him a map will not change performance.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Data-Streamer-tp8409p8428.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data Streamer

[Manu]

Hi,

Your are creating new data streamer on each loop call... 
[...]
for (int i = 0; i < 100; i++){
//  
CacheManager.getInstallBaseCache().put(name+"-"+i, new
TestPojo());

CacheManager.getInstance().dataStreamer(CACHE).addData(name+"-"+i, new
TestPojo());
}
[...]

Ignite does this when you call dataStreamer(cache name) method...
[...]
  /**
 * @param cacheName Cache name ({@code null} for default cache).
 * @return Data loader.
 */
public DataStreamerImpl dataStreamer(@Nullable String cacheName) {
if (!busyLock.enterBusy())
throw new IllegalStateException("Failed to create data streamer
(grid is stopping).");

try {
final DataStreamerImpl ldr = new DataStreamerImpl<>(ctx,
cacheName, flushQ);

ldrs.add(ldr);
[...]

So try create a data streamer instance only once. 
[...]
*IgniteStream stream = CacheManager.getInstance().dataStreamer(CACHE);*

for (int i = 0; i < 100; i++){
stream.addData(name+"-"+i, new 
TestPojo());
}
[...]

Another improvement is send data on a "buffered fashion", so you reduce
calls to cluster... try stream.addData(data); // where data buffer =
Map



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Data-Streamer-tp8409p8427.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: SLF4J AND LOG4J delegation exception with ignite dependency

[chevy]

1. Now I am getting below exception. Saw in one of the threads that removing
ignite-rest-http will solve the issue which it does. But I need to include
rest-api as I will be using ignite rest services. Please help me fix this.

java.lang.NoSuchMethodError:
org.eclipse.jetty.util.log.StdErrLog.setProperties(Ljava/util/Properties;)V
at
org.apache.ignite.internal.processors.rest.protocols.http.jetty.GridJettyRestProtocol.(GridJettyRestProtocol.java:72)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
org.apache.ignite.internal.processors.rest.GridRestProcessor.startHttpProtocol(GridRestProcessor.java:831)
at
org.apache.ignite.internal.processors.rest.GridRestProcessor.start(GridRestProcessor.java:451)
at
org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1549)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:876)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1736)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1589)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1042)
at
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:964)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:850)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:749)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:619)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:589)
at org.apache.ignite.Ignition.start(Ignition.java:347)
at com.boot.NodeStartup.main(NodeStartup.java:21)

-
2. If I include ignite-indexing dependency (h2 is available in class path),
I get below exception -

java.lang.NoClassDefFoundError: org/h2/constant/SysProperties
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.start(IgniteH2Indexing.java:1487)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.start(GridQueryProcessor.java:171)
at
org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1549)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:869)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1736)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1589)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1042)
at
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:964)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:850)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:749)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:619)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:589)
at org.apache.ignite.Ignition.start(Ignition.java:347)
at com.boot.NodeStartup.main(NodeStartup.java:21)
Caused by: java.lang.ClassNotFoundException: org.h2.constant.SysProperties
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 14 more




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/SLF4J-AND-LOG4J-delegation-exception-with-ignite-dependency-tp8415p8425.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite query threads

[Anil]

Thanks Val. Computations and Queries using different thread pools. this
should be enough.



On 22 October 2016 at 00:09, vkulichenko 
wrote:

> Hi,
>
> Public thread is used for computations and system pool is used for all
> cache
> operations (gets, puts, queries, etc.). Having separate thread pool for
> queries will not give you performance improvement, because it's OS
> responsibility to schedule threads execution anyway.
>
> -Val
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Ignite-query-threads-tp8398p8413.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Data Streamer

[Anil]

HI Val,

I have attached sample reproducer. thanks.


On 21 October 2016 at 21:12, Vladislav Pyatkov  wrote:

> Hi Anil,
>
> This sounds is very questionable.
> Could you please attach your sources?
>
> On Fri, Oct 21, 2016 at 5:16 PM, Anil  wrote:
>
>> HI,
>>
>> I was loading data into ignite cache using parallel tasks by broadcasting
>> the task. Each taks (IgniteCallable implementation) has its own data
>> streamer. was it correct approach.
>>
>> Loading data into ignite cache using data streamer is very slow compared
>> normal cache.put.
>>
>> is that expected ? or need to some configurations to improve the
>> performance.
>>
>> Thanks.
>>
>>
>
>
> --
> Vladislav Pyatkov
>


CacheManager.java
Description: Binary data