Re: IgniteUtils#currentTimeMillis

[Evgeniy Stanilovskiy]

I assume that Vladimir mention this mesurements:
https://shipilev.net/blog/2014/nanotrusting-nanotime/
can we simple measure with JMH x86 and arm our realization vs system call?


As Dmitry P mentioned System.currentTimeMillis() is JVM intrinsic.
Moreover, there is a daemon thread that updates the internal value which
will not be needed after the change.

If we remove U.currentTimeMillis() code will become more clear and
consistent. Why we think that we can implement this particular timer  
better

than JVM developers?

Vladmir, can you please share a benchmark that will show the problems? If
it will then it will be a strong argument to keep the current
implementation.

--Yakov

Re: Performance vs correctness: I vote fore the second

[Evgeniy Stanilovskiy]

Guys, hope i can add one more example here.
Ones we use IgniteAtomicSequence, after topology changes some assertions  
can be catched due to default AtomicConfiguration

i.e.
public static final int DFLT_BACKUPS = 0;
public static final CacheMode DFLT_CACHE_MODE = PARTITIONED;

minimal improvements here would be to set DFLT_BACKUPS = 1; or change into  
REPLICATED mode.


thanks.


Folks,

I received a number of complaints from users that our default setting  
favor

performance at the cost of correctness and subtle behavior. Yesterday I
faced one such situation on my own.

I started REPLICATED cache on several nodes, put some data, executed  
simple
SQL and got wrong result. No errors, no warnings. The problem was caused  
by

default PRIMARY_SYNC mode. WTF, our cache doesn't work out of the box!

Another widely known examples are data streamer behavior, "read form
backups" + continuous queries.

I propose to change our defaults to favor *correctness* over performance,
and create good documentation and JavaDocs to explain users how to tune  
our

product. Proposed changes:

1) FULL_SYNC as default;
2) "readFromBackups=false" as default;
3) "IgniteDataStreamer.allowOverwrite=true" as default.

Users should not think how to make Ignite work correctly. It should be
correct out of the box.

Vladimir.

compute broadcast with PCL environment question

[Evgeniy Stanilovskiy]

hi, Igniters.
I have a question about compute broadcast behavior with peer class loading  
enabled.

Look, my test schema was like:

1-jvm PureServerApp - pure server GG node.
2-jvm FirstComputeRunner - first compute runner node.
3-jvm SecondComputeRunner - second compute runner node.

and my case was to check simultaneous execution same Callable classes but  
with different functional inside.
For example Callable from first runner - print 1-st jvm, from second -  
2-nd jvm.
All these node are running in separate jvm. All runner node call  
.broadcast 4 times in random manner.


Class clazz1 = classLoader1.loadClass("CallFunction");
...
ignite.compute().broadcast((IgniteCallable)clazz1.getDeclaredConstructor().newInstance());

I found:
PureServerApp output like :
2-nd jvm
2-nd jvm
2-nd jvm
2-nd jvm
1-st jvm
1-st jvm
2-nd jvm
1-st jvm
(5/3)

FirstComputeRunner :
2-nd jvm
1-st jvm
2-nd jvm
2-nd jvm
1-st jvm
1-st jvm
2-nd jvm
1-st jvm
(4/4)

and

SecondComputeRunner :
2-nd jvm
2-nd jvm
2-nd jvm
2-nd jvm
2-nd jvm
2-nd jvm
2-nd jvm
2-nd jvm
(8/0)

all grid settings are default, plz point me if this is correct behavior.

Thanks !

IGNITE-3605

[Evgeniy Stanilovskiy]

hello Igniters, i try to investigate bug in jmx metrics ticket:
https://issues.apache.org/jira/browse/IGNITE-3605, wrote some comments  
there,

can anyone hint me - what kind of changes are applicable here:
1. client node need to send "cacheMetrics" too, through  
TcpDiscoveryClientHeartbeatMessage.

or
2. server needs somehow triggers this case.

thanks !

ignite analysis with jepsen

[Evgeniy Stanilovskiy]

Hello Igniters,

There is fresh doc about mongodb analysis with jepsen tool:
https://jepsen.io/analyses/mongodb-3-4-0-rc3
Looks like mongo soon would be production ready database )
And what do you think about the same analysis for ignite?

I have no deal with jepsen but has a lot of positive feed backs.
Spend a little time for introduction into
https://github.com/jepsen-io/jepsen
found all testing config under clojure language witch may cause some  
initial troubles i think ..

Re: Inaccurate documentation about transactions

[Evgeniy Stanilovskiy]

postgres has the different viewpoint, i hope.

https://www.postgresql.org/docs/9.1/static/transaction-iso.html

Read Committed Isolation Level

Read Committed is the default isolation level in PostgreSQL. When a  
transaction uses this isolation level, a SELECT query (without a FOR  
UPDATE/SHARE clause) sees only data committed before the query began; it  
never sees either uncommitted data or changes committed during query  
execution by concurrent transactions. In effect, a SELECT query sees a  
snapshot of the database as of the instant the query begins to run.  
However, SELECT does see the effects of previous updates executed within  
its own transaction, even though they are not yet committed.



Alexandr,

If you PUT some value within transaction, this is absolutely normal that
you will see it on successive GET in the _same_ transaction. DIRTY_READ  
is

a different thing - it allows reads of not-yet-committed changes from
_another_ transaction.

On Wed, Feb 15, 2017 at 9:41 AM, Alexandr Kuramshin  


wrote:

After doing some tests with transactions I've found transactions work  
not

as expected after reading the documentation [1].

First of all, nowhere's written which methods of the cache are
transactional and which are not. Quite the contrary, after reading
documentation we get know that each TRANSACTIONAL cache is fully
ACID-compliant without exceptions.

Only after deep multi-thread testing, and consulting with other  
developers,

I get know that only get and put methods are running within transaction,
but iterator and query methods are running outside (in autonomous)
transaction with READ_COMMITTED isolation level.

Later I've understood that only methods throwing
TransactionTimeoutException/TransactionRollbackException/
TransactionHeuristicException
are fully transactional. I think all methods on page [2] should be  
directly

described - are they transactional or not. Btw, why these exceptions are
not derived from the common base class, e.g. TransactionException?

Secondary, using the transactional get() method inside the  
READ_COMMITTED
transaction we expect to get the committed value, as the documentation  
[1]

claims:

* READ_COMMITTED - Data is read without a lock and is never cached in  
the

transaction itself.

Ok, but what about put()? After doing the put() a new value, we get
successive reads of the new value, that is actually DIRTY READ. Hence  
the

value is cached within transaction. It's not documented behavior.

[1] https://apacheignite.readme.io/docs/transactions

[2]
https://ignite.apache.org/releases/1.8.0/javadoc/org/
apache/ignite/IgniteCache.html

--
Thanks,
Alexandr Kuramshin

Re: How to improve SQL testing

[Evgeniy Stanilovskiy]

Sergey, did you take a look into https://en.wikipedia.org/wiki/YCSB ?
https://github.com/joshwilliams/YCSB
there is simple jdbc connector looks like it would be ok for us ?


Hi,

That’s true that the utility/testing framework needs to be as flexible  
as possible. If the community can’t reuse an existing one let’s make it  
up.


As for VoltDB SQL Coverage Suite, if it’s Python based then can we  
implement those Python APIs directly calling Ignite.C++?


—
Denis


On Jan 30, 2017, at 4:39 AM, Sergey Kozlov  wrote:

Alexey, I suppose that it's a hard way to do that and at least see a few
disadvantages below:

1. The VoltDB has no multithreaded mode. It means huge execution time
consumption (say we talk about thousands statement multiplied on caches
count, it's the millions of queries).
2. We should add support of Ignite.
3. We should add support of H2 as base engine for us (MySQL/Postgress  
have

less priority)
4. They run it under Jenkins so we need to integrate it with TC somehow.

From my standpoint better to make the utility in Ignite than spending  
time
and efforts to adopt an existing tool. At least we have good (or even  
best)

Java experts/developers in community and the task doesn't look a complex
one.


On Mon, Jan 30, 2017 at 2:58 PM, Alexey Kuznetsov  


wrote:


Sergey,

Could we use http://www.jython.org/ to make it works as-is?

On Mon, Jan 30, 2017 at 6:06 PM, Sergey Kozlov 
wrote:


Hi

Hi reviewed the links below (thanks for Vova Ozerov to pointing me)  
and

found that the provided solution is really excellent!
Unfortunately we can't use it as is (from my standpoint) due lacks of
Python support in Ignite but we can re-use that idea for Ignite and  
make

SQL combinations coverage significantly better.

I filed the ticket IGNITE-4627 Tester for SQL functionality


Please share your thoughts and ideas.

[1] https://www.voltdb.com/blog/testing-voltdb-sqlcoverage
[2] https://github.com/VoltDB/voltdb/tree/master/tests/sqlcoverage

--
Sergey Kozlov
www.gridgain.com





--
Alexey Kuznetsov





--
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: scripting languages base cases using swig

[Evgeniy Stanilovskiy]

Hi Denis,

I don`t see any problem here, i have to speak with @vozerov about this  
issue and he recommends me to write it into dev list. All problems that i  
have with swig is :

1. jni overhead (no miracle here ...)
2. light troubles with collections API wrapping, like std::map, std::list  
and so on.


Partially my open source FreeLing java wrapper you can see here
https://github.com/TALP-UPC/FreeLing/tree/master/APIs/java

std_list.i - little stub :) i have talking about.


Hi Evgeniy,

Presently we’re trying to fill this gap offering SQL Grid [1]. In a  
nutshell, you can connect to an Ignite cluster from your favorite  
language or tool with ODBC/JDBC drivers and work with the cluster using  
SQL SELECT, INSERT, UPDATE, DELETE statements.


Here is how everything works for PHP that is not natively supported by  
Ignite:

https://dzone.com/articles/apache-ignite-enables-full-fledged-sql-support-for

However, as for SWIG. Do you think it’s feasible to implement on top of  
Ignite.C++ client which is tightly coupled with JVM?


[1] https://apacheignite.readme.io/docs/sql-grid  
<https://apacheignite.readme.io/docs/sql-grid>


—
Denis

On Jan 16, 2017, at 11:46 PM, Evgeniy Stanilovskiy  
<estanilovs...@gridgain.com> wrote:


Hi all.
Not so long ago i had to know that ignite had reduced functionality  
support in scripting languages.
So, idea was to take an existing C++ client and using SWIG  
(http://www.swig.org) as automatic wrapper, generate clients for  
absence scripting languages.

What do you think about this case ?

Thanks !

scripting languages base cases using swig

[Evgeniy Stanilovskiy]

Hi all.
Not so long ago i had to know that ignite had reduced functionality  
support in scripting languages.
So, idea was to take an existing C++ client and using SWIG  
(http://www.swig.org) as automatic wrapper, generate clients for absence  
scripting languages.

What do you think about this case ?

Thanks !