[jira] [Created] (IGNITE-14370) BinaryMarshallerSelfTest not available for code re-use
Koala Lam created IGNITE-14370:
--
Summary: BinaryMarshallerSelfTest not available for code re-use
Key: IGNITE-14370
URL: https://issues.apache.org/jira/browse/IGNITE-14370
Project: Ignite
Issue Type: Task
Components: binary
Affects Versions: 2.9.1
Reporter: Koala Lam
We are building a set of filters which work with BinaryObject format of our
POJOs. We want to build test to ensure these filters are built correctly. The
idea is:
1) Use BinaryMarshaller to marshal our POJO into BinraryObject format
2) Run our filters against these BinaryObject
3) Verify the result
We found below 2 functions may be useful for our use case. Are they made
available for use? Please also advise any recommended way to test user-defined
filters against BinaryObject.
https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/binary/BinaryMarshallerSelfTest.java#L3873
{code:java}
private BinaryObjectImpl marshal(T obj, BinaryMarshaller marsh) throws
IgniteCheckedException {
{code}
https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/binary/BinaryMarshallerSelfTest.java#L3940
{code:java}
protected BinaryMarshaller binaryMarshaller(
BinaryNameMapper nameMapper,
BinaryIdMapper mapper,
BinarySerializer serializer,
Collection cfgs,
Collection excludedClasses
) throws IgniteCheckedException {
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
IGNITE-12951 Update documents for migrated extensions
Hi, I have raised a PR for the below jira issue. https://issues.apache.org/jira/browse/IGNITE-12951 - Update documents for migrated extensions PR : https://github.com/apache/ignite-website/pull/85 Please review and share feedback. Regards, Saikat
Re: IEP-70: Async Continuation Executor
Simple benchmark added - see JmhCacheAsyncListenBenchmark in the PR.
There is a 6-8% drop (1 client, 2 servers, 1 machine, int key/val).
I expect this difference to become barely observable on real-world
workloads.
On Thu, Mar 18, 2021 at 12:35 PM Pavel Tupitsyn
wrote:
> Denis,
>
> For a reproducer, please see CacheAsyncContinuationExecutorTest.java in
> the linked PoC [1]
>
> [1]
> https://github.com/apache/ignite/pull/8870/files#diff-c788c12013622093df07d8f628a6e8c5fb0c15007f0787f2d459dbb3e377fc5aR54
>
> On Thu, Mar 18, 2021 at 1:58 AM Raymond Wilson
> wrote:
>
>> We implemented the ContinueWith() suggestion from Pavel, and it works well
>> so far in testing, though we do not have data to support if there is or is
>> not a performance penalty in our use case..
>>
>> To lend another vote to the 'Don't do continuations on the striped thread
>> pool' line of thinking: Deadlocking is an issue as is starvation. In some
>> ways starvation is more insidious because by the time things stop working
>> the cause and effect distance may be large.
>>
>> I appreciate the documentation does make statements about not performing
>> cache operations in a continuation due to deadlock possibilities, but that
>> statement does not reveal why this is an issue. It is less a case of a
>> async cache operation being followed by some other cache operation (an
>> immediate issue), and more a general case of the continuation of
>> application logic using a striped pool thread in a way that might mean
>> that
>> thread is never given back - it's now just a piece of the application
>> infrastructure until some other application activity schedules away from
>> that thread (eg: by ContinueWith or some other async operation in the
>> application code that releases the thread). To be fair, beyond structures
>> like ContinueWith(), it is not obvious how that continuation thread should
>> be handed back. This will be the same behaviour for dedicated continuation
>> pools, but with far less risk in the absence of ContinueWith() constructs.
>>
>> In the .Net world this is becoming more of an issue as fewer .Net use
>> cases
>> outside of UI bother with synchronization contexts by default.
>>
>> Raymond.
>>
>>
>> On Thu, Mar 18, 2021 at 9:56 AM Valentin Kulichenko <
>> valentin.kuliche...@gmail.com> wrote:
>>
>> > Hi Denis,
>> >
>> > I think Pavel's main point is that behavior is unpredictable. For
>> example,
>> > AFAIK, putAsync can be executed in the main thread instead of the
>> striped
>> > pool thread if the operation is local. The listener can also be
>> executed in
>> > the main thread - this happens if the future is completed prior to
>> listener
>> > invocation (this is actually quite possible in the unit test environment
>> > causing the test to pass). Finally, I'm pretty sure there are many cases
>> > when a deadlock does not occur right away, but instead it will reveal
>> > itself under high load due to thread starvation. The latter type of
>> issues
>> > is the most dangerous because they are often reproduced only in
>> production.
>> > Finally, there are performance considerations as well - cache operations
>> > and listeners share the same fixed-sized pool which can have negative
>> > effects.
>> >
>> > I'm OK with the change. Although, it might be better to introduce a new
>> > fixed-sized pool instead of ForkJoinPool for listeners, simply because
>> this
>> > is the approach taken throughout the project. But this is up to a
>> debate.
>> >
>> > -Val
>> >
>> > On Wed, Mar 17, 2021 at 11:31 AM Denis Garus
>> wrote:
>> >
>> > > Pavel,
>> > > I tried this:
>> > >
>> > > @Test
>> > > public void test() throws Exception {
>> > > IgniteCache cache =
>> > > startGrid().getOrCreateCache("test_cache");
>> > >
>> > > cache.putAsync(1, "one").listen(f -> cache.replace(1, "two"));
>> > >
>> > > assertEquals("two", cache.get(1));
>> > > }
>> > >
>> > > and this test is green.
>> > > I believe that an user can make listener that leads to deadlock, but
>> > > the example in the IEP does not reflect this.
>> > >
>> > >
>> > >
>> > > ср, 17 мар. 2021 г. в 17:36, Вячеслав Коптилин <
>> slava.kopti...@gmail.com
>> > >:
>> > >
>> > > > Hi Pavel,
>> > > >
>> > > > > Not a good excuse really. We have a usability problem, you have to
>> > > admit
>> > > > it.
>> > > > Fair enough. I agree that this is a usability issue, but I have
>> doubts
>> > > that
>> > > > the proposed approach to overcome it is the best one.
>> > > >
>> > > > > Documentation won't help - no one is going to read the Javadoc
>> for a
>> > > > trivial method like putAsync
>> > > > That is sad... However, I don't think that this is a strong argument
>> > > here.
>> > > >
>> > > > This is just my opinion. Let's see what other community members
>> have to
>> > > > say.
>> > > >
>> > > > Thanks,
>> > > > S.
>> > > >
>> > > >
>> > > > ср, 17 мар. 2021 г. в 17:01, Pavel Tupitsyn :
>> > > >
>> > > > > > the user should use the right API
>> > > > >
>> > > > > Not a
[jira] [Created] (IGNITE-14369) Node.js: incorrect Hash Code calculcation
Bojidar Marinov created IGNITE-14369: Summary: Node.js: incorrect Hash Code calculcation Key: IGNITE-14369 URL: https://issues.apache.org/jira/browse/IGNITE-14369 Project: Ignite Issue Type: Bug Components: thin client Affects Versions: 2.10 Reporter: Bojidar Marinov The Node.js thin client calculates wrong hashcodes, possibly leading to duplicated rows and inability to read rows with complex key types written from other languages: # BinaryUtils.contentHashCode is called with wrong end parameter at [BinaryObject.ts:397|https://github.com/apache/ignite-nodejs-thin-client/blob/76c7d7eb2b1856295f877434ef358beaa7155d91/src/BinaryObject.ts#L397]. The second parameter is the end position, not the content length, and thus should be relative to this._startOffset. Experimentally confirmed that changing to this._startOffset + this._schemaOffset - 1 works. # BinaryUtils.contentHashCode uses unsigned bytes at [BinaryUtils.ts:632|https://github.com/apache/ignite-nodejs-thin-client/blob/76c7d7eb2b1856295f877434ef358beaa7155d91/src/internal/BinaryUtils.ts#L632]. [`buffer[idx]`|https://nodejs.org/api/buffer.html#buffer_buf_index] returns a number between 0..255, while Java's byte is -128..127. Experimentally confirmed that switching to [Buffer.readInt8|https://nodejs.org/api/buffer.html#buffer_buf_readint8_offset] works. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14368) System view for DataStructures
Nikolay Izhikov created IGNITE-14368: Summary: System view for DataStructures Key: IGNITE-14368 URL: https://issues.apache.org/jira/browse/IGNITE-14368 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Currently, there is no way for the user to know what DataStructures exist in Ignite. We should provide a system view for it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14367) WAL reader. Read only specific Record Types
Nikolay Izhikov created IGNITE-14367: Summary: WAL reader. Read only specific Record Types Key: IGNITE-14367 URL: https://issues.apache.org/jira/browse/IGNITE-14367 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov CDC application uses only specific WALRecord types. So we should provide an ability to skip WALRecord without full deserializing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14366) Support separate JVM options for ignite-cdc.sh and ignite.sh
Nikolay Izhikov created IGNITE-14366:
Summary: Support separate JVM options for ignite-cdc.sh and
ignite.sh
Key: IGNITE-14366
URL: https://issues.apache.org/jira/browse/IGNITE-14366
Project: Ignite
Issue Type: Improvement
Reporter: Nikolay Izhikov
Currently, ignite-cdc.sh just reuse ignite.sh with it's own main class name
({{org.apache.ignite.cdc.CommandLineStartup}})
Should fix it and provide a way to separate configuration ignite and cdc
applications.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14365) CDC Documentation
Nikolay Izhikov created IGNITE-14365: Summary: CDC Documentation Key: IGNITE-14365 URL: https://issues.apache.org/jira/browse/IGNITE-14365 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
