[jira] [Updated] (IGNITE-13213) Improve Queue performance
[
https://issues.apache.org/jira/browse/IGNITE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manuel Núñez updated IGNITE-13213:
--
Description:
Queue performance can be improved by making some changes, related to Queue Item
Key(1) , offer method (2) and collection configuration (3), and related issues
IGNITE-13203 and IGNITE-13221.
With those changes queue structure is 10%-30% faster than first Ignite's
competitor:
!image-2020-07-04-15-35-43-879.png!
Queue performance benchmark (lower value is better) with different queue
message types, multiple clients (M) and 2 server nodes (IMDG):
* string messages: 1KB, 10KB, 100KB
* byte[] messages: 1KB, 10KB, 100KB
* complex objects with cycle references (cpx) messages
Legend:
* {color:#4c9aff}Blue{color}: first Ignite's competitor - on heap only, open
source version does not support off-heap neither OOTB persistence, we can see
GC pauses
* {color:#ffab00}Orange{color}: Ignite off-heap with native persistence enabled
* {color:#de350b}Red{color}: Ignite off-heap only
1. Queue Item Key -> currently, queue hash code is computed for every single
item key using full queue name (> 20 chars) on Queue Item Key constructor.
Proposal compute hash on GridCacheQueueAdapter constructor and pass it to
itemKey method
{code:java}
private static QueueItemKey itemKey(IgniteUuid id,
int queueNameHash,
boolean collocated,
long idx) {
return collocated ?
new CollocatedQueueItemKey(id, queueNameHash, idx) : new
GridCacheQueueItemKey(id, queueNameHash, idx);
}
{code}
1.1. CollocatedQueueItemKey (fully compatible)
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.internal.processors.datastructures;
import org.apache.ignite.cache.affinity.AffinityKeyMapped;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.lang.IgniteUuid;
public class CollocatedQueueItemKey implements QueueItemKey {
/** */
private IgniteUuid queueId;
/** */
@AffinityKeyMapped
private int queueNameHash;
/** */
private long idx;
/**
* @param queueId Queue unique ID.
* @param queueName Queue name.
* @param idx Item index.
*/
public CollocatedQueueItemKey(IgniteUuid queueId, int queueNameHash, long
idx) {
this.queueId = queueId;
this.queueNameHash = queueNameHash;
this.idx = idx;
}
/** {@inheritDoc} */
@Override public boolean equals(Object o) {
if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
CollocatedQueueItemKey itemKey = (CollocatedQueueItemKey)o;
return idx == itemKey.idx && queueId.equals(itemKey.queueId);
}
/** {@inheritDoc} */
@Override public int hashCode() {
int res = queueId.hashCode();
res = 31 * res + (int)(idx ^ (idx >>> 32));
return res;
}
/** {@inheritDoc} */
@Override public String toString() {
return S.toString(CollocatedQueueItemKey.class, this);
}
}
{code}
1.2. GridCacheQueueItemKey (not compatible with existing queues, need to be
analysed), remove queueName field as is not required at all (use queueId
instead), this will reduce network traffic and improve serialisation
performance.
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
[jira] [Updated] (IGNITE-13213) Improve Queue performance
[
https://issues.apache.org/jira/browse/IGNITE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manuel Núñez updated IGNITE-13213:
--
Description:
Queue performance can be improved by making some changes, related to Queue Item
Key(1) , offer method (2) and collection configuration (3), and related issues
IGNITE-13203 and IGNITE-13221.
With those changes queue structure is 10%-30% faster than first Ignite's
competitor:
!image-2020-07-04-15-35-43-879.png!
Queue performance benchmark with different queue message types, multiple
clients (M) and 2 server nodes (IMDG):
* string messages: 1KB, 10KB, 100KB
* byte[] messages: 1KB, 10KB, 100KB
* complex objects with cycle references (cpx) messages
Legend:
* {color:#4c9aff}Blue{color}: first Ignite's competitor - on heap only, open
source version does not support off-heap neither OOTB persistence, we can see
GC pauses
* {color:#ffab00}Orange{color}: Ignite off-heap with native persistence enabled
* {color:#de350b}Red{color}: Ignite off-heap only
1. Queue Item Key -> currently, queue hash code is computed for every single
item key using full queue name (> 20 chars) on Queue Item Key constructor.
Proposal compute hash on GridCacheQueueAdapter constructor and pass it to
itemKey method
{code:java}
private static QueueItemKey itemKey(IgniteUuid id,
int queueNameHash,
boolean collocated,
long idx) {
return collocated ?
new CollocatedQueueItemKey(id, queueNameHash, idx) : new
GridCacheQueueItemKey(id, queueNameHash, idx);
}
{code}
1.1. CollocatedQueueItemKey (fully compatible)
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.internal.processors.datastructures;
import org.apache.ignite.cache.affinity.AffinityKeyMapped;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.lang.IgniteUuid;
public class CollocatedQueueItemKey implements QueueItemKey {
/** */
private IgniteUuid queueId;
/** */
@AffinityKeyMapped
private int queueNameHash;
/** */
private long idx;
/**
* @param queueId Queue unique ID.
* @param queueName Queue name.
* @param idx Item index.
*/
public CollocatedQueueItemKey(IgniteUuid queueId, int queueNameHash, long
idx) {
this.queueId = queueId;
this.queueNameHash = queueNameHash;
this.idx = idx;
}
/** {@inheritDoc} */
@Override public boolean equals(Object o) {
if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
CollocatedQueueItemKey itemKey = (CollocatedQueueItemKey)o;
return idx == itemKey.idx && queueId.equals(itemKey.queueId);
}
/** {@inheritDoc} */
@Override public int hashCode() {
int res = queueId.hashCode();
res = 31 * res + (int)(idx ^ (idx >>> 32));
return res;
}
/** {@inheritDoc} */
@Override public String toString() {
return S.toString(CollocatedQueueItemKey.class, this);
}
}
{code}
1.2. GridCacheQueueItemKey (not compatible with existing queues, need to be
analysed), remove queueName field as is not required at all (use queueId
instead), this will reduce network traffic and improve serialisation
performance.
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
[jira] [Updated] (IGNITE-13213) Improve Queue performance
[
https://issues.apache.org/jira/browse/IGNITE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manuel Núñez updated IGNITE-13213:
--
Fix Version/s: (was: 2.9)
> Improve Queue performance
> -
>
> Key: IGNITE-13213
> URL: https://issues.apache.org/jira/browse/IGNITE-13213
> Project: Ignite
> Issue Type: Improvement
> Components: data structures
>Affects Versions: 2.4, 2.5, 2.6, 2.7, 2.8, 2.7.5, 2.7.6, 2.8.1
>Reporter: Manuel Núñez
>Priority: Major
> Attachments: image-2020-07-04-15-35-43-879.png
>
>
> Queue performance can be improved by making some changes, related to Queue
> Item Key(1) , offer method (2) and collection configuration (3) and related
> IGNITE-13203.
> With those changes queue structure is 10%-30% faster than first Ignite's
> competitor:
> !image-2020-07-04-15-35-43-879.png!
> Queue performance benchmark with different queue message types, multiple
> clients (M) and 2 server nodes (IMDG):
> * string messages: 1KB, 10KB, 100KB
> * byte[] messages: 1KB, 10KB, 100KB
> * complex objects with cycle references (cpx) messages
> Legend:
> * {color:#4c9aff}Blue{color}: first Ignite's competitor - on heap only, open
> source version does not support off-heap neither OOTB persistence, we can see
> GC pauses
> * {color:#ffab00}Orange{color}: Ignite off-heap with native persistence
> enabled
> * {color:#de350b}Red{color}: Ignite off-heap only
> 1. Queue Item Key -> currently, queue hash code is computed for every single
> item key using full queue name (> 20 chars) on Queue Item Key constructor.
> Proposal compute hash on GridCacheQueueAdapter constructor and pass it to
> itemKey method
> {code:java}
> private static QueueItemKey itemKey(IgniteUuid id,
> int queueNameHash,
> boolean collocated,
> long idx) {
> return collocated ?
> new CollocatedQueueItemKey(id, queueNameHash, idx) : new
> GridCacheQueueItemKey(id, queueNameHash, idx);
> }
> {code}
> 1.1. CollocatedQueueItemKey (fully compatible)
> {code:java}
> /*
> * Licensed to the Apache Software Foundation (ASF) under one or more
> * contributor license agreements. See the NOTICE file distributed with
> * this work for additional information regarding copyright ownership.
> * The ASF licenses this file to You under the Apache License, Version 2.0
> * (the "License"); you may not use this file except in compliance with
> * the License. You may obtain a copy of the License at
> *
> * http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
> package org.apache.ignite.internal.processors.datastructures;
> import org.apache.ignite.cache.affinity.AffinityKeyMapped;
> import org.apache.ignite.internal.util.typedef.internal.S;
> import org.apache.ignite.lang.IgniteUuid;
> public class CollocatedQueueItemKey implements QueueItemKey {
> /** */
> private IgniteUuid queueId;
> /** */
> @AffinityKeyMapped
> private int queueNameHash;
> /** */
> private long idx;
> /**
> * @param queueId Queue unique ID.
> * @param queueName Queue name.
> * @param idx Item index.
> */
> public CollocatedQueueItemKey(IgniteUuid queueId, int queueNameHash, long
> idx) {
> this.queueId = queueId;
> this.queueNameHash = queueNameHash;
> this.idx = idx;
> }
> /** {@inheritDoc} */
> @Override public boolean equals(Object o) {
> if (this == o)
> return true;
> if (o == null || getClass() != o.getClass())
> return false;
> CollocatedQueueItemKey itemKey = (CollocatedQueueItemKey)o;
> return idx == itemKey.idx && queueId.equals(itemKey.queueId);
> }
> /** {@inheritDoc} */
> @Override public int hashCode() {
> int res = queueId.hashCode();
> res = 31 * res + (int)(idx ^ (idx >>> 32));
> return res;
> }
> /** {@inheritDoc} */
> @Override public String toString() {
> return S.toString(CollocatedQueueItemKey.class, this);
> }
> }
> {code}
> 1.2. GridCacheQueueItemKey (not compatible with existing queues, need to be
> analysed), remove queueName field as is not required at all (use queueId
> instead), this will reduce network traffic and improve serialisation
> performance.
> {code:java}
> /*
> * Licensed to the Apache Software Foundation (ASF) under one or more
> * contributor license agreements. See the NOTICE file distributed with
> * this work for additional information
[jira] [Updated] (IGNITE-13213) Improve Queue performance
[
https://issues.apache.org/jira/browse/IGNITE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manuel Núñez updated IGNITE-13213:
--
Description:
Queue performance can be improved by making some changes, related to Queue Item
Key(1) , offer method (2) and collection configuration (3) and related
IGNITE-13203.
With those changes queue structure is 10%-30% faster than first Ignite's
competitor:
!image-2020-07-04-15-35-43-879.png!
Queue performance benchmark with different queue message types, multiple
clients (M) and 2 server nodes (IMDG):
* string messages: 1KB, 10KB, 100KB
* byte[] messages: 1KB, 10KB, 100KB
* complex objects with cycle references (cpx) messages
Legend:
* {color:#4c9aff}Blue{color}: first Ignite's competitor - on heap only, open
source version does not support off-heap neither OOTB persistence, we can see
GC pauses
* {color:#ffab00}Orange{color}: Ignite off-heap with native persistence enabled
* {color:#de350b}Red{color}: Ignite off-heap only
1. Queue Item Key -> currently, queue hash code is computed for every single
item key using full queue name (> 20 chars) on Queue Item Key constructor.
Proposal compute hash on GridCacheQueueAdapter constructor and pass it to
itemKey method
{code:java}
private static QueueItemKey itemKey(IgniteUuid id,
int queueNameHash,
boolean collocated,
long idx) {
return collocated ?
new CollocatedQueueItemKey(id, queueNameHash, idx) : new
GridCacheQueueItemKey(id, queueNameHash, idx);
}
{code}
1.1. CollocatedQueueItemKey (fully compatible)
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.internal.processors.datastructures;
import org.apache.ignite.cache.affinity.AffinityKeyMapped;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.lang.IgniteUuid;
public class CollocatedQueueItemKey implements QueueItemKey {
/** */
private IgniteUuid queueId;
/** */
@AffinityKeyMapped
private int queueNameHash;
/** */
private long idx;
/**
* @param queueId Queue unique ID.
* @param queueName Queue name.
* @param idx Item index.
*/
public CollocatedQueueItemKey(IgniteUuid queueId, int queueNameHash, long
idx) {
this.queueId = queueId;
this.queueNameHash = queueNameHash;
this.idx = idx;
}
/** {@inheritDoc} */
@Override public boolean equals(Object o) {
if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
CollocatedQueueItemKey itemKey = (CollocatedQueueItemKey)o;
return idx == itemKey.idx && queueId.equals(itemKey.queueId);
}
/** {@inheritDoc} */
@Override public int hashCode() {
int res = queueId.hashCode();
res = 31 * res + (int)(idx ^ (idx >>> 32));
return res;
}
/** {@inheritDoc} */
@Override public String toString() {
return S.toString(CollocatedQueueItemKey.class, this);
}
}
{code}
1.2. GridCacheQueueItemKey (not compatible with existing queues, need to be
analysed), remove queueName field as is not required at all (use queueId
instead), this will reduce network traffic and improve serialisation
performance.
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
[jira] [Updated] (IGNITE-13213) Improve Queue performance
[
https://issues.apache.org/jira/browse/IGNITE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manuel Núñez updated IGNITE-13213:
--
Description:
Queue performance can be improved by making some changes, related to Queue Item
Key(1) , offer method (2) and collection configuration (3) and related
IGNITE-13203.
With those changes queue structure is 10%-30% faster than first Ignite's
competitor:
!image-2020-07-04-15-35-43-879.png!
Queue performance benchmark with:
- different queue messages and sizes:
* strings: 1k, 10k, 100k
- byte[]: 1k, 10k, 100k
- complex objects with cycle references (cpx)
- Multiple clients (M) and 2 server nodes (IMDG)
Legend:
{color:#4C9AFF}Blue{color}: first Ignite's competitor (pure on memory)
{color:#FFAB00}Orange{color}: Ignite with native persistence enabled
{color:#DE350B}Red{color}: Ignite (pure on memory)
1. Queue Item Key -> currently, queue hash code is computed for every single
item key using full queue name (> 20 chars) on Queue Item Key constructor.
Proposal compute hash on GridCacheQueueAdapter constructor and pass it to
itemKey method
{code:java}
private static QueueItemKey itemKey(IgniteUuid id,
int queueNameHash,
boolean collocated,
long idx) {
return collocated ?
new CollocatedQueueItemKey(id, queueNameHash, idx) : new
GridCacheQueueItemKey(id, queueNameHash, idx);
}
{code}
1.1. CollocatedQueueItemKey (fully compatible)
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.internal.processors.datastructures;
import org.apache.ignite.cache.affinity.AffinityKeyMapped;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.lang.IgniteUuid;
public class CollocatedQueueItemKey implements QueueItemKey {
/** */
private IgniteUuid queueId;
/** */
@AffinityKeyMapped
private int queueNameHash;
/** */
private long idx;
/**
* @param queueId Queue unique ID.
* @param queueName Queue name.
* @param idx Item index.
*/
public CollocatedQueueItemKey(IgniteUuid queueId, int queueNameHash, long
idx) {
this.queueId = queueId;
this.queueNameHash = queueNameHash;
this.idx = idx;
}
/** {@inheritDoc} */
@Override public boolean equals(Object o) {
if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
CollocatedQueueItemKey itemKey = (CollocatedQueueItemKey)o;
return idx == itemKey.idx && queueId.equals(itemKey.queueId);
}
/** {@inheritDoc} */
@Override public int hashCode() {
int res = queueId.hashCode();
res = 31 * res + (int)(idx ^ (idx >>> 32));
return res;
}
/** {@inheritDoc} */
@Override public String toString() {
return S.toString(CollocatedQueueItemKey.class, this);
}
}
{code}
1.2. GridCacheQueueItemKey (not compatible with existing queues, need to be
analysed), remove queueName field as is not required at all (use queueId
instead), this will reduce network traffic and improve serialisation
performance.
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.internal.processors.datastructures;
import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
[jira] [Updated] (IGNITE-13213) Improve Queue performance
[
https://issues.apache.org/jira/browse/IGNITE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manuel Núñez updated IGNITE-13213:
--
Description:
Queue performance can be improved by making some changes, related to Queue Item
Key(1) , offer method (2) and collection configuration (3) and related
IGNITE-13203.
With those changes queue structure is 10%-30% faster than first Ignite's
competitor:
!image-2020-07-04-15-35-43-879.png!
Queue performance benchmark with:
- different queue messages and sizes:
- strings: 1k, 10k, 100k
- byte[]: 1k, 10k, 100k
- complex objects with cycle references (cpx)
- Multiple clients (M) and 2 server nodes (IMDG)
Legend:
{color:#4C9AFF}Blue{color}: first Ignite's competitor (pure on memory)
{color:#FFAB00}Orange{color}: Ignite with native persistence enabled
{color:#DE350B}Red{color}: Ignite (pure on memory)
1. Queue Item Key -> currently, queue hash code is computed for every single
item key using full queue name (> 20 chars) on Queue Item Key constructor.
Proposal compute hash on GridCacheQueueAdapter constructor and pass it to
itemKey method
{code:java}
private static QueueItemKey itemKey(IgniteUuid id,
int queueNameHash,
boolean collocated,
long idx) {
return collocated ?
new CollocatedQueueItemKey(id, queueNameHash, idx) : new
GridCacheQueueItemKey(id, queueNameHash, idx);
}
{code}
1.1. CollocatedQueueItemKey (fully compatible)
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.internal.processors.datastructures;
import org.apache.ignite.cache.affinity.AffinityKeyMapped;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.lang.IgniteUuid;
public class CollocatedQueueItemKey implements QueueItemKey {
/** */
private IgniteUuid queueId;
/** */
@AffinityKeyMapped
private int queueNameHash;
/** */
private long idx;
/**
* @param queueId Queue unique ID.
* @param queueName Queue name.
* @param idx Item index.
*/
public CollocatedQueueItemKey(IgniteUuid queueId, int queueNameHash, long
idx) {
this.queueId = queueId;
this.queueNameHash = queueNameHash;
this.idx = idx;
}
/** {@inheritDoc} */
@Override public boolean equals(Object o) {
if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
CollocatedQueueItemKey itemKey = (CollocatedQueueItemKey)o;
return idx == itemKey.idx && queueId.equals(itemKey.queueId);
}
/** {@inheritDoc} */
@Override public int hashCode() {
int res = queueId.hashCode();
res = 31 * res + (int)(idx ^ (idx >>> 32));
return res;
}
/** {@inheritDoc} */
@Override public String toString() {
return S.toString(CollocatedQueueItemKey.class, this);
}
}
{code}
1.2. GridCacheQueueItemKey (not compatible with existing queues, need to be
analysed), remove queueName field as is not required at all (use queueId
instead), this will reduce network traffic and improve serialisation
performance.
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.internal.processors.datastructures;
import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import
[jira] [Updated] (IGNITE-13213) Improve Queue performance
[
https://issues.apache.org/jira/browse/IGNITE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manuel Núñez updated IGNITE-13213:
--
Description:
Queue performance can be improved by making some changes, related to Queue Item
Key(1) , offer method (2) and collection configuration (3) and related
IGNITE-13203.
With those changes queue structure is 10%-30% faster than first Ignite's
competidor (benchmark with different queue messages and sizes, with multiple
clients (M) and 2 server nodes (IMDG) : strings(1k, 10k, 100k); byte[] (1k,
10k, 100k) and complex (cpx) objects with cycle references)
!image-2020-07-04-15-35-43-879.png!
{color:#4C9AFF}Blue{color}: first Ignite's competidor (pure on memory)
{color:#FFAB00}Orange{color}: Ignite with native persistence enabled
{color:#DE350B}Red{color}: Ignite (pure on memory)
1. Queue Item Key -> currently, queue hash code is computed for every single
item key using full queue name (> 20 chars) on Queue Item Key constructor.
Proposal compute hash on GridCacheQueueAdapter constructor and pass it to
itemKey method
{code:java}
private static QueueItemKey itemKey(IgniteUuid id,
int queueNameHash,
boolean collocated,
long idx) {
return collocated ?
new CollocatedQueueItemKey(id, queueNameHash, idx) : new
GridCacheQueueItemKey(id, queueNameHash, idx);
}
{code}
1.1. CollocatedQueueItemKey (fully compatible)
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.internal.processors.datastructures;
import org.apache.ignite.cache.affinity.AffinityKeyMapped;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.lang.IgniteUuid;
public class CollocatedQueueItemKey implements QueueItemKey {
/** */
private IgniteUuid queueId;
/** */
@AffinityKeyMapped
private int queueNameHash;
/** */
private long idx;
/**
* @param queueId Queue unique ID.
* @param queueName Queue name.
* @param idx Item index.
*/
public CollocatedQueueItemKey(IgniteUuid queueId, int queueNameHash, long
idx) {
this.queueId = queueId;
this.queueNameHash = queueNameHash;
this.idx = idx;
}
/** {@inheritDoc} */
@Override public boolean equals(Object o) {
if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
CollocatedQueueItemKey itemKey = (CollocatedQueueItemKey)o;
return idx == itemKey.idx && queueId.equals(itemKey.queueId);
}
/** {@inheritDoc} */
@Override public int hashCode() {
int res = queueId.hashCode();
res = 31 * res + (int)(idx ^ (idx >>> 32));
return res;
}
/** {@inheritDoc} */
@Override public String toString() {
return S.toString(CollocatedQueueItemKey.class, this);
}
}
{code}
1.2. GridCacheQueueItemKey (not compatible with existing queues, need to be
analysed), remove queueName field as is not required at all (use queueId
instead), this will reduce network traffic and improve serialisation
performance.
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.internal.processors.datastructures;
import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
import
[jira] [Updated] (IGNITE-13213) Improve Queue performance
[
https://issues.apache.org/jira/browse/IGNITE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manuel Núñez updated IGNITE-13213:
--
Description:
Queue performance can be improved by making some changes, related to Queue Item
Key(1) , offer method (2) and collection configuration (3) and related
IGNITE-13203.
With those changes queue structure is 10%-30% faster than first Ignite's
competidor.
!image-2020-07-04-15-35-43-879.png!
{color:#4C9AFF}Blue{color}: first Ignite's competidor (pure on memory)
{color:#FFAB00}Orange{color}: Ignite with native persistence enabled
{color:#DE350B}Red{color}: Ignite (pure on memory)
1. Queue Item Key -> currently, queue hash code is computed for every single
item key using full queue name (> 20 chars) on Queue Item Key constructor.
Proposal compute hash on GridCacheQueueAdapter constructor and pass it to
itemKey method
{code:java}
private static QueueItemKey itemKey(IgniteUuid id,
int queueNameHash,
boolean collocated,
long idx) {
return collocated ?
new CollocatedQueueItemKey(id, queueNameHash, idx) : new
GridCacheQueueItemKey(id, queueNameHash, idx);
}
{code}
1.1. CollocatedQueueItemKey (fully compatible)
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.internal.processors.datastructures;
import org.apache.ignite.cache.affinity.AffinityKeyMapped;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.lang.IgniteUuid;
public class CollocatedQueueItemKey implements QueueItemKey {
/** */
private IgniteUuid queueId;
/** */
@AffinityKeyMapped
private int queueNameHash;
/** */
private long idx;
/**
* @param queueId Queue unique ID.
* @param queueName Queue name.
* @param idx Item index.
*/
public CollocatedQueueItemKey(IgniteUuid queueId, int queueNameHash, long
idx) {
this.queueId = queueId;
this.queueNameHash = queueNameHash;
this.idx = idx;
}
/** {@inheritDoc} */
@Override public boolean equals(Object o) {
if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
CollocatedQueueItemKey itemKey = (CollocatedQueueItemKey)o;
return idx == itemKey.idx && queueId.equals(itemKey.queueId);
}
/** {@inheritDoc} */
@Override public int hashCode() {
int res = queueId.hashCode();
res = 31 * res + (int)(idx ^ (idx >>> 32));
return res;
}
/** {@inheritDoc} */
@Override public String toString() {
return S.toString(CollocatedQueueItemKey.class, this);
}
}
{code}
1.2. GridCacheQueueItemKey (not compatible with existing queues, need to be
analysed), remove queueName field as is not required at all (use queueId
instead), this will reduce network traffic and improve serialisation
performance.
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.internal.processors.datastructures;
import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.internal.util.typedef.internal.U;
import org.apache.ignite.lang.IgniteUuid;
/**
* Queue item key.
*/
class GridCacheQueueItemKey implements
[jira] [Updated] (IGNITE-13213) Improve Queue performance
[
https://issues.apache.org/jira/browse/IGNITE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manuel Núñez updated IGNITE-13213:
--
Description:
Queue performance can be improved by make some changes, related to Queue Item
Key(1) , offer method (2) and collection configuration (3). With these changes
queue structure is 10%-30% faster than first Ignite competidor.
!image-2020-07-04-15-35-43-879.png!
{color:#4C9AFF}Blue{color}: first Ignite competidor(pure on memory)
{color:#FFAB00}Orange{color}: Ignite with native persistence enabled
{color:#DE350B}Red{color}: Ignite (pure on memory)
1. Queue Item Key -> currently, queue hash code is computed for every single
item key using full queue name (> 20 chars) on Queue Item Key constructor.
Proposal compute hash on GridCacheQueueAdapter constructor and pass it to
itemKey method
{code:java}
private static QueueItemKey itemKey(IgniteUuid id,
int queueNameHash,
boolean collocated,
long idx) {
return collocated ?
new CollocatedQueueItemKey(id, queueNameHash, idx) : new
GridCacheQueueItemKey(id, queueNameHash, idx);
}
{code}
1.1. CollocatedQueueItemKey (fully compatible)
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.internal.processors.datastructures;
import org.apache.ignite.cache.affinity.AffinityKeyMapped;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.lang.IgniteUuid;
public class CollocatedQueueItemKey implements QueueItemKey {
/** */
private IgniteUuid queueId;
/** */
@AffinityKeyMapped
private int queueNameHash;
/** */
private long idx;
/**
* @param queueId Queue unique ID.
* @param queueName Queue name.
* @param idx Item index.
*/
public CollocatedQueueItemKey(IgniteUuid queueId, int queueNameHash, long
idx) {
this.queueId = queueId;
this.queueNameHash = queueNameHash;
this.idx = idx;
}
/** {@inheritDoc} */
@Override public boolean equals(Object o) {
if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
CollocatedQueueItemKey itemKey = (CollocatedQueueItemKey)o;
return idx == itemKey.idx && queueId.equals(itemKey.queueId);
}
/** {@inheritDoc} */
@Override public int hashCode() {
int res = queueId.hashCode();
res = 31 * res + (int)(idx ^ (idx >>> 32));
return res;
}
/** {@inheritDoc} */
@Override public String toString() {
return S.toString(CollocatedQueueItemKey.class, this);
}
}
{code}
1.2. GridCacheQueueItemKey (not compatible with existing queues, need to be
analysed), remove queueName field as is not required at all (use queueId
instead), this will reduce network traffic and improve serialisation
performance.
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.internal.processors.datastructures;
import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.internal.util.typedef.internal.U;
import org.apache.ignite.lang.IgniteUuid;
/**
* Queue item key.
*/
class GridCacheQueueItemKey implements Externalizable, QueueItemKey {
/**
