[jira] [Created] (IGNITE-12107) Docs: request setting IGNITE_HOME to permanent location in Docker
Denis Magda created IGNITE-12107: Summary: Docs: request setting IGNITE_HOME to permanent location in Docker Key: IGNITE-12107 URL: https://issues.apache.org/jira/browse/IGNITE-12107 Project: Ignite Issue Type: Bug Components: documentation Reporter: Denis Magda Fix For: 2.7.6 Update Ignite Docker and Kubernetes documentation stating that IGNITE_HOME, as well as WAL and persistent store directories, must point to the volumes that are not erased or destroyed on restarts. Please refer to this thread for more details: http://apache-ignite-developers.2346864.n4.nabble.com/Metastore-disappears-in-Docker-on-restarts-td43107.html -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IGNITE-12106) Rework ignite binary metadata exception for ease of troubleshooting
Denis Magda created IGNITE-12106:
Summary: Rework ignite binary metadata exception for ease of
troubleshooting
Key: IGNITE-12106
URL: https://issues.apache.org/jira/browse/IGNITE-12106
Project: Ignite
Issue Type: Bug
Components: cache, persistence
Affects Versions: 2.7.5, 2.7.6
Reporter: Denis Magda
Fix For: 2.7.6
An exception of this kind happens regularly and its message doesn't suggest why
it took place and how to fix the issue:
{noformat}
Cannot find metadata for object with compact footer: 2097659979
{noformat}
See this thread for more details:
http://apache-ignite-developers.2346864.n4.nabble.com/Metastore-disappears-in-Docker-on-restarts-td43107.html
As a solution, let's rework this exception to ensure that the way like that
"Ignite work directory might be cleared on restarts. Ensure that IGNITE_HOME
doesn't point to a temp folder or any other folder that is destroyed/erased on
restarts. Presently, IGNITE_HOME points to..."
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (IGNITE-12101) IgniteQueue.removeAll throws NPE
Denis Magda created IGNITE-12101:
Summary: IgniteQueue.removeAll throws NPE
Key: IGNITE-12101
URL: https://issues.apache.org/jira/browse/IGNITE-12101
Project: Ignite
Issue Type: Bug
Affects Versions: 2.5
Reporter: Denis Magda
See more details here:
https://stackoverflow.com/questions/57473783/ignite-2-5-ignitequeue-removeall-throwing-npe
{noformat}
2019-08-09 18:18:39,241 ERROR [Inbound-Main-Pool-13] [TransactionId:
e5b5bfe3-5246-4d54-a4d6-acd550240e13 Request ID - 27845] [ APP=Server,
ACTION=APP_PROCESS, USER=tsgops ] ProcessWorkflowProcessor - Error while
processing CLIENT process
class org.apache.ignite.IgniteException: Failed to serialize object
[typeName=LinkedList]
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:990)
at
org.apache.ignite.internal.processors.datastructures.GridCacheQueueAdapter$QueueIterator.remove(GridCacheQueueAdapter.java:687)
at
java.util.AbstractCollection.removeAll(AbstractCollection.java:376)
at
org.apache.ignite.internal.processors.datastructures.GridCacheQueueProxy.removeAll(GridCacheQueueProxy.java:180)
at
com.me.app.service.support.APPOrderProcessIgniteQueueService.removeAll(APPOrderProcessIgniteQueueService.java:63)
at
com.me.app.service.support.APPOrderContextProcessInputManager.removeAllFromCurrentProcessing(APPOrderContextProcessInputManager.java:201)
at
com.me.app.service.support.APPOrderContextProcessInputManager.lambda$removeAll$3(APPOrderContextProcessInputManager.java:100)
at java.lang.Iterable.forEach(Iterable.java:75)
at
com.me.app.service.support.APPOrderContextProcessInputManager.removeAll(APPOrderContextProcessInputManager.java:100)
at
com.me.app.service.support.APPOrderContextProcessInputManager.removeAll(APPOrderContextProcessInputManager.java:90)
at
com.me.app.processor.support.ProcessWorkflowProcessor.processOrders(ProcessWorkflowProcessor.java:602)
at
com.me.app.processor.support.ProcessWorkflowProcessor.lambda$null$13(ProcessWorkflowProcessor.java:405)
at java.util.HashMap.forEach(HashMap.java:1289)
at
com.me.app.processor.support.ProcessWorkflowProcessor.lambda$null$14(ProcessWorkflowProcessor.java:368)
at java.util.HashMap.forEach(HashMap.java:1289)
at
com.me.app.processor.support.ProcessWorkflowProcessor.lambda$null$15(ProcessWorkflowProcessor.java:354)
at java.util.HashMap.forEach(HashMap.java:1289)
at
com.me.app.processor.support.ProcessWorkflowProcessor.lambda$null$16(ProcessWorkflowProcessor.java:345)
at java.util.HashMap.forEach(HashMap.java:1289)
at
com.me.app.processor.support.ProcessWorkflowProcessor.lambda$executeProcess$17(ProcessWorkflowProcessor.java:337)
at java.util.HashMap.forEach(HashMap.java:1289)
at
com.me.app.processor.support.ProcessWorkflowProcessor.executeProcess(ProcessWorkflowProcessor.java:330)
at
com.me.app.processor.support.ProcessWorkflowProcessor.executeProcess(ProcessWorkflowProcessor.java:302)
at
com.me.app.processor.support.ProcessWorkflowProcessor.lambda$processProcessFromQueue$6(ProcessWorkflowProcessor.java:282)
at
com.me.app.locking.support.IgniteLockingService.execute(IgniteLockingService.java:39)
at
com.me.app.locking.support.IgniteLockingService.execute(IgniteLockingService.java:68)
at
com.me.app.processor.support.ProcessWorkflowProcessor.processProcessFromQueue(ProcessWorkflowProcessor.java:281)
at
com.me.app.facade.listener.support.APPProcessEventListener.listen(APPProcessEventListener.java:49)
at
com.me.app.facade.listener.support.APPProcessEventListener.listen(APPProcessEventListener.java:19)
at
com.me.app.common.listener.support.AbstractEventListener.onMessage(AbstractEventListener.java:44)
at
com.me.app.common.listener.support.AbstractEventListener$$FastClassBySpringCGLIB$$f1379f74.invoke()
at
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
at
com.me.app.common.aop.aspect.MetricsTrackingAspect.trackMetrics(MetricsTrackingAspect.java:142)
at
[jira] [Created] (IGNITE-12054) Upgrade Spark module to 2.4
Denis Magda created IGNITE-12054: Summary: Upgrade Spark module to 2.4 Key: IGNITE-12054 URL: https://issues.apache.org/jira/browse/IGNITE-12054 Project: Ignite Issue Type: Task Components: spark Affects Versions: 2.7.5 Reporter: Denis Magda Fix For: 2.7.6 Users can't use APIs that are already available in Spark 2.4: https://stackoverflow.com/questions/57392143/persisting-spark-dataframe-to-ignite Let's upgrade Spark from 2.3 to 2.4 until we extract the Spark Integration as a separate module that can support multiple Spark versions. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (IGNITE-11942) IGFS and Hadoop Accelerator Discontinuation
Denis Magda created IGNITE-11942: Summary: IGFS and Hadoop Accelerator Discontinuation Key: IGNITE-11942 URL: https://issues.apache.org/jira/browse/IGNITE-11942 Project: Ignite Issue Type: Task Reporter: Denis Magda Fix For: 2.8 The community has voted for the following decision: * IGFS and In-Memory Hadoop Accelerator components are to be discontinued and no longer supported by the community * The existing source code of IGFS and In-Memory Hadoop Accelerator is to be removed from Ignite master. Before that, a special branch like "ignite-igfs-and-hadoop-accelerator" to be forked off the master in order to preserve the sources in Git history for those who might need it. The voting thread: http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42405.html Once the changes are made for Ignite 2.8, please contact Denis Magda to update a public documentation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11919) Change message format for incompatible fields' types changes
Denis Magda created IGNITE-11919:
Summary: Change message format for incompatible fields' types
changes
Key: IGNITE-11919
URL: https://issues.apache.org/jira/browse/IGNITE-11919
Project: Ignite
Issue Type: Task
Reporter: Denis Magda
Fix For: 2.8
Follow this discussion thread to find out the reason for the change:
http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-stops-working-suddenly-during-dev-td42207.html
It's suggested to apply the following format:
_ERROR: Type 'ClassName/TypeName' has a different/incorrect type
for field 'FieldName'. Expected 'type1' but 'type2' was provided. Field
type's modification is unsupported, clean {root_path}/marshaller directory
if the type change is required._
That's an example of how it will look like:
_ERROR: Type 'Person' has a different/incorrect type
for field 'salary'. Expected 'double' but 'string' was provided. Field
type's modification is unsupported, clean {root_path}/marshaller directory
if the type change is required._
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11252) Docs: Index corruption recovery procedure
Denis Magda created IGNITE-11252: Summary: Docs: Index corruption recovery procedure Key: IGNITE-11252 URL: https://issues.apache.org/jira/browse/IGNITE-11252 Project: Ignite Issue Type: Task Components: documentation Affects Versions: 2.7 Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.8 We need to document a recovery procedure if an index corruption happens. Refer to this thread for details and examples of the exception dumped to the logs if the issue occurs: http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-index-corruption-issue-gt-unrecoverable-cluster-td39869.html # Recovering from an index corruption ## Applicable if It is known that an index of a cache is corrupted, but the main data (partition files and WAL) is fine. Show code snippets of possible examples. Find via the references shared in the dev list discussion. ## Steps to recover 1. Stop the node 2. Delete index.bin of the affected caches (path is db//cache-/index.bin) 3. Start the node - Note: At this point the node is active in the cluster but don’t have indexes. It means that it serves SQL queries but their performance can be low. Avoid running SQL queries on large tables at this point 4. Wait for message “Finished indexes rebuilding for cache ” in the Ignite log # Recovering from a persistent storage corruption ## Applicable if A part of the persistent storage (partition files, checkpoint markers or WAL) was corrupted and there is no other way to recover it, but there are healthy copies of all data on other nodes. ## Steps to recover 1. Stop the node 2. Delete all persistence files of the node (best to clear Ignite working directory, storage directory, WAL and WAL archive directories) 3. Make sure consistentId is explicitly set in the configuration of the node - If it isn’t, lookup the generated consistentId using control.sh and set it explicitly in the config or via IGNITE_CONSISTENT_ID (2.8+ only) 4. Start the node 5. Wait for messages for all caches -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11216) Ignite.sh fails on Mac OS and Linux - Java 11
Denis Magda created IGNITE-11216:
Summary: Ignite.sh fails on Mac OS and Linux - Java 11
Key: IGNITE-11216
URL: https://issues.apache.org/jira/browse/IGNITE-11216
Project: Ignite
Issue Type: Task
Affects Versions: 2.7
Reporter: Denis Magda
Ignite.sh fails on Mac OS Mojave with the following JDK version:
java version "11.0.2" 2019-01-15 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.2+9-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.2+9-LTS, mixed mode)
The same issue is reproduced on Linux and the workaround is discussed here:
https://issues.apache.org/jira/browse/IGNITE-3
The exception is as follows:
{noformat}
/Users/dmagda/Downloads/apache-ignite-2.7.0-bin/bin/include/functions.sh: line
40: [: -eq: unary operator expected
./ignite.sh: line 152: [: -eq: unary operator expected
./ignite.sh: line 157: [: -gt: unary operator expected
./ignite.sh: line 170: [: -eq: unary operator expected
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by
org.apache.ignite.internal.util.GridUnsafe$2
(file:/Users/dmagda/Downloads/apache-ignite-2.7.0-bin/libs/ignite-core-2.7.0.jar)
to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of
org.apache.ignite.internal.util.GridUnsafe$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal
reflective access operations
WARNING: All illegal access operations will be denied in a future release
Exception in thread "main" java.lang.ExceptionInInitializerError
at
org.apache.ignite.internal.util.IgniteUtils.(IgniteUtils.java:795)
at
org.apache.ignite.lang.IgniteProductVersion.fromString(IgniteProductVersion.java:305)
at
org.apache.ignite.internal.IgniteVersionUtils.(IgniteVersionUtils.java:71)
at
org.apache.ignite.startup.cmdline.CommandLineStartup.(CommandLineStartup.java:99)
Caused by: java.lang.RuntimeException: jdk.internal.misc.JavaNioAccess class is
unavailable.
at
org.apache.ignite.internal.util.GridUnsafe.javaNioAccessObject(GridUnsafe.java:1453)
at
org.apache.ignite.internal.util.GridUnsafe.(GridUnsafe.java:112)
... 4 more
Caused by: java.lang.IllegalAccessException: class
org.apache.ignite.internal.util.GridUnsafe cannot access class
jdk.internal.misc.SharedSecrets (in module java.base) because module java.base
does not export jdk.internal.misc to unnamed module @4f83df68
at
java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:361)
at
java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:591)
at java.base/java.lang.reflect.Method.invoke(Method.java:558)
at
org.apache.ignite.internal.util.GridUnsafe.javaNioAccessObject(GridUnsafe.java:1450)
... 5 more
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11113) JVM 9-11 startup parameters have to be mentioned in every section of getting started
Denis Magda created IGNITE-3: Summary: JVM 9-11 startup parameters have to be mentioned in every section of getting started Key: IGNITE-3 URL: https://issues.apache.org/jira/browse/IGNITE-3 Project: Ignite Issue Type: Task Components: documentation Reporter: Artem Budnikov Ignite.sh fails on Mac OS with JDK 11. The following startup parameters [1] have to be mentioned in callout blocks of every section in the getting started guide [2] where it's explained how to start a node with ignite.sh or launch an application. The user has to know how to: * Fix ignite.sh/ignite.bat to make them workable * What to do in an IDEA like IntellijIdean and Eclipse to make examples and apps workable [1] https://apacheignite.readme.io/docs/getting-started#section-running-ignite-with-java-9-10-11 [2] https://apacheignite.readme.io/docs/getting-started -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10845) Ignite Production Readiness Section Enhancement
Denis Magda created IGNITE-10845: Summary: Ignite Production Readiness Section Enhancement Key: IGNITE-10845 URL: https://issues.apache.org/jira/browse/IGNITE-10845 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.8 Enhance Ignite production readiness section with points from here: https://www.gridgain.com/resources/blog/checklist-assembling-your-first-apacher-ignitetm-cluster -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9953) Dropping hadoop accelerator downloads
Denis Magda created IGNITE-9953: --- Summary: Dropping hadoop accelerator downloads Key: IGNITE-9953 URL: https://issues.apache.org/jira/browse/IGNITE-9953 Project: Ignite Issue Type: Task Components: hadoop Reporter: Denis Magda Assignee: Peter Ivanov Fix For: 2.7 The community agreed to drop the Hadoop distributions of Apache Ignite. Note, that it does not mean dropping any features. We are still going to support IGFS, in-memory map reduce, spark integration, etc. However, the hadoop-accelerator downloadable is putting extra pressure on the community to test and document, which can be avoided. All the features will still be supported through regular Ignite distribution. Discussion: http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Dropping-hadoop-accelerator-download-td36617.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9798) Add TensorFlow Integration Page to Ignite website
Denis Magda created IGNITE-9798: --- Summary: Add TensorFlow Integration Page to Ignite website Key: IGNITE-9798 URL: https://issues.apache.org/jira/browse/IGNITE-9798 Project: Ignite Issue Type: Task Components: site Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.7 We need to create a dedicated page for Ignite and TensorFlow integration. Please put it under Machine Learning item of the Features menu. [~abchaudhri], will provide a reference to the readme.io page with in-depth integration description. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9797) Refer to PHP, Python and Node.JS getting started guides from the website
Denis Magda created IGNITE-9797: --- Summary: Refer to PHP, Python and Node.JS getting started guides from the website Key: IGNITE-9797 URL: https://issues.apache.org/jira/browse/IGNITE-9797 Project: Ignite Issue Type: Task Components: site Affects Versions: 2.7 Reporter: Denis Magda Assignee: Prachi Garg This page includes a section with the list of references to getting started guides: https://ignite.apache.org/features/multilanguage.html Add references to Python, PHP and Node.JS docs on readme.io (Instantiation and Configuration pages on readme). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9564) Ignite K8 service and sticky session with AWS K8 deployments
Denis Magda created IGNITE-9564: --- Summary: Ignite K8 service and sticky session with AWS K8 deployments Key: IGNITE-9564 URL: https://issues.apache.org/jira/browse/IGNITE-9564 Project: Ignite Issue Type: Task Reporter: Denis Magda Assignee: Ilya Murchenko Fix For: 2.8 _sessionAffinity_ parameter is not supported by AWS Kubernetes which causes the failure of the standard Ignite service deployment: http://apache-ignite-users.70518.x6.nabble.com/Error-installing-Ignite-on-K8s-td23999.html See how Ignite service configuration have to be updated considering sticky session capabilities of AWS: https://aws.amazon.com/elasticloadbalancing/details/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9503) Visor CMD shows wrong REPLICATED cache max size
Denis Magda created IGNITE-9503:
---
Summary: Visor CMD shows wrong REPLICATED cache max size
Key: IGNITE-9503
URL: https://issues.apache.org/jira/browse/IGNITE-9503
Project: Ignite
Issue Type: Task
Components: visor
Affects Versions: 2.5
Reporter: Denis Magda
Assignee: Alexey Kuznetsov
Fix For: 2.6
Started 2 nodes with a single replicated cache. Preloaded _50_ entries
there.
Visor CMD *cache* command shows a wrong total
{code}
++
|Name(@)|Mode| Nodes | Entries (Heap /
Off-heap) | Hits| Misses | Reads | Writes |
++
| CacheDataStreamerExample(@c0) | REPLICATED | 2 | min: 246106 (0 / 246106)
| min: 0| min: 0| min: 0| min: 0|
| || | avg: 25.00 (0.00 /
25.00) | avg: 0.00 | avg: 0.00 | avg: 0.00 | avg: 0.00 |
| || | max: 253894 (0 / 253894)
| max: 0| max: 0| max: 0| max: 0|
++
{code}
You find a correct total number only if *cache -a* is used and you calculate
the sum of entries on each node manually:
{code}
+===+
| Node ID8(@), IP | CPUs | Heap Used | CPU Load | Up Time|
Size | Hi/Mi/Rd/Wr |
+===+
| 44A2CF9C(@n1), 192.168.1.64 | 4| 19.63 % | 0.43 % | 00:01:46.229 |
Total: 253894| Hi: 0 |
| | | | | |
Heap: 0| Mi: 0 |
| | | | | |
Off-Heap: 253894 | Rd: 0 |
| | | | | |
Off-Heap Memory: 0 | Wr: 0 |
+-+--+---+--+--+--+-+
| 72DEC7B5(@n0), 192.168.1.64 | 4| 9.69 %| 0.50 % | 00:02:00.456 |
Total: 246106| Hi: 0 |
| | | | | |
Heap: 0| Mi: 0 |
| | | | | |
Off-Heap: 246106 | Rd: 0 |
| | | | | |
Off-Heap Memory: 0 | Wr: 0 |
+---+
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-9305) Wrong off-heap size is reported for a node
Denis Magda created IGNITE-9305:
---
Summary: Wrong off-heap size is reported for a node
Key: IGNITE-9305
URL: https://issues.apache.org/jira/browse/IGNITE-9305
Project: Ignite
Issue Type: Task
Affects Versions: 2.6
Reporter: Denis Magda
Fix For: 2.7
Was troubleshooting an Ignite deployment today and couldn't find out from the
logs what was the actual off-heap space used.
Those were the given memory resoures (Ignite 2.6):
{code}
[2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager] Topology snapshot
[ver=1, servers=1, clients=0, CPUs=64, offheap=30.0GB, heap=24.0GB]
{code}
And that weird stuff was reported by the node (pay attention to the last line):
{code}
[2018-08-16 15:45:50,211][INFO
][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
^-- Node [id=c033026e, name=cluster_31-Dec-2017, uptime=00:38:00.257]
^-- H/N/C [hosts=1, nodes=1, CPUs=64]
^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
^-- PageMemory [pages=6997377]
^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
^-- Non heap [used=144MB, free=-1%, comm=148MB] - this line is always the
same!
{code}
Had to change the code by using {code}dataRegion.getPhysicalMemoryPages(){code}
to find out that actual off-heap usage size was
{code}
>>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
{code}
The logs have to report the following instead:
{code}
^-- Off-heap {Data Region 1} [used={dataRegion1.getPhysicalMemorySize()},
free=X%, comm=dataRegion1.maxSize()]
^-- Off-heap {Data Region 2} [used={dataRegion2.getPhysicalMemorySize()},
free=X%, comm=dataRegion2.maxSize()]
{code}
If Ignite persistence is enabled then the following extra lines have to be
added to see the disk used space:
{code}
^-- Off-heap {Data Region 1}: used={dataRegion1.getTotalAllocatedSize() -
dataRegion1.getPhysicalMemorySize()}
^-- Off-heap {Data Region 2} [used={dataRegion2.getTotalAllocatedSize() -
dataRegion2.getPhysicalMemorySize()}]
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-8994) Configuring dedicated volumes for WAL and data with Kuberenetes
Denis Magda created IGNITE-8994: --- Summary: Configuring dedicated volumes for WAL and data with Kuberenetes Key: IGNITE-8994 URL: https://issues.apache.org/jira/browse/IGNITE-8994 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Fix For: 2.7 The current StatefulSet documentation request only one persistent volume for both WAL and data/index files: https://apacheignite.readme.io/docs/stateful-deployment#section-statefulset-deployment However, according to Ignite performance guide the WAL has to be located on a dedicated volume: https://apacheignite.readme.io/docs/durable-memory-tuning#section-separate-disk-device-for-wal Provide StatefulSet configuration that shows how to request separate volumes for the WAL and data/index files. If needed, provide YAML configs for StorageClass and volume claims. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8993) Configuring sticky LoadBalancer for Ignite Service with Kubernetes
Denis Magda created IGNITE-8993: --- Summary: Configuring sticky LoadBalancer for Ignite Service with Kubernetes Key: IGNITE-8993 URL: https://issues.apache.org/jira/browse/IGNITE-8993 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Fix For: 2.7 Ignite service used for Ignite pods auto-discovery and access to the cluster from remote applications is deployed as LoadBalancer: https://apacheignite.readme.io/docs/ignite-service This might lead to problems when a stateful session is needed between an app and the cluster. For instance, Ignite JDBC driver preserves the state of an opened connection meaning that once LoadBalancer connects the driver to an Ignite pod, all the queries have to be redirected to that Ignite pod only (unless the pod is down). We need to show how to configure a sticky LoadBalancer that will assign a client connection to a specific pod and won't change it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8967) Automatic Handling of Long Stop-the-World Pauses
Denis Magda created IGNITE-8967: --- Summary: Automatic Handling of Long Stop-the-World Pauses Key: IGNITE-8967 URL: https://issues.apache.org/jira/browse/IGNITE-8967 Project: Ignite Issue Type: New Feature Reporter: Denis Magda Based on the discussion on the dev list: http://apache-ignite-developers.2346864.n4.nabble.com/Automatic-Handling-of-Long-Stop-the-World-Pauses-td31847.html Ignite goes with a number of self-healing capabilities: * Ignite can already handle critical failures such as OOM, File I/O issues, etc. [1] * There is an endeavor to fix cluster lock-ins due to partition map exchange issues. [2] There is one more notorious problem that might affect Ignite deployments which is long stop-the-world GC pauses. We did a little progress in this direction [3] by providing particular metrics that help to monitor the pauses. Presently, I would either create specific policies similar to the critical failures policies [4] or just add a long STP issue to the list of critical failures [1]. [1] https://apacheignite.readme.io/docs/critical-failures-handling [2] http://apache-ignite-developers.2346864.n4.nabble.com/IEP-25-Partition-Map-Exchange-hangs-resolving-td31819.html [3] https://issues.apache.org/jira/browse/IGNITE-6171 [4] https://apacheignite.readme.io/docs/critical-failures-handling#section-failure-handling -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8804) Windows WSL configuration has to be added to Ignite configs
Denis Magda created IGNITE-8804: --- Summary: Windows WSL configuration has to be added to Ignite configs Key: IGNITE-8804 URL: https://issues.apache.org/jira/browse/IGNITE-8804 Project: Ignite Issue Type: Task Reporter: Denis Magda Assignee: Peter Ivanov Fix For: 2.6 Ignite requires to use a special config file when a node is spanned up in the WSL environment: https://apacheignite.readme.io/docs/rpm-and-deb-setup#section-starting-ignite-cluster The following needs to be done: * Add `wsl-default-config` to `ignite/config` folder * The header of the file needs to include a comments section that explains how to start several nodes with it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8695) RPM|DEB: Remove requirement to run Ignite stand-along under root
Denis Magda created IGNITE-8695: --- Summary: RPM|DEB: Remove requirement to run Ignite stand-along under root Key: IGNITE-8695 URL: https://issues.apache.org/jira/browse/IGNITE-8695 Project: Ignite Issue Type: Task Reporter: Denis Magda Assignee: Peter Ivanov Fix For: 2.6 If Ignite is started as a stand-alone process we require to do that under the root user: https://apacheignite.readme.io/docs/getting-started#section-run-ignite-as-a-stand-alone-process Noone would do that in prod. We need to remove this requirement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8654) Move SET command documentation to Operational Commands section
Denis Magda created IGNITE-8654: --- Summary: Move SET command documentation to Operational Commands section Key: IGNITE-8654 URL: https://issues.apache.org/jira/browse/IGNITE-8654 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.6 We need to be consistent and document SQL commands under SQL Reference section. So, move and slightly rework (no need to show step by step JDBC example) the doc from here: https://apacheignite-sql.readme.io/v2.5/docs/jdbc-driver#section-streaming there: https://apacheignite-sql.readme.io/v2.5/docs/set Once moved, ensure the format complies with the rest of the commands and there is a callout saying the feature is supported for JDBC only. Refer to COPY as to an example: https://apacheignite-sql.readme.io/v2.5/docs/copy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8653) Elaborate on split brain and failures handling with ZooKeeper Discovery
Denis Magda created IGNITE-8653: --- Summary: Elaborate on split brain and failures handling with ZooKeeper Discovery Key: IGNITE-8653 URL: https://issues.apache.org/jira/browse/IGNITE-8653 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Fix For: 2.6 Attachments: Zookeeper Discovery SPI - Split Brain 2.png, Zookeeper Discovery SPI - Split Brain.png, Zookeeper Discovery SPI - ZooKeeper Cluster.png Split-brain and failures handling section don't do a good job explaining how these scenarios are handled: https://apacheignite.readme.io/docs/zookeeper-discovery#section-failures-and-split-brain-handling This needs to be improved. Consider using the graphics attached. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8589) Prepare Node.JS Thin Client documentation
Denis Magda created IGNITE-8589: --- Summary: Prepare Node.JS Thin Client documentation Key: IGNITE-8589 URL: https://issues.apache.org/jira/browse/IGNITE-8589 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Pavel Petroshenko Fix For: 2.6 The hidden page is presently here: https://apacheignite.readme.io/v2.4/docs/nodejs-thin-client Once all the suggestions are resolved, we might split this page into several pages. I can do this closer to 2.6 release. [~pavel.petroshenko], [~alexey.kosenchuk], please address the following suggestions for now: Installation section: * Ignite repository has to be used instead of the private one * The section needs to explain the installation steps once the NPM module is released. It's fine to put the current installation instructions under "Installing from Sources" section. Data Types: * The term "field" is truly confusing. Does it really imply any possible data (value, key, element of an array)? Why can't we use the term "value" instead? * Provide a code snippet that shows how to configure the explicit mapping * Default mapping has to be documented in place on readme.io (no references to the private GIT repo) Configuring Ignite Client *Failover on a reconnect code snippet. How to pass a list of the endpoints. General: * You use CacheClient or just Cache notion to refer to an instance of the cache class. Let's use one term. I prefer the Cache one or IgniteCache if it's named this way. * Add a reference to the examples once they are merged to Ignite repo (see the latest section on the readme page) * Add a reference to Node.JS APIs within supported APIs section. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8548) Make Apache Ignite JCache 1.1 specification compliant
Denis Magda created IGNITE-8548: --- Summary: Make Apache Ignite JCache 1.1 specification compliant Key: IGNITE-8548 URL: https://issues.apache.org/jira/browse/IGNITE-8548 Project: Ignite Issue Type: Task Reporter: Denis Magda Fix For: 2.6 JCache specification's license was changed to Apache 2.0, and 1.1 version was released: https://groups.google.com/forum/#!topic/jsr107/BC1qKqknzKU Ignite needs to: * Upgrade to JCacahe 1.1 in general to use Apache 2.0 license * Become JCache 1.1 compliant implementing new interfaces and passing TCK -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8524) Document consistency check utilities
Denis Magda created IGNITE-8524: --- Summary: Document consistency check utilities Key: IGNITE-8524 URL: https://issues.apache.org/jira/browse/IGNITE-8524 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Fix For: 2.5 Ignite 2.5 will go with special consistency check utilities that, for instance, ensure that the data stays consistent across backups, indexes are correct and many other things. More details can be taken from here: * https://issues.apache.org/jira/browse/IGNITE-8277 * https://issues.apache.org/jira/browse/IGNITE-7467 Here is an empty page that is created for the documentation: https://apacheignite.readme.io/docs/consistency-check-utilities -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8455) Cover main Ignite memory modes
Denis Magda created IGNITE-8455: --- Summary: Cover main Ignite memory modes Key: IGNITE-8455 URL: https://issues.apache.org/jira/browse/IGNITE-8455 Project: Ignite Issue Type: Task Components: site Reporter: Denis Magda Assignee: Denis Magda Fix For: 2.6 Ignite website doesn't mention various modes on how Ignite native persistence can be used: - no disk, data is in memory-only (potentially over a 3rd-party database) - disk is a copy of the memory (only for recovery purposes) - disk is a data storage with memory used as a performant caching layer - indexes in memory and data on disk for better technical cost of ownership. I believe we should put it in a table and explain it on the memory-centric page. More details: http://apache-ignite-developers.2346864.n4.nabble.com/missing-website-info-td30145.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8349) Move GA Grid description to ML site page
Denis Magda created IGNITE-8349: --- Summary: Move GA Grid description to ML site page Key: IGNITE-8349 URL: https://issues.apache.org/jira/browse/IGNITE-8349 Project: Ignite Issue Type: Task Components: site Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.5 Prachi, please move GA grid overview from here [1] to a dedicated section of the main ML page [2]. Don't forget to remove GA grid beta from the features page [3]. [1] https://ignite.apache.org/addons.html#ga-grid [2] https://ignite.apache.org/features/machinelearning.html [3] https://ignite.apache.org/features.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8330) Docs: Put JDBC/ODBC URL in Brackets When Connecting From Bash
Denis Magda created IGNITE-8330: --- Summary: Docs: Put JDBC/ODBC URL in Brackets When Connecting From Bash Key: IGNITE-8330 URL: https://issues.apache.org/jira/browse/IGNITE-8330 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.5 We need to add a warning callout with the content below to both JDBC and ODBC documentation pages: Title: Put JDBC/ODBC URL in Brackets When Connecting From Bash Description: Make sure to put the connection URL in " brackets when opening it up from a bash environment, as follows: "jdbc:ignite:thin://[address]:[port];user=ignite;password=[password]" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8305) Update metrics page with new storage and memory related metics
Denis Magda created IGNITE-8305: --- Summary: Update metrics page with new storage and memory related metics Key: IGNITE-8305 URL: https://issues.apache.org/jira/browse/IGNITE-8305 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.5 The new metrics are added by the endeavor: https://issues.apache.org/jira/browse/IGNITE-8078 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8304) Document failures handing in Ignite
Denis Magda created IGNITE-8304: --- Summary: Document failures handing in Ignite Key: IGNITE-8304 URL: https://issues.apache.org/jira/browse/IGNITE-8304 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Andrey Gura Fix For: 2.5 IEP-14: https://cwiki.apache.org/confluence/display/IGNITE/IEP-14+Ignite+failures+handling The following failures will be treated as critical and can lead to an automatic node termination depending on configuration: * System critical errors (e.g. OutOfMemoryError); * Unintentional system worker termination (e.g. due to an unhandled exception); * Cluster node segmentation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8241) Docs: Triggering automatic rebalancing if the whole baseline topology is not recovered
Denis Magda created IGNITE-8241: --- Summary: Docs: Triggering automatic rebalancing if the whole baseline topology is not recovered Key: IGNITE-8241 URL: https://issues.apache.org/jira/browse/IGNITE-8241 Project: Ignite Issue Type: Task Components: documentation Affects Versions: 2.4 Reporter: Denis Magda Assignee: Denis Magda Fix For: 2.5 The ticket is created as a result of the following discussion: http://apache-ignite-developers.2346864.n4.nabble.com/Triggering-rebalancing-on-timeout-or-manually-if-the-baseline-topology-is-not-reassembled-td29299.html The rebalancing doesn't happen if one of the nodes goes down, thus, shrinking the baseline topology. It complies with our assumption that the node should be recovered soon and there is no need to waste CPU/memory/networking resources of the cluster shifting the data around. However, there are always edge cases. I was reasonably asked how to trigger the rebalancing within the baseline topology manually or on timeout if: * It's not expected that the failed node would be resurrected in the nearest time and * It's not likely that that node will be replaced by the other one. Until we embedd special facilities in the baseline topology that would consider such situations we can document the following workaround. A user application/tool/script has to subscribe to node_left events and remove the failed node from the baseline topology in some time. Once the node is removed, the baseline topology will be changed, and the rebalancing will be kicked off. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8171) Document how to rollback transactions to let PME complete
Denis Magda created IGNITE-8171:
---
Summary: Document how to rollback transactions to let PME complete
Key: IGNITE-8171
URL: https://issues.apache.org/jira/browse/IGNITE-8171
Project: Ignite
Issue Type: New Feature
Components: documentation
Reporter: Denis Magda
Fix For: 2.5
Some Ignite operations provoke partition map exchange process within Ignite to
ensure the partitions distribution state is synchronized cluster-wide. Topology
update events and a start of a new distributed cache are examples of those
operations.
When the partition map exchange starts, Ignite acquires a global lock at a
particular stage. The lock can't be obtained until pending transactions are
running in parallel. If there is a transaction that runs for a while, then it
will prevent the partition map exchange process from the start freezing some
operations such as a new node join process.
This property allows to rollback such long transactions to let Ignite acquire
the lock faster and initiate the partition map exchange process. The timeout is
enforced only at the time of the partition map exchange process.
See {{TransactionConfiguraion}} and
{{IgniteTransactions.localActiveTransactions and withLabel}} methods.
Original ticket:
https://issues.apache.org/jira/browse/IGNITE-6827
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-8137) Explain how to see SQL schema and table structure in Ignite
Denis Magda created IGNITE-8137:
---
Summary: Explain how to see SQL schema and table structure in
Ignite
Key: IGNITE-8137
URL: https://issues.apache.org/jira/browse/IGNITE-8137
Project: Ignite
Issue Type: New Feature
Components: documentation
Reporter: Denis Magda
Fix For: 2.6
Presently, Ignite doesn't ship {{SHOW TABLE}} command. Thus, it's not clear how
to get tables and schema structures in Ignite.
Until this command is not supported we have to explain that:
* Many tools use JDBC and ODBC metadata related parameters can pull that
information out of there and present it to an end user. For instance, SQLline
utilizes {{!table}} command that gets data from JDBC metadata methods. DBeaver
utilizes the same approach.
* Let's create {{Database Administration}} section under SQL reference
(https://apacheignite-sql.readme.io/docs/sql-reference-overview) and presently
list approaches supported by various tools (sqlline, dbeaver, etc) and how to
get the same information from the drivers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-8091) Create a documentation section describing metrics and monitoring facilities
Denis Magda created IGNITE-8091: --- Summary: Create a documentation section describing metrics and monitoring facilities Key: IGNITE-8091 URL: https://issues.apache.org/jira/browse/IGNITE-8091 Project: Ignite Issue Type: New Feature Components: documentation Reporter: Denis Magda Fix For: 2.6 Ignite doesn't have a single documentation section which would aggregate all the metrics the project has. There are random documentation pages spread out across data grid, memory, persistence and other topics. The new section will include the following: * How to gather metrics (JMX, logs, APIs) * Detailed metrics coverage for memory, persistence, rebalancing, compute, etc. Consider IGNITE-8078 and IGNITE-8077 * Monitoring tools * How to integrate with 3rd party tools. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8090) Explain disk space compaction by WAL and compaction techniques
Denis Magda created IGNITE-8090: --- Summary: Explain disk space compaction by WAL and compaction techniques Key: IGNITE-8090 URL: https://issues.apache.org/jira/browse/IGNITE-8090 Project: Ignite Issue Type: New Feature Components: documentation Reporter: Denis Magda Fix For: 2.5 Explain how WAL occupies the disk space and how to enable the compaction: https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-WALHistorySize There needs to be a special section in WAL documentation as well as in the capacity planning. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8081) Document Kubernetes RBAC configuration to avoid 403 exception
Denis Magda created IGNITE-8081: --- Summary: Document Kubernetes RBAC configuration to avoid 403 exception Key: IGNITE-8081 URL: https://issues.apache.org/jira/browse/IGNITE-8081 Project: Ignite Issue Type: New Feature Components: documentation Reporter: Denis Magda Fix For: 2.5 It's reported by the users that sometimes Ignite Kubernetes IP finder fails to join the cluster due to security issues. To prevent the exception happening we need to document how to set up a Service Account for Ignite pods: https://stackoverflow.com/questions/49395481/how-to-setmasterurl-in-ignite-xml-config-for-kubernetes-ipfinder/49405879#49405879 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8040) Add Ignite nightly builds references to the site
Denis Magda created IGNITE-8040: --- Summary: Add Ignite nightly builds references to the site Key: IGNITE-8040 URL: https://issues.apache.org/jira/browse/IGNITE-8040 Project: Ignite Issue Type: New Feature Components: site Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.5 We should add a link to the nightly builds somewhere on the site: https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_NightlyRelease_RunApacheIgniteNightlyRelease Here is a relevant discussion: http://apache-ignite-developers.2346864.n4.nabble.com/Apache-Ignite-nightly-release-builds-td27966.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7992) Clarify how expiration policy works for durable memory and 3rd pary
Denis Magda created IGNITE-7992: --- Summary: Clarify how expiration policy works for durable memory and 3rd pary Key: IGNITE-7992 URL: https://issues.apache.org/jira/browse/IGNITE-7992 Project: Ignite Issue Type: New Feature Components: documentation Affects Versions: 2.4 Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.5 Update the expiration policies page with the result of the following discussion: http://apache-ignite-developers.2346864.n4.nabble.com/Data-eviction-expiration-from-Ignite-persistence-td24419i20.html#a28081 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7907) Revisit and update the DBeaver documentations
Denis Magda created IGNITE-7907: --- Summary: Revisit and update the DBeaver documentations Key: IGNITE-7907 URL: https://issues.apache.org/jira/browse/IGNITE-7907 Project: Ignite Issue Type: New Feature Components: documentation Reporter: Denis Magda Assignee: Akmal Chaudhri Fix For: 2.5 Seems that our documentation is outdated: http://apache-ignite-users.70518.x6.nabble.com/Ignite-with-dBeaver-td20368.html#a20376 Need to go from the beginning and update the existing parts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7906) Create languages comparison page
Denis Magda created IGNITE-7906: --- Summary: Create languages comparison page Key: IGNITE-7906 URL: https://issues.apache.org/jira/browse/IGNITE-7906 Project: Ignite Issue Type: New Feature Components: documentation Reporter: Denis Magda Fix For: 2.5 In addition to Java, .NET and C++ thick clients, the community is going to support a variety of thin client for many other languages such as Python, Node.JS, etc. It will be beneficial to create a page that depicts capabilities of every language like this one: https://hazelcast.org/clients-languages/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7903) Explain how to tune performance by using WAL modes
Denis Magda created IGNITE-7903: --- Summary: Explain how to tune performance by using WAL modes Key: IGNITE-7903 URL: https://issues.apache.org/jira/browse/IGNITE-7903 Project: Ignite Issue Type: New Feature Components: documentation Reporter: Denis Magda Fix For: 2.5 Explain how to improve performance by switching between the WAL modes: https://apacheignite.readme.io/docs/durable-memory-tuning Basically, LOG_ONLY and BACKGROUND mode can be considered as an alternative for FULL_SYNC. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7893) Release Java Thin client documentation
Denis Magda created IGNITE-7893: --- Summary: Release Java Thin client documentation Key: IGNITE-7893 URL: https://issues.apache.org/jira/browse/IGNITE-7893 Project: Ignite Issue Type: New Feature Components: documentation Reporter: Denis Magda Fix For: 2.5 Java thin client is already available for usage and documented: https://apacheignite.readme.io/v2.3/docs/java-thin-client It will be officially released in 2.5. For now, the users have to build it from sources. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7884) Ship SQL scripts with same database in Ignite
Denis Magda created IGNITE-7884:
---
Summary: Ship SQL scripts with same database in Ignite
Key: IGNITE-7884
URL: https://issues.apache.org/jira/browse/IGNITE-7884
Project: Ignite
Issue Type: New Feature
Components: examples, sql
Affects Versions: 2.5
Reporter: Denis Magda
Fix For: 2.5
As per discussion on @dev list, let's ship two kind of SQL scripts that can be
used to create and populate a database quickly:
[http://apache-ignite-developers.2346864.n4.nabble.com/Shipping-SQL-script-CSV-files-in-Ignite-examples-td27440.html]
The database below is suggested to be used because it represents real data,
allows setting up affinity collocation and demonstrate meaningful SQL queries
with joins:
[https://github.com/dmagda/ignite_world_demo/blob/master/ignite_world.sql]
First, this script can be used as is and executed in any SQL tool such as
SQLline.
Second, to demonstrate the use of {{COPY}} command we should take the same
script, create a file that will store DDL statements and turn INSERTS into a
CSV file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-7870) Update Discovery and Communication Technical Docs
Denis Magda created IGNITE-7870: --- Summary: Update Discovery and Communication Technical Docs Key: IGNITE-7870 URL: https://issues.apache.org/jira/browse/IGNITE-7870 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.5 Attachments: IgniteNetworkingOverview.md Existing discovery and communication pages are not in the best shape and require an overhaul. Use the revisited version (attached) contributed by a community member and see how to improve or mix discovery, communication, servers & clients docs from readme. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7802) Clarify how to hook Ignite and a 3rd party persistence
Denis Magda created IGNITE-7802: --- Summary: Clarify how to hook Ignite and a 3rd party persistence Key: IGNITE-7802 URL: https://issues.apache.org/jira/browse/IGNITE-7802 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Akmal Chaudhri Fix For: 2.5 The existing documentation doesn't explain all the steps needed to integrate Ignite with a 3rd party db: [https://apacheignite.readme.io/docs/3rd-party-store] Specifically, the following has to be covered: * Dedicated section for RDBMS setup. Copying of a JDBC driver, enabling a right CacheStore implementation, defining the mapping, or automating this process with Web Console. * NoSQL databases section. Refer to Cassandra pages. * Other 3rd party stores. Clarify how to write a custom CacheStore that is not supported out of the box. The goal is to avoid questions like this: http://apache-ignite-users.70518.x6.nabble.com/Ignite-with-mariadb-td20101.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7728) Put together a doc that shows how to blend SQL with k/v APIs
Denis Magda created IGNITE-7728: --- Summary: Put together a doc that shows how to blend SQL with k/v APIs Key: IGNITE-7728 URL: https://issues.apache.org/jira/browse/IGNITE-7728 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Denis Magda Fix For: 2.5 More and more people start blending SQL with key-value APIs in Ignite. Usually, they create tables/caches with DDL and wish to use key-value later as well: [https://stackoverflow.com/questions/48795533/how-do-i-read-data-from-cache-using-javaapi-after-i-put-it-through-jdbc] We already have a project that demonstrates this approach: [https://github.com/dmagda/ignite_world_demo] Put together a doc that points out to it and elaborates on this topic. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7711) Explain the case-sensitivity of schema and table/cache
Denis Magda created IGNITE-7711:
---
Summary: Explain the case-sensitivity of schema and table/cache
Key: IGNITE-7711
URL: https://issues.apache.org/jira/browse/IGNITE-7711
Project: Ignite
Issue Type: Task
Components: documentation
Reporter: Denis Magda
Fix For: 2.5
Users don't get how to use table and schema name in Ignite:
[https://stackoverflow.com/questions/48723946/apache-ignite-querying-a-cache-through-an-sql-driver]
The confusion comes from the interpretation of term "case-insensitive". All SQL
databases work this way:
1) If I create some object and define its name without quotes, a database will
convert it to upper case (sometimes lower case). Then if I query this object
without quotes, a query is also transformed to upper case. This is what I call
"case-insensitivity" - it doesn't matter how you refer to this object, it works
still.
CREATE TABLE myTable -> CREATE TABLE MYTABLE
SELECT FROM myTable -> SELECT FROM MYTABLE -> OK
SELECT FROM MyTable -> SELECT FROM MYTABLE -> OK
SELECT FROM "myTable" -> SELECT FROM myTable -> *{color:#ff}FAIL!{color}*
2) If I create an object with quotes, then the database doesn't convert it to
upper case, and the original name in quotes is *the only way* to access the
object:
CREATE TABLE "myTable" -> CREATE TABLE myTable
SELECT FROM myTable -> SELECT FROM MYTABLE -> *{color:#ff}FAIL!{color}*
SELECT FROM MyTable -> SELECT FROM MYTABLE ->
*{color:#ff}FAIL!{color}*SELECT FROM "myTable" -> SELECT FROM myTable -> OK
Now back to Ignite case. Our schemas are case-sensitive. so the only way to
access it is through quotes:
CREATE SCHEMA "cacheName"; // This is what Ignite does internally
SELECT * FROM "cacheName".myTable -> OK
SELECT * FROM cacheName.myTable -> *{color:#ff}FAIL!{color}*
Our users know this, and all their queries are written with schema names in
quotation marks. Now consider what would happen should we change schema names
to case-insensitive mode:
CREATE SCHEMA cacheName;
SELECT * FROM "cacheName".myTable -> *{color:#ff}FAIL, +all old quries
start to fail!+{color}*
SELECT * FROM cacheName.myTable -> OK, now it works
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-7668) Cover menu and main page references with GA labels
Denis Magda created IGNITE-7668: --- Summary: Cover menu and main page references with GA labels Key: IGNITE-7668 URL: https://issues.apache.org/jira/browse/IGNITE-7668 Project: Ignite Issue Type: Sub-task Components: site Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.4 It's useful to see insights on how frequent people click on specific references shown on the main page or menus. Let's add labels to all the benefits and features so that GA can track these events for us. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7645) Clarify eviction policy documentation
Denis Magda created IGNITE-7645: --- Summary: Clarify eviction policy documentation Key: IGNITE-7645 URL: https://issues.apache.org/jira/browse/IGNITE-7645 Project: Ignite Issue Type: Task Reporter: Denis Magda Assignee: Denis Magda Fix For: 2.4 Eviction policies work differently depending on the configuration that might be one of the following: * Just off-heap memory w/o Ignite persistence * off-heap memory + on-heap cache * off-heap memory + Ignite persistence * off-heap memory + swap or cache store Cover all these scenarios on the main eviction doc page:https://apacheignite.readme.io/docs/evictions More details: http://apache-ignite-developers.2346864.n4.nabble.com/Eviction-policies-with-persistence-td26588.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7595) Find and switch to alternate documentation engine
Denis Magda created IGNITE-7595: --- Summary: Find and switch to alternate documentation engine Key: IGNITE-7595 URL: https://issues.apache.org/jira/browse/IGNITE-7595 Project: Ignite Issue Type: Task Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.5 Current readme.io documentation has many drawbacks that make the life of Ignite technical writers hard. Some of the problems are: * Each "version" is just a copy of the previous one. When fixing something, you have to update all the versions. * No good way to review changes. * "Propose edit" functionality is a not suitable for review. You can only accept or reject an edit, no way to communicate with a contributor, etc * There is no way to prevent Google from indexing old documentation versions. Thus, it's common to come across old doc version in a google search. We might consider GitHub based documentation or another approach. The discussion is here: http://apache-ignite-developers.2346864.n4.nabble.com/Move-documentation-from-readme-io-to-GitHub-pages-td16409.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7574) Ignite Getting Started confuses developers
Denis Magda created IGNITE-7574: --- Summary: Ignite Getting Started confuses developers Key: IGNITE-7574 URL: https://issues.apache.org/jira/browse/IGNITE-7574 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Denis Magda Fix For: 2.4 By some reason, we suggest to build Ignite from sources at the very beginning of the getting started guide: [https://apacheignite.readme.io/docs/getting-started] I got a feedback that this confuses a lot especially because it's 100% optional! The reporter wasted much of his time trying to build Ignite with JDK 9 and could succeed only after a private conversation with me. Revisit and rework the guide. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7560) Baseline topology documentation
Denis Magda created IGNITE-7560: --- Summary: Baseline topology documentation Key: IGNITE-7560 URL: https://issues.apache.org/jira/browse/IGNITE-7560 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Denis Magda Fix For: 2.4 Document baseline topology architecture, benefits, and usage: https://cwiki.apache.org/confluence/display/IGNITE/IEP-4+Baseline+topology+for+caches -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7536) Document baseline topology feature and its WebConsole screen
Denis Magda created IGNITE-7536: --- Summary: Document baseline topology feature and its WebConsole screen Key: IGNITE-7536 URL: https://issues.apache.org/jira/browse/IGNITE-7536 Project: Ignite Issue Type: Task Reporter: Denis Magda Assignee: Denis Magda Fix For: 2.4 Document Base Line topology: [https://cwiki.apache.org/confluence/display/IGNITE/IEP-4+Baseline+topology+for+caches] and how to manage it with Web Console and control.sh script (Ignite activation doc has to be updated). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7490) Doc: clarify what's table name and how to change fields/indexes for QueryEntity/Annotations configs
Denis Magda created IGNITE-7490: --- Summary: Doc: clarify what's table name and how to change fields/indexes for QueryEntity/Annotations configs Key: IGNITE-7490 URL: https://issues.apache.org/jira/browse/IGNITE-7490 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Denis Magda Fix For: 2.4 Users got confused on: * what should be a table name if SQL is configured with QueryEntity/Annotations * how to add/remove columns if QueryEntity/Annotations were used for initial configurations http://apache-ignite-users.70518.x6.nabble.com/query-on-BinaryObject-index-and-table-td19631.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7470) Eviction policy for SQL on-heap row cache
Denis Magda created IGNITE-7470: --- Summary: Eviction policy for SQL on-heap row cache Key: IGNITE-7470 URL: https://issues.apache.org/jira/browse/IGNITE-7470 Project: Ignite Issue Type: New Feature Components: sql Affects Versions: 2.4 Reporter: Denis Magda SQL on-heap row cache was introduced in Ignite 2.4: [https://apacheignite-sql.readme.io/v2.3/docs/performance-and-debugging_24_hidden#sql-on-heap-row-cache] The cache can occupy as much RAM as allocated and used for off-heap data regions. If a user stores dozens or hundreds of gigabytes in the data regions the on-heap row cache will become enormous and the cluster will suffer from atrocious GC pauses. Let's come up with an eviction policy for this cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7419) Document swap usage in Ignite 2.x memory architecture
Denis Magda created IGNITE-7419: --- Summary: Document swap usage in Ignite 2.x memory architecture Key: IGNITE-7419 URL: https://issues.apache.org/jira/browse/IGNITE-7419 Project: Ignite Issue Type: Task Components: documentation Environment: Explain how swap is supported and works in Ignite. Provide a rationale on Ignite persistence vs swap. In addition, looks people don't catch what happens when memory region goes beyond the maximum size. Revisit the persistence configuration: http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-3-Swap-Path-configuration-is-causing-issue-td19040.html#a19046 Reporter: Denis Magda Assignee: Denis Magda Fix For: 2.4 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7404) Use various backgrounds for Ignite main page's main blocks
Denis Magda created IGNITE-7404: --- Summary: Use various backgrounds for Ignite main page's main blocks Key: IGNITE-7404 URL: https://issues.apache.org/jira/browse/IGNITE-7404 Project: Ignite Issue Type: Task Components: site Reporter: Denis Magda Assignee: Vica Abramova Fix For: 2.4 This ticket is intended to improve the main page: https://ignite.apache.org All the content that goes below the banner is shown on a *white* background. As a result, it's difficult to separate main building blocks from each other *visually*. Suggest to come up with a second background and rotate it together with the white one among the building blocks. For instance: - Apache Ignite definition and use cases (white) - Features (new background - probably grey) - How Ignite compares (white) - Screencast (new background) - News (white) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7403) Improve content on What's Ignite page
Denis Magda created IGNITE-7403: --- Summary: Improve content on What's Ignite page Key: IGNITE-7403 URL: https://issues.apache.org/jira/browse/IGNITE-7403 Project: Ignite Issue Type: Task Components: site Affects Versions: 2.4 Reporter: Denis Magda Assignee: Vica Abramova A proposed of a new draft for the What's Ignite page: https://ignite.apache.org/whatisignite-2.html The goal is to make the page more informative and tell about the main things of Ignite or provide references to them. Overall, the structure should be as follows: - Product name and definition. - Diagram. - Features and Benefits - Ignite facts -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7358) Document ELB based AWS IP Finder
Denis Magda created IGNITE-7358: --- Summary: Document ELB based AWS IP Finder Key: IGNITE-7358 URL: https://issues.apache.org/jira/browse/IGNITE-7358 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Prachi Garg Priority: Critical Fix For: 2.4 Document the new IP finder that can be used for AWS deployments - IGNITE-7304. The main documentation should be hosted here: https://apacheignite-mix.readme.io/docs/amazon-aws and there we need to provide a reference to ELB IP finder: https://apacheignite.readme.io/docs/cluster-config -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7351) Create a page for continuous queries
Denis Magda created IGNITE-7351: --- Summary: Create a page for continuous queries Key: IGNITE-7351 URL: https://issues.apache.org/jira/browse/IGNITE-7351 Project: Ignite Issue Type: Sub-task Components: site Reporter: Denis Magda Fix For: 2.4 Create a page for continuous queries putting it under Features menu on the site. This capability is a strong distinguisher from relational databases that do not support continuous notifications and processing built around them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7346) Enable Ignite cache events per cache
Denis Magda created IGNITE-7346:
---
Summary: Enable Ignite cache events per cache
Key: IGNITE-7346
URL: https://issues.apache.org/jira/browse/IGNITE-7346
Project: Ignite
Issue Type: Improvement
Components: cache
Reporter: Denis Magda
Fix For: 2.4
Whenever a cache event from {{EventType.EVTS_CACHE}},
{{EventType.EVTS_CACHE_REBALANCE}} or {{EventType.EVTS_CACHE_QUERY}} set is
enabled and a listener is registered, the cluster will accumulate and send out
events for all the deployed caches.
Such behavior brings down performance especially if hundreds and thousands of
caches are used while the user is interested in the monitoring of specific
caches.
Let's add a configuration property like {{CacheConfiguration.setEventsEnabled}}
to enable or disable the events per cache. The events will be enabled by
default to comply with existing behavior.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-7345) Spark Data Frames and Ignite Documentation
Denis Magda created IGNITE-7345: --- Summary: Spark Data Frames and Ignite Documentation Key: IGNITE-7345 URL: https://issues.apache.org/jira/browse/IGNITE-7345 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Assignee: Nikolay Izhikov Fix For: 2.4 Spark Data frames integration [1] needs to be documented in this [2] domain. [1] http://apache-ignite-developers.2346864.n4.nabble.com/Spark-data-frames-integration-merged-td25817.html [2] https://apacheignite-fs.readme.io/docs -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7318) Move all SQL examples out of "data grid" package
Denis Magda created IGNITE-7318: --- Summary: Move all SQL examples out of "data grid" package Key: IGNITE-7318 URL: https://issues.apache.org/jira/browse/IGNITE-7318 Project: Ignite Issue Type: Task Reporter: Denis Magda Assignee: Denis Magda Priority: Blocker Fix For: 2.4 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7292) Optimize Ignite main page load performance
Denis Magda created IGNITE-7292: --- Summary: Optimize Ignite main page load performance Key: IGNITE-7292 URL: https://issues.apache.org/jira/browse/IGNITE-7292 Project: Ignite Issue Type: Task Reporter: Denis Magda Assignee: Prachi Garg Priority: Blocker Fix For: 2.4 Attachments: Screen Shot 2017-12-22 at 9.49.02 AM.png Ignite main page load performance is poor - takes from 3 to 5 seconds to load in completely. That might affect the SEO optimizations we've done so far. These are immediate actions to alleviate the situation: # Reduce banner and all the images size to 72 ppi resolution and save in the JPG format. Use photoshop "Image Size" dialog. The banner and thumbnails of screencasts were already changed, just need to regenerate the CSS and merge the patch. # The next fierce contributors to the poor performance are screencasts youtube video. It takes 700ms for each screencast to make a roundtrip to YouTube. See the screenshot attached where 3 base.js calls are triggered by the video links. This has to be done asynchronously. # Preload GitHub, Twitter butter and feed asynchronously too. The overall goal of this optimization is to speed up the loading to 2.5 seconds (no more). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7291) Apache Ignite "in use at" section
Denis Magda created IGNITE-7291: --- Summary: Apache Ignite "in use at" section Key: IGNITE-7291 URL: https://issues.apache.org/jira/browse/IGNITE-7291 Project: Ignite Issue Type: Bug Reporter: Denis Magda Assignee: Denis Magda Fix For: 2.4 Put together a list of publicly disclosed Apache Ignite use cases and present them on the main page right below the benefits. The "Ignite is in use at" section might be like Cassandra's "PROVEN" section: http://cassandra.apache.org A list of the known use cases: * https://www.imcsummit.org/us/sessions/implementation-investment-book-record-ibor-using-apache-ignitegridgain * https://www.imcsummit.org/us/sessions/high-availability-and-disaster-recovery-imdg * https://www.imcsummit.org/us/sessions/ignite-compute-grid-in-cloud * https://youtu.be/1D8hyLWMtfM * https://www.imcsummit.org/us/sessions/how-in-memory-solutions-can-assist-saas-integrations -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7268) SQL Tutorials Per Language
Denis Magda created IGNITE-7268: --- Summary: SQL Tutorials Per Language Key: IGNITE-7268 URL: https://issues.apache.org/jira/browse/IGNITE-7268 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Fix For: 2.5 Create SQL getting started tutorials for major programming languages relying on JDBC/ODBC drivers. Have Spanner's tutorials in mind as an example: https://cloud.google.com/spanner/docs/getting-started/nodejs/ This is an umbrella ticket. Language-specific tutorials will be handled as subtasks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7251) Remove term "fabric" from Ignite deliverables
Denis Magda created IGNITE-7251:
---
Summary: Remove term "fabric" from Ignite deliverables
Key: IGNITE-7251
URL: https://issues.apache.org/jira/browse/IGNITE-7251
Project: Ignite
Issue Type: Task
Reporter: Denis Magda
Priority: Blocker
Fix For: 2.4
Apache Ignite binary releases still include “fabric” word in their names:
https://ignite.apache.org/download.cgi#binaries
For instance, this is a full name of the previous release -
apache-ignite-fabric-2.3.0-bin.
It’s a little oversight on our side because the project has not been positioned
as a fabric for a while.
Remove “fabric” from the name and have the binary releases named as -
apache-ignite-{version}-bin.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-7250) Apache Ignite Transactional Subsystem Architecture
Denis Magda created IGNITE-7250: --- Summary: Apache Ignite Transactional Subsystem Architecture Key: IGNITE-7250 URL: https://issues.apache.org/jira/browse/IGNITE-7250 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Fix For: 2.4 A couple of community members prepared a well-defined document about Ignite transactions architecture. See the PDF version attached. Use this document to enrich the existing page on Ignite transactions. Even more, it makes sense to define a unique category for transactions and create several pages. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7198) Integrate with Apache Beam
Denis Magda created IGNITE-7198: --- Summary: Integrate with Apache Beam Key: IGNITE-7198 URL: https://issues.apache.org/jira/browse/IGNITE-7198 Project: Ignite Issue Type: Task Reporter: Denis Magda Fix For: 2.5 Apache Beam (https://beam.apache.org/) provides a unified API for batch and streaming processing. It can be seen as a generic/portable API with multiple implementations (Spark, Flink, etc.). Apache Ignite perfectly fits Beam architecture as a Runner: https://beam.apache.org/contribute/runner-guide/ Here is Beam's capability matrix. Let's support as much as we're capable of: https://beam.apache.org/documentation/runners/capability-matrix/ Most likely the integration should go to Beam's repository: https://github.com/apache/beam/tree/master/runners Discussion on the dev list: http://apache-ignite-developers.2346864.n4.nabble.com/Apache-Ignite-as-a-distributed-processing-back-ends-td25122.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7178) Ignite fails to start on non x86/64 architectures
Denis Magda created IGNITE-7178:
---
Summary: Ignite fails to start on non x86/64 architectures
Key: IGNITE-7178
URL: https://issues.apache.org/jira/browse/IGNITE-7178
Project: Ignite
Issue Type: Bug
Affects Versions: 2.3
Reporter: Denis Magda
Priority: Blocker
Fix For: 2.4
Ignite fails with the exception below on AIX and Solar Spark. Most likely
because of the bug that pops up on big-endian and non x86/64 CPU architectures:
{code}
ERROR 2017-11-02 11:59:01.331 [grid-nio-worker-tcp-comm-1-#22]
org.apache.ignite.internal.util.nio.GridDirectParser: Failed to read message
[msg=GridIoMessage [plc=0, topic=null, topicOrd=-1, ordered=false,
timeout=0, skipOnTimeout=false, msg=null],
buf=java.nio.DirectByteBuffer[pos=16841 lim=16844 cap=32768],
reader=DirectMessageReader [state=DirectMessageState [pos=0,
stack=[StateItem [stream=DirectByteBufferStreamImplV2
[baseOff=1100144027456, arrOff=-1, tmpArrOff=0, tmpArrBytes=0,
msgTypeDone=true, msg=GridDhtPartitionsFullMessage [parts=null,
partCntrs=null, partCntrs2=null, partHistSuppliers=null, partsToReload=null,
topVer=null, errs=null, compress=false, resTopVer=null, partCnt=0,
super=GridDhtPartitionsAbstractMessage [exchId=GridDhtPartitionExchangeId
[topVer=AffinityTopologyVersion [topVer=5, minorTopVer=0], discoEvt=null,
nodeId=e3ac3f40, evt=NODE_JOINED], lastVer=GridCacheVersion [topVer=0,
order=1509612963224, nodeOrder=0], super=GridCacheMessage [msgId=369,
depInfo=null, err=null, skipPrepare=false]]], mapIt=null, it=null,
arrPos=-1, keyDone=false, readSize=-1, readItems=0, prim=0, primShift=0,
uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0, lastFinished=true],
state=0], StateItem [stream=DirectByteBufferStreamImplV2
[baseOff=1100144027456, arrOff=-1, tmpArrOff=0, tmpArrBytes=0,
msgTypeDone=true, msg=CacheGroupAffinityMessage [], mapIt=null, it=null,
arrPos=-1, keyDone=true, readSize=7, readItems=2, prim=0, primShift=0,
uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0, lastFinished=true],
state=0], StateItem [stream=DirectByteBufferStreamImplV2
[baseOff=1100144027456, arrOff=-1, tmpArrOff=0, tmpArrBytes=0,
msgTypeDone=true, msg=GridLongList [idx=0, arr=[]], mapIt=null, it=null,
arrPos=-1, keyDone=false, readSize=512, readItems=47, prim=0, primShift=0,
uuidState=0, uuidMost=0, uuidLeast=0, uuidLocId=0, lastFinished=true],
state=0], StateItem [stream=DirectByteBufferStreamImplV2
[baseOff=1100144027456, arrOff=-1, tmpArrOff=40, tmpArrBytes=40,
msgTypeDone=false, msg=null, mapIt=null, it=null, arrPos=-1, keyDone=false,
readSize=-1, readItems=0, prim=0, primShift=0, uuidState=0, uuidMost=0,
uuidLeast=0, uuidLocId=0, lastFinished=true], state=0], null, null, null,
null, null, null]], lastRead=true], ses=GridSelectorNioSessionImpl
[worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=1,
bytesRcvd=404253, bytesSent=1989, bytesRcvd0=16886, bytesSent0=28,
select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-1,
igniteInstanceName=null, finished=false, hashCode=-2134841549,
interrupted=false, runner=grid-nio-worker-tcp-comm-1-#22]]],
writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
readBuf=java.nio.DirectByteBuffer[pos=16841 lim=16844 cap=32768],
inRecovery=GridNioRecoveryDescriptor [acked=6, resendCnt=0, rcvCnt=0,
sentCnt=6, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode
[id=7683662b-16c9-42b7-aa0d-8328a60fc58e, addrs=[127.0.0.1],
sockAddrs=[/127.0.0.1:6250], discPort=6250, order=1, intOrder=1,
lastExchangeTime=1509612963744, loc=false, ver=2.3.0#20171028-sha1:8add7fd5,
isClient=false], connected=true, connectCnt=11, queueLimit=131072,
reserveCnt=175, pairedConnections=false],
outRecovery=GridNioRecoveryDescriptor [acked=6, resendCnt=0, rcvCnt=0,
sentCnt=6, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode
[id=7683662b-16c9-42b7-aa0d-8328a60fc58e, addrs=[127.0.0.1],
sockAddrs=[/127.0.0.1:6250], discPort=6250, order=1, intOrder=1,
lastExchangeTime=1509612963744, loc=false, ver=2.3.0#20171028-sha1:8add7fd5,
isClient=false], connected=true, connectCnt=11, queueLimit=131072,
reserveCnt=175, pairedConnections=false], super=GridNioSessionImpl
[locAddr=/127.0.0.1:6284, rmtAddr=/127.0.0.1:61790,
createTime=1509613141318, closeTime=0, bytesSent=28, bytesRcvd=16886,
bytesSent0=28, bytesRcvd0=16886, sndSchedTime=1509613141318,
lastSndTime=1509613141318, lastRcvTime=1509613141318, readsPaused=false,
filterChain=FilterChain[filters=[GridNioCodecFilter
[parser=o.a.i.i.util.nio.GridDirectParser@8e66d834, directMode=true],
GridConnectionBytesVerifyFilter], accepted=true]]]
java.lang.IllegalArgumentException: null
at java.nio.Buffer.position(Buffer.java:255) ~[?:1.8.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readArrayLE(DirectByteBufferStreamImplV2.java:1587)
~[ignite-core-2.3.0.jar:2.3.0]
at
[jira] [Created] (IGNITE-7132) Investigate and document Ignite Persistence usage in Docker
Denis Magda created IGNITE-7132: --- Summary: Investigate and document Ignite Persistence usage in Docker Key: IGNITE-7132 URL: https://issues.apache.org/jira/browse/IGNITE-7132 Project: Ignite Issue Type: Task Components: documentation Reporter: Denis Magda Fix For: 2.4 Inspired by the following talk: http://apache-ignite-users.70518.x6.nabble.com/Ignite-in-docker-Native-Persistence-td18426.html The aim is to investigate how is better to deploy Ignite persistence in Docker. Probably with the usage of Docker volumes: https://docs.docker.com/engine/admin/volumes/volumes/#use-a-read-only-volume Once the configuration option is clear the following documentation needs to be updated: https://apacheignite.readme.io/docs/docker-deployment -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7131) Document Web Console deployment in Kubernetes
Denis Magda created IGNITE-7131: --- Summary: Document Web Console deployment in Kubernetes Key: IGNITE-7131 URL: https://issues.apache.org/jira/browse/IGNITE-7131 Project: Ignite Issue Type: Task Affects Versions: 2.5 Reporter: Denis Magda The ticket is inspired by the following topic: http://apache-ignite-users.70518.x6.nabble.com/Web-Console-on-Kubernetes-Cluster-td18591.html It will be great to put together a documentation about Web Console deployment on Kubernetes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7062) Ignite page with video resources and recording
Denis Magda created IGNITE-7062: --- Summary: Ignite page with video resources and recording Key: IGNITE-7062 URL: https://issues.apache.org/jira/browse/IGNITE-7062 Project: Ignite Issue Type: Task Components: site Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.4 There is a plenty of recordings of Ignite meetups, webinars and conference talks available on the Internet. Some of them introduce basic components and capabilities, some share best practices and pitfalls while the other share use cases. Generally, it's beneficial for both Ignite community and users to gather and expose the most useful ones under a special video recording section. For instance, we might consider these talks to be added right away: * Ignite use case: https://youtu.be/1D8hyLWMtfM * Ignite essentials: https://www.youtube.com/watch?v=G22L2KW9gEQ * Kubernetes: https://www.youtube.com/watch?v=igDB0wyodr8 Instead of creating a new page for this purpose I would rework the screencasts' page combining all the media content there: https://ignite.apache.org/screencasts.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7061) Rework Features menu and page
Denis Magda created IGNITE-7061: --- Summary: Rework Features menu and page Key: IGNITE-7061 URL: https://issues.apache.org/jira/browse/IGNITE-7061 Project: Ignite Issue Type: Task Components: site Reporter: Denis Magda Fix For: 2.4 The features menu and the page [1] is overloaded and confusing. As a technical guy, I feel lost trying to grasp what’s important and what’s secondary. That deters me from digging into the project. Rework the menu and page in accordance with this discussion: http://apache-ignite-developers.2346864.n4.nabble.com/Reworking-Ignite-site-s-quot-Features-quot-menu-td24569.html [1] https://ignite.apache.org/features.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7060) Prepare Architecture section for the site
Denis Magda created IGNITE-7060: --- Summary: Prepare Architecture section for the site Key: IGNITE-7060 URL: https://issues.apache.org/jira/browse/IGNITE-7060 Project: Ignite Issue Type: Task Components: site Reporter: Denis Magda Fix For: 2.4 In addition to the features, it's useful to introduce Ignite architecture right in the Features menu covering the following: * Overview * Clustering and Deployment * Distributed Database * Durable Memory -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7058) Make out a site page for ACID Transactions
Denis Magda created IGNITE-7058: --- Summary: Make out a site page for ACID Transactions Key: IGNITE-7058 URL: https://issues.apache.org/jira/browse/IGNITE-7058 Project: Ignite Issue Type: Task Components: site Reporter: Denis Magda Fix For: 2.4 ACID transactions are a major feature of Ignite and have to be exposed under the Features menu on the site. Make out the page covering the following: * 2Phase Commit Protocol * Pessimistic and Optimistic Modes * Deadlock detection -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7057) Create Key-Value Page for the site
Denis Magda created IGNITE-7057: --- Summary: Create Key-Value Page for the site Key: IGNITE-7057 URL: https://issues.apache.org/jira/browse/IGNITE-7057 Project: Ignite Issue Type: Task Components: site Reporter: Denis Magda Fix For: 2.4 Prepare a page to describe Ignite key-value APIs. The page will go to the Features menu and should cover the following: * Scope of K/V APIs. * JCache * Benefits of JCache * How to combine K/V and SQL APIs * Examples Rework existing data grid and key-value store pages: https://ignite.apache.org/features/datagrid.html https://ignite.apache.org/use-cases/database/key-value-store.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7056) Prepare Multi-Language support page for the site
Denis Magda created IGNITE-7056: --- Summary: Prepare Multi-Language support page for the site Key: IGNITE-7056 URL: https://issues.apache.org/jira/browse/IGNITE-7056 Project: Ignite Issue Type: Task Components: site Reporter: Denis Magda Fix For: 2.4 Prepare Ignite's multi-language page that will go under the features menu. The page should encompass the following: * Java, .NET and C++ native APIs. * Supported drivers and protocols that can be used in various languages. * A couple of examples. Update the image by combining Java, NET and C++ logos. Remove the pages below setting up a redirect the multi-languages page: https://ignite.apache.org/features/java.html https://ignite.apache.org/features/dotnet.html https://ignite.apache.org/features/cpp.html https://ignite.apache.org/features/clientprotos.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7009) NPE when Ignite persistence is enabled for Hadoop accelerator
Denis Magda created IGNITE-7009:
---
Summary: NPE when Ignite persistence is enabled for Hadoop
accelerator
Key: IGNITE-7009
URL: https://issues.apache.org/jira/browse/IGNITE-7009
Project: Ignite
Issue Type: Bug
Reporter: Denis Magda
Assignee: Vladimir Ozerov
Priority: Blocker
Fix For: 2.4
Reported from the user list. Print out a meaningful exception instead of the
NPE.
I have installed apache-ignite-hadoop-2.3.0-bin on CDH and able start it
successfully. But when I try to enable persistence providing the config file,
it fails.
{code}
[root@hdfc01gw01 ~]# echo $IGNITE_HOME
/applocaldata/ce_project/apache_ignite/apache-ignite-hadoop-2.3.0-bin
[ad_aagaiton@hdfc01gw01 ~]$ sudo su -
[root@hdfc01gw01 ~]# $IGNITE_HOME/bin/ignite.sh
/applocaldata/ce_project/apache_ignite/apache-ignite-fabric-2.3.0-bin/examples/config/persistentstore/example-persistent-store.xml
[05:15:47]__
[05:15:47] / _/ ___/ |/ / _/_ __/ __/
[05:15:47] _/ // (7 7// / / / / _/
[05:15:47] /___/\___/_/|_/___/ /_/ /___/
[05:15:47]
[05:15:47] ver. 2.3.0#20171027-sha1:8add7fd5
[05:15:47] 2017 Copyright(C) Apache Software Foundation
[05:15:47]
[05:15:47] Ignite documentation: http://ignite.apache.org
[05:15:47]
[05:15:47] Quiet mode.
[05:15:47] ^-- Logging to file
'/applocaldata/ce_project/apache_ignite/apache-ignite-hadoop-2.3.0-bin/work/log/ignite-e570a3f7.log'
[05:15:47] ^-- To see **FULL** console log here add -DIGNITE_QUIET=false
or "-v" to ignite.{sh|bat}
[05:15:47]
[05:15:47] OS: Linux 2.6.32-504.3.3.el6.x86_64 amd64
[05:15:47] VM information: Java(TM) SE Runtime Environment 1.7.0_67-b01
Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 24.65-b04
[05:15:47] Configured plugins:
[05:15:47] ^-- None
[05:15:47]
[05:15:47] Message queue limit is set to 0 which may lead to potential OOMEs
when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to
message queues growth on sender and receiver sides.
[05:15:47] Security status [authentication=off, tls/ssl=off]
[05:15:48] HADOOP_HOME is set to
/opt/cloudera/parcels/CDH-5.4.9-1.cdh5.4.9.p1230.998/lib/hadoop
[05:15:48] Resolved Hadoop classpath locations:
/opt/cloudera/parcels/CDH-5.4.9-1.cdh5.4.9.p1230.998/lib/hadoop,
/opt/cloudera/parcels/CDH-5.4.9-1.cdh5.4.9.p1230.998/lib/hadoop-hdfs,
/opt/cloudera/parcels/CDH-5.4.9-1.cdh5.4.9.p1230.998/lib/hadoop-mapreduce
[2017-11-23 05:15:49,518][ERROR][main][IgniteKernal] Got exception while
starting (will rollback startup routine).
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.hadoop.jobtracker.HadoopJobTracker.jobMetaCache(HadoopJobTracker.java:206)
at
org.apache.ignite.internal.processors.hadoop.jobtracker.HadoopJobTracker.onKernalStart(HadoopJobTracker.java:239)
at
org.apache.ignite.internal.processors.hadoop.HadoopProcessor.onKernalStart(HadoopProcessor.java:105)
at
org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1060)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1909)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1652)
at
org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1080)
at
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:998)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:884)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:783)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:653)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:622)
at org.apache.ignite.Ignition.start(Ignition.java:347)
at
org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302)
[05:15:49] Ignite node stopped OK [uptime=00:00:03.151]
class org.apache.ignite.IgniteException: null
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:966)
at org.apache.ignite.Ignition.start(Ignition.java:350)
at
org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:302)
Caused by: class org.apache.ignite.IgniteCheckedException: null
at
org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1110)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1909)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1652)
at
org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1080)
at
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:998)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:884)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:783)
at
[jira] [Created] (IGNITE-6993) Document SMTP server documentation for Web Console
Denis Magda created IGNITE-6993: --- Summary: Document SMTP server documentation for Web Console Key: IGNITE-6993 URL: https://issues.apache.org/jira/browse/IGNITE-6993 Project: Ignite Issue Type: Improvement Components: documentation Affects Versions: 2.3 Reporter: Denis Magda Assignee: Alexey Kuznetsov Fix For: 2.4 Document how to configure a custom SMTP server for Web Console on this domain under "Ignite Web Console" section: https://apacheignite-tools.readme.io/docs -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6980) Automatic cancelling of hanging Ignite operations
Denis Magda created IGNITE-6980: --- Summary: Automatic cancelling of hanging Ignite operations Key: IGNITE-6980 URL: https://issues.apache.org/jira/browse/IGNITE-6980 Project: Ignite Issue Type: Bug Reporter: Denis Magda Priority: Critical Fix For: 2.4 If an Ignite operation hangs due to some reason due to an internal problem or buggy application code it needs to eventual fail after a timeout fires. Take atomic operations case brought by Val to our attention recently: http://apache-ignite-developers.2346864.n4.nabble.com/Timeouts-in-atomic-cache-td19839.html An application must not freeze waiting for a human being intervention if an atomic update fails internally. Even more, I would let all possible operation to fail after a timeout fires: - Ignite compute computations. - Ignite services calls. - Atomic/transactional cache updates. - SQL queries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6966) Average time metrics are not calculated for client driven operations
Denis Magda created IGNITE-6966:
---
Summary: Average time metrics are not calculated for client driven
operations
Key: IGNITE-6966
URL: https://issues.apache.org/jira/browse/IGNITE-6966
Project: Ignite
Issue Type: Bug
Reporter: Denis Magda
Priority: Critical
Fix For: 2.4
Cache operations executed from a client-side are not accounted in average time
metrics. Use this reproducer [1] performing the following:
* Start a server node [2] that will report
{{getAveragePutTime}}/{{getAverageGetTime}} metrics in a loop.
* Start a client node [3] that will report the same metrics and do cache
updates/reads.
Both nodes show {{0}} for those metrics.
[1] https://github.com/dmagda/IgniteMetricsExampe/
[2]
https://github.com/dmagda/IgniteMetricsExampe/blob/master/src/main/java/IgniteMetricsExample.java
[3]
https://github.com/dmagda/IgniteMetricsExampe/blob/master/src/main/java/IgniteClientMetricsExample.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6953) OOM on the client node freezes the whole cluster
Denis Magda created IGNITE-6953: --- Summary: OOM on the client node freezes the whole cluster Key: IGNITE-6953 URL: https://issues.apache.org/jira/browse/IGNITE-6953 Project: Ignite Issue Type: Bug Reporter: Denis Magda Fix For: 2.4 It's reported that if an OOM happens on the client side the whole cluster becomes unresponsive: http://apache-ignite-users.70518.x6.nabble.com/Out-of-memory-in-client-node-freezes-complete-cluster-td18044.html Let's reproduce and prevent this by bringing the client node down automatically. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6952) Deserialization failures on AIX
Denis Magda created IGNITE-6952:
---
Summary: Deserialization failures on AIX
Key: IGNITE-6952
URL: https://issues.apache.org/jira/browse/IGNITE-6952
Project: Ignite
Issue Type: Bug
Affects Versions: 2.3
Reporter: Denis Magda
Assignee: Vladimir Ozerov
Fix For: 2.4
Ignite fails on AIX with the following stack trace:
{code}
GridConnectionBytesVerifyFilter], accepted=true]]]
java.lang.IllegalArgumentException: null
at java.nio.Buffer.position(Buffer.java:255) ~[?:1.8.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readArrayLE(DirectByteBufferStreamImplV2.java:1587)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readArrayLE(DirectByteBufferStreamImplV2.java:1542)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readLongArray(DirectByteBufferStreamImplV2.java:1013)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.DirectMessageReader.readLongArray(DirectMessageReader.java:212)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.GridLongList.readFrom(GridLongList.java:558)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImplV2.java:1165)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.read(DirectByteBufferStreamImplV2.java:1785)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readCollection(DirectByteBufferStreamImplV2.java:1244)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.DirectMessageReader.readCollection(DirectMessageReader.java:333)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.CacheGroupAffinityMessage.readFrom(CacheGroupAffinityMessage.java:292)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImplV2.java:1165)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.read(DirectByteBufferStreamImplV2.java:1785)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMap(DirectByteBufferStreamImplV2.java:1294)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.DirectMessageReader.readMap(DirectMessageReader.java:345)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsFullMessage.readFrom(GridDhtPartitionsFullMessage.java:645)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImplV2.java:1165)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.DirectMessageReader.readMessage(DirectMessageReader.java:311)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.managers.communication.GridIoMessage.readFrom(GridIoMessage.java:262)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridDirectParser.decode(GridDirectParser.java:90)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioCodecFilter.onMessageReceived(GridNioCodecFilter.java:114)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridConnectionBytesVerifyFilter.onMessageReceived(GridConnectionBytesVerifyFilter.java:133)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioServer$HeadFilter.onMessageReceived(GridNioServer.java:3388)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioFilterChain.onMessageReceived(GridNioFilterChain.java:175)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1261)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2272)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2048)
[ignite-core-2.3.0.jar:2.3.0]
at
[jira] [Created] (IGNITE-6925) Simplify cache metrics activation
Denis Magda created IGNITE-6925:
---
Summary: Simplify cache metrics activation
Key: IGNITE-6925
URL: https://issues.apache.org/jira/browse/IGNITE-6925
Project: Ignite
Issue Type: Bug
Security Level: Public (Viewable by anyone)
Reporter: Denis Magda
The user needs to do 3 things to enabled cache metrics:
- set {{statisticsEnabled}} to {{true}}.
- set not a dummy {{EventsStorageSpi}}
- list metrics of the interest.
This process has to be reduced to 2 steps or, preferably, to 1.
More details are here:
http://apache-ignite-developers.2346864.n4.nabble.com/Annoying-extra-steps-for-enabling-metrics-td21865.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6830) Document how to set up secured channel for JDBC Client Driver
Denis Magda created IGNITE-6830:
---
Summary: Document how to set up secured channel for JDBC Client
Driver
Key: IGNITE-6830
URL: https://issues.apache.org/jira/browse/IGNITE-6830
Project: Ignite
Issue Type: Task
Security Level: Public (Viewable by anyone)
Reporter: Denis Magda
JDBC Client Driver allows opening secured connections to a cluster by means of
{{SecurityCredentialsProvider}}. This features needs to be documented:
https://apacheignite-sql.readme.io/docs/jdbc-client-driver
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6814) Detailed memory consumption on start and OOM reporting
Denis Magda created IGNITE-6814: --- Summary: Detailed memory consumption on start and OOM reporting Key: IGNITE-6814 URL: https://issues.apache.org/jira/browse/IGNITE-6814 Project: Ignite Issue Type: Improvement Security Level: Public (Viewable by anyone) Affects Versions: 2.0 Reporter: Denis Magda Priority: Critical Fix For: 2.4 Presently Ignite allocates 20% of RAM on a node startup, however, the user doesn't see automatically chosen memory settings. Also, if there a node runs out of RAM and throws an OOM error there are no hints on why this happened and how to fix it. Suggestions: * Print out memory settings on startup for every data region set up (maximum size, initial size, etc.) * Provide guidelines on how to overcome OOM when it happens. Specify a data region name with all its current parameters and suggest three possible things - tweak maximum memory, setup eviction policies or enable Ignite persistence. All this is covered in official docs: https://apacheignite.readme.io/docs#section-data-regions -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6813) Update diagrams on readme.io documentation
Denis Magda created IGNITE-6813: --- Summary: Update diagrams on readme.io documentation Key: IGNITE-6813 URL: https://issues.apache.org/jira/browse/IGNITE-6813 Project: Ignite Issue Type: Bug Security Level: Public (Viewable by anyone) Components: documentation Reporter: Denis Magda Assignee: Prachi Garg Priority: Minor Fix For: 2.4 The last page on Apache Ignite site were updated with new version of diagrams: https://ignite.apache.org/ features/deploy.html https://ignite.apache.org/ features/igfs.html https://ignite.apache.org/ features/mapreduce.html https://ignite.apache.org/ features/streaming.html https://ignite.apache.org/use- cases/caching/database- caching.html https://ignite.apache.org/use- cases/caching/hibernate-l2- cache.html https://ignite.apache.org/use- cases/caching/web-session- clustering.html https://ignite.apache.org/use- cases/hadoop/hdfs-cache https://ignite.apache.org/use- cases/hadoop/mapreduce Please go through all the readme.io pages replacing the old diagrams with new ones. Make this for every "overview" page of every documentation section we have. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6806) Document logs forwarding to ELK stack
Denis Magda created IGNITE-6806: --- Summary: Document logs forwarding to ELK stack Key: IGNITE-6806 URL: https://issues.apache.org/jira/browse/IGNITE-6806 Project: Ignite Issue Type: Task Security Level: Public (Viewable by anyone) Components: documentation Reporter: Denis Magda Assignee: Prachi Garg Fix For: 2.4 Ignite logs can be forwarded to remote servers and software stacks for advanced processing. For instance, this project shows how to do this for ELK stack: https://github.com/akuramshingg/elk-monitor Mention it on Ignite logging page: https://apacheignite.readme.io/docs/logging -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
Denis Magda created IGNITE-6804: --- Summary: Print a warning if HashMap is passed into bulk update operations Key: IGNITE-6804 URL: https://issues.apache.org/jira/browse/IGNITE-6804 Project: Ignite Issue Type: Improvement Security Level: Public (Viewable by anyone) Components: cache Reporter: Denis Magda Priority: Critical Fix For: 2.4 Ignite newcomers tend to stumble on deadlocks simply because the keys are passed in an unordered HashMap. Propose to do the following: * update bulk operations Java docs. * print out a warning if HashMap is passed into a bulk method (instead of SortedMap) and contains more than 1 element. However, we should make sure that we only print that warning once and not every time the API is called. More details are here: http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6785) Affinity field name forced to be upper-case
Denis Magda created IGNITE-6785:
---
Summary: Affinity field name forced to be upper-case
Key: IGNITE-6785
URL: https://issues.apache.org/jira/browse/IGNITE-6785
Project: Ignite
Issue Type: Task
Security Level: Public (Viewable by anyone)
Components: sql
Affects Versions: 2.1
Reporter: Denis Magda
Fix For: 2.4
If an SQL schema and cache is created with CREATE TABLE command and a user
wants to use key-value APIs creating its own custom key class, then (at least)
the key class's affinity field forced to be written in upper-case.
Steps to reproduce using the project attached:
* start a node with {{./ignite.sh ../examples/config/example-ignite.xml}}.
* create {{City}} table using {{ignite_world.sql}}. SQLline is one of the
quickest ways: https://apacheignite-sql.readme.io/docs/sqlline
* Run {{KeyValueDataProcessing}} to catch the exception below
{noformat}
Exception in thread "main" class
org.apache.ignite.binary.BinaryObjectException: Binary type has different
affinity key fields [typeName=demo.model.CityKey, affKeyFieldName1=COUNTRYCODE,
affKeyFieldName2=countryCode]
at
org.apache.ignite.internal.binary.BinaryUtils.mergeMetadata(BinaryUtils.java:987)
{noformat}
If fact {{CityKey}} names the affinity field in the same way as in CREATE TABLE
- {{countryCode}}.
Next, run {{KeyValueBinaryDataProcessing}} to spot another weird thing:
* BinaryObject key accepts `countryCode` as the affinity field name.
* If to print our a binary object value then all the fields are in the
upper-case (they were not defined this way in CREATE TABLE):
{noformat}
demo.model.City [idHash=1613627715, hash=-1386587499, DISTRICT=Noord-Holland,
POPULATION=711200, NAME=Amsterdam]
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6784) Document cluster activation via REST protocol
Denis Magda created IGNITE-6784:
---
Summary: Document cluster activation via REST protocol
Key: IGNITE-6784
URL: https://issues.apache.org/jira/browse/IGNITE-6784
Project: Ignite
Issue Type: Task
Security Level: Public (Viewable by anyone)
Components: documentation
Affects Versions: 2.3
Reporter: Denis Magda
Assignee: Prachi Garg
Fix For: 2.3
Starting Ignite 2.3 cluster can be activated with REST protocol. The following
commands are supported:
- {{activate}} - actives the cluster.
- {{deactivate}} - deactivates the cluster.
- {{currentstate}} - checks current cluster state.
Update both pages below:
https://apacheignite.readme.io/docs/rest-api
https://apacheignite.readme.io/v2.2/docs/cluster-activation
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6717) Hashcode/equals is not needed for custom keys serialized into a binary form
Denis Magda created IGNITE-6717: --- Summary: Hashcode/equals is not needed for custom keys serialized into a binary form Key: IGNITE-6717 URL: https://issues.apache.org/jira/browse/IGNITE-6717 Project: Ignite Issue Type: Task Security Level: Public (Viewable by anyone) Reporter: Denis Magda Assignee: Denis Magda Fix For: 2.3 It turns out that hashCode/equals implementation is no longer required for custom complex keys if a key is serialized to the binary form. Discussed here: http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2-3-troubles-with-key-value-APIs-in-the-cluster-configured-with-DDL-td23501.html#a23506 The marshaller calculates the hash code automatically. This info has to be added to the binary marshaller page: https://apacheignite.readme.io/docs/binary-marshaller -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6687) SQL Error Codes Documentation
Denis Magda created IGNITE-6687: --- Summary: SQL Error Codes Documentation Key: IGNITE-6687 URL: https://issues.apache.org/jira/browse/IGNITE-6687 Project: Ignite Issue Type: Task Security Level: Public (Viewable by anyone) Reporter: Denis Magda Assignee: Vladimir Ozerov Fix For: 2.3 Presently we have JDBC error codes documented: https://apacheignite-sql.readme.io/docs/jdbc-error-codes Two things have to be done: * If there are similar error codes shared between the ODBC and JDBC drivers, and native APIs then let's put them under SQL reference category below Data Types: https://apacheignite-sql.readme.io/docs/sql-reference-overview * ODBC specific error codes with an example of an error handling need to be listed in ODBC category [~isapego], please assist with this: https://apacheignite-sql.readme.io/docs/odbc-driver Discussion on the dev list: http://apache-ignite-developers.2346864.n4.nabble.com/SQL-error-codes-td23415.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6686) Document SQL Data Types
Denis Magda created IGNITE-6686: --- Summary: Document SQL Data Types Key: IGNITE-6686 URL: https://issues.apache.org/jira/browse/IGNITE-6686 Project: Ignite Issue Type: Task Security Level: Public (Viewable by anyone) Reporter: Denis Magda Assignee: Vladimir Ozerov The data types page is empty for now: https://apacheignite-sql.readme.io/docs/data-types It should include references to language or driver specific pages like the one we have for ODBC: https://apacheignite-sql.readme.io/docs/odbc-data-types Guys please help with the following (create a separate page like we have for ODBC): # [~ptupitsyn] - .NET data types. # [~isapego] - C++ data type. # [~al.psc] - JDBC and JAVA data types. [~vozerov], please coordinate the activity. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6683) Move WAL and Checkpointing sections to separate pages
Denis Magda created IGNITE-6683: --- Summary: Move WAL and Checkpointing sections to separate pages Key: IGNITE-6683 URL: https://issues.apache.org/jira/browse/IGNITE-6683 Project: Ignite Issue Type: Bug Security Level: Public (Viewable by anyone) Reporter: Denis Magda Assignee: Prachi Garg Priority: Critical Fix For: 2.3 Please move WAL and checkpointing sections from this page [1] to dedicated ones. Page [1] should mention and refer to the new pages. Make the new pages invisible until 2.3 release. [1] https://dash.readme.io/project/apacheignite/v2.2/docs/ignite-persistence-23 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6682) Document manual cluster activation feature
Denis Magda created IGNITE-6682:
---
Summary: Document manual cluster activation feature
Key: IGNITE-6682
URL: https://issues.apache.org/jira/browse/IGNITE-6682
Project: Ignite
Issue Type: Bug
Security Level: Public (Viewable by anyone)
Reporter: Denis Magda
Assignee: Prachi Garg
Once Ignite persistence is enabled, the cluster has to be activated manually
before any operation can be processed by it. That's needed to ensure that all
the nodes that have data on disk have joined the cluster after a restart and
that the topology is considered to be stable.
Some details can be found in this discussion:
* Usage section of this doc [1]. Look for a paragraph that starts with _"Next,
when Ignite sees that the persistence is enabled it moves the cluster from
active to inactive state making sure that applications cannot modify the data
until allowed."_
*
http://apache-ignite-developers.2346864.n4.nabble.com/Control-sh-script-and-cluster-activation-td20821.html
Document the manual cluster activation on a dedicated page [2] at least
considering the following:
* Overview. Why it's required? Move the paragraph from [1] and expand if
required.
* Show an exception that happens if you try to execute a SQL or key-value query
or run a computation against an activated cluster.
* Explain how to activate the cluster with:
* From command line with control.sh script.
* From web console.
* From source code with {{Ignite.activate()}} method.
* Add a callout that the automatic cluster activation is to be added in the
future. You may refer to this discussion [3].
[1] https://apacheignite.readme.io/v2.2/docs/ignite-persistence-23#section-usage
[2] https://apacheignite.readme.io/v2.2/docs/cluster-activation
[3]
http://apache-ignite-developers.2346864.n4.nabble.com/Cluster-auto-activation-design-proposal-td20295.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6656) SQLLine Documentation
Denis Magda created IGNITE-6656:
---
Summary: SQLLine Documentation
Key: IGNITE-6656
URL: https://issues.apache.org/jira/browse/IGNITE-6656
Project: Ignite
Issue Type: Task
Security Level: Public (Viewable by anyone)
Reporter: Denis Magda
Assignee: Prachi Garg
Priority: Blocker
SQLLine is officially used as a default command line tool for SQL connectivity
in Ignite. Document the tool usage on a dedicated page:
https://apacheignite-sql.readme.io/docs/sqlline
Consider the following sections:
# The tool overview with a list of commands that are supported and not by
Ignite. The current list is here [1]. Split the list into 2 parts on readme.io
(supported and not) and briefly describe eache command.
# Connectivity section. Start a cluster and connect to the tool using
{{./ignitesql.sh --color=true --verbose=true -u jdbc:ignite:thin://127.0.0.1/}}
script. Add a tab for ignite.bat file as well.
# Usage section. Execute DDL and DML commands taken from the getting started as
it's done here [2]. When tables and indexes are created, run {{!metadata}} to
show the structure.
Use private 2.3 builds generated by TC to make sure the doc is ready for the
release.
[1] https://cwiki.apache.org/confluence/display/IGNITE/Overview+sqlline+tool
[2] https://apacheignite-sql.readme.io/v2.1/docs/sql-tooling
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6645) Security issues in Ignite that allows users with write access to datagrid to execute arbitrary code
Denis Magda created IGNITE-6645: --- Summary: Security issues in Ignite that allows users with write access to datagrid to execute arbitrary code Key: IGNITE-6645 URL: https://issues.apache.org/jira/browse/IGNITE-6645 Project: Ignite Issue Type: Bug Security Level: Public (Viewable by anyone) Affects Versions: 1.0 Reporter: Denis Magda Assignee: Yakov Zhdanov Priority: Critical Fix For: 2.4 The security breach was reported by an end-user: https://mail-search.apache.org/pmc/private-arch/ignite-private/201710.mbox/%3c7099cd44-92a7-4254-89c5-d8270b5a6...@apache.org%3e Details shared by the user: I would like to report some security issues that we found using the query language QL from lgtm.com. These are unsafe deserialization issues that allow users, possibly remote, that have rights to put entities on the datagrid to execute arbitrary code on an ignite server node. As there are more than one of these issues, I will send them to you in separate emails. The first issue affects the socket streaming server. The PoC code are included and are modifications of the `wordcount.socket` example in the examples package. A bit of set up is needed to see the full effect of code execution, so I will not include the details here, but if you want to try it out yourself, then please let me know and I can include the full PoC. First add commons-beantil to the dependency, any version will work. Then download the file `obj`, which contains the serialized data of a malicious object. Change line 44 in `SocketStreamClient` so that it opens this file. First start a server node using the example config `config/example-ignite.xml`, then start up the streaming server `SocketStreamerServer`. Now when you run `SocketStreamClient`, you will get an error, but somewhere in the stacktrace on the log in `SocketStreamerServer`, you will see this: Caused by: java.lang.RuntimeException: InvocationTargetException: java.lang.reflect.InvocationTargetException at org.apache.commons.beanutils.BeanComparator.compare(BeanComparator.java:171) at java.util.PriorityQueue.siftDownUsingComparator(PriorityQueue.java:721) at java.util.PriorityQueue.siftDown(PriorityQueue.java:687) at java.util.PriorityQueue.heapify(PriorityQueue.java:736) at java.util.PriorityQueue.readObject(PriorityQueue.java:795) This shows that the node running the `SocketStreamerServer` is deserializing the payload object that I send it. When properly set up, an attacker will have a remote ldap server that contains a second malicious Java object. Then when the above deserialization happens, an ldap look up will cause the second malicious object to be instantiated, which can then be used to execute arbitrary code. Also, although this exploit relies on having commons-beanutils to be on the classpath, there are other exploits that will work for different third party libraries, so it is not so much of a problem in commons-beanutils, but an issue in the handling of deserialization in ignite. These results are using a slightly more ahead version of the QL library with we haven't made available on lgtm yet, but should be in a few days, if you are interested, I can share a link to the query and results to you when it is ready. Thanks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
