Repository: cassandra Updated Branches: refs/heads/trunk a7926b339 -> 94df2a9c8
Add Development section to online docs Patch by Stefan Podkowinski; reviewed by Tyler Hobbs for CASSANDRA-12449 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/94df2a9c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/94df2a9c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/94df2a9c Branch: refs/heads/trunk Commit: 94df2a9c81a58f820141fbd6fe69231d916da868 Parents: a7926b3 Author: Stefan Podkowinski <s.podkowin...@gmail.com> Authored: Tue Aug 23 14:06:42 2016 -0500 Committer: Tyler Hobbs <tylerlho...@gmail.com> Committed: Tue Aug 23 14:06:42 2016 -0500 ---------------------------------------------------------------------- doc/source/_templates/indexcontent.html | 10 +- doc/source/bugs.rst | 7 +- doc/source/development/code_style.rst | 94 +++++++++++ doc/source/development/how_to_commit.rst | 75 +++++++++ doc/source/development/how_to_review.rst | 71 +++++++++ doc/source/development/ide.rst | 157 +++++++++++++++++++ .../development/images/eclipse_debug0.png | 0 .../development/images/eclipse_debug1.png | 0 .../development/images/eclipse_debug2.png | 0 .../development/images/eclipse_debug3.png | 0 .../development/images/eclipse_debug4.png | 0 .../development/images/eclipse_debug5.png | 0 .../development/images/eclipse_debug6.png | 0 doc/source/development/index.rst | 28 ++++ doc/source/development/patches.rst | 125 +++++++++++++++ doc/source/development/testing.rst | 86 ++++++++++ doc/source/index.rst | 1 + 17 files changed, 646 insertions(+), 8 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/_templates/indexcontent.html ---------------------------------------------------------------------- diff --git a/doc/source/_templates/indexcontent.html b/doc/source/_templates/indexcontent.html index 60dd454..5d4b485 100644 --- a/doc/source/_templates/indexcontent.html +++ b/doc/source/_templates/indexcontent.html @@ -42,16 +42,18 @@ <span class="linkdescr">{% trans %}CQL reference documentation{% endtrans %}</span></p> </td> <td class="right-column"> - <p class="biglink"><a class="biglink" href="{{ pathto("faq/index") }}">{% trans %}FAQs{% endtrans %}</a><br/> - <span class="linkdescr">{% trans %}Frequently Asked Questions (with answers!){% endtrans %}</span></p> + <p class="biglink"><a class="biglink" href="{{ pathto("development/index") }}">{% trans %}Cassandra Development{% endtrans %}</a><br/> + <span class="linkdescr">{% trans %}Learn how to improve Cassandra and contribute patches{% endtrans %}</span></p> </td> </tr> <tr> <td class="left-column"> - <p class="biglink"><a class="biglink" href="{{ pathto("configuration/index") }}">{% trans %}Configuration{% endtrans %}</a><br/> - <span class="linkdescr">{% trans %}Cassandra's handles and knobs{% endtrans %}</span></p> + <p class="biglink"><a class="biglink" href="{{ pathto("faq/index") }}">{% trans %}FAQs{% endtrans %}</a><br/> + <span class="linkdescr">{% trans %}Frequently Asked Questions (with answers!){% endtrans %}</span></p> </td> <td class="right-column"> + <p class="biglink"><a class="biglink" href="{{ pathto("configuration/index") }}">{% trans %}Configuration{% endtrans %}</a><br/> + <span class="linkdescr">{% trans %}Cassandra's handles and knobs{% endtrans %}</span></p> </td> </tr> </table> http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/bugs.rst ---------------------------------------------------------------------- diff --git a/doc/source/bugs.rst b/doc/source/bugs.rst index 90efb14..240cfd4 100644 --- a/doc/source/bugs.rst +++ b/doc/source/bugs.rst @@ -14,7 +14,7 @@ .. See the License for the specific language governing permissions and .. limitations under the License. -Reporting bugs and contributing +Reporting Bugs and Contributing =============================== If you encounter a problem with Cassandra, the first places to ask for help are the :ref:`user mailing list @@ -25,7 +25,6 @@ ticket through the `Apache Cassandra JIRA <https://issues.apache.org/jira/browse details as you can on your problem, and don't forget to indicate which version of Cassandra you are running and on which environment. -If you would like to contribute, please check `the section on contributing -<https://wiki.apache.org/cassandra/HowToContribute>`__ on the Cassandra wiki. Please note that the source of this -documentation is part of the Cassandra git repository and hence contributions to the documentation should follow the +Further details on how to contribute can be found at our :doc:`development/index` section. Please note that the source of +this documentation is part of the Cassandra git repository and hence contributions to the documentation should follow the same path. http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/code_style.rst ---------------------------------------------------------------------- diff --git a/doc/source/development/code_style.rst b/doc/source/development/code_style.rst new file mode 100644 index 0000000..5a486a4 --- /dev/null +++ b/doc/source/development/code_style.rst @@ -0,0 +1,94 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +.. highlight:: none + +Code Style +========== + +General Code Conventions +------------------------ + + - The Cassandra project follows `Sun's Java coding conventions <http://java.sun.com/docs/codeconv/html/CodeConvTOC.doc.html>`_ with an important exception: ``{`` and ``}`` are always placed on a new line + +Exception handling +------------------ + + - Never ever write ``catch (...) {}`` or ``catch (...) { logger.error() }`` merely to satisfy Java's compile-time exception checking. Always propagate the exception up or throw ``RuntimeException`` (or, if it "can't happen," ``AssertionError``). This makes the exceptions visible to automated tests. + - Avoid propagating up checked exceptions that no caller handles. Rethrow as ``RuntimeException`` (or ``IOError``, if that is more applicable). + - Similarly, logger.warn() is often a cop-out: is this an error or not? If it is don't hide it behind a warn; if it isn't, no need for the warning. + - If you genuinely know an exception indicates an expected condition, it's okay to ignore it BUT this must be explicitly explained in a comment. + +Boilerplate +----------- + + - Avoid redundant ``@Override`` annotations when implementing abstract or interface methods. + - Do not implement equals or hashcode methods unless they are actually needed. + - Prefer public final fields to private fields with getters. (But prefer encapsulating behavior in "real" methods to either.) + - Prefer requiring initialization in the constructor to setters. + - Avoid redundant ``this`` references to member fields or methods. + - Do not extract interfaces (or abstract classes) unless you actually need multiple implementations of it. + - Always include braces for nested levels of conditionals and loops. Only avoid braces for single level. + +Multiline statements +-------------------- + + - Try to keep lines under 120 characters, but use good judgement -- it's better to exceed 120 by a little, than split a line that has no natural splitting points. + - When splitting inside a method call, use one line per parameter and align them, like this: + + :: + + SSTableWriter writer = new SSTableWriter(cfs.getTempSSTablePath(), + columnFamilies.size(), + StorageService.getPartitioner()); + + - When splitting a ternary, use one line per clause, carry the operator, and align like this: + + :: + + var = bar == null + ? doFoo() + : doBar(); + +Whitespace +---------- + + - Please make sure to use 4 spaces instead of the tab character for all your indentation. + - Many lines in many files have a bunch of trailing whitespace... Please either clean these up in a separate patch, or leave them alone, so that reviewers now and anyone reading code history later doesn't have to pay attention to whitespace diffs. + +Imports +------- + +Please observe the following order for your imports:: + + java + [blank line] + com.google.common + org.apache.commons + org.junit + org.slf4j + [blank line] + everything else alphabetically + +Format files for IDEs +--------------------- + + - IntelliJ: `intellij-codestyle.jar <https://wiki.apache.org/cassandra/CodeStyle?action=AttachFile&do=view&target=intellij-codestyle.jar>`_ + - IntelliJ 13: `gist for IntelliJ 13 <https://gist.github.com/jdsumsion/9ab750a05c2a567c6afc>`_ (this is a work in progress, still working on javadoc, ternary style, line continuations, etc) + - Eclipse (https://github.com/tjake/cassandra-style-eclipse) + + + http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/how_to_commit.rst ---------------------------------------------------------------------- diff --git a/doc/source/development/how_to_commit.rst b/doc/source/development/how_to_commit.rst new file mode 100644 index 0000000..d956c72 --- /dev/null +++ b/doc/source/development/how_to_commit.rst @@ -0,0 +1,75 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +.. highlight:: none + +How-to Commit +============= + +If you are a committer, feel free to pick any process that works for you - so long as you are planning to commit the work yourself. + +Here is how committing and merging will usually look for merging and pushing for tickets that follow the convention (if patch-based): + +Hypothetical CASSANDRA-12345 ticket is a cassandra-3.0 based bug fix that requires different code for cassandra-3.3, and trunk. Contributor Jackie supplied a patch for the root branch (12345-3.0.patch), and patches for the remaining branches (12345-3.3.patch, 12345-trunk.patch). + +On cassandra-3.0: + #. ``git am -3 12345-3.0.patch`` (if we have a problem b/c of CHANGES.txt not merging anymore, we fix it ourselves, in place) + +On cassandra-3.3: + #. ``git merge cassandra-3.0 -s ours`` + #. ``git apply -3 12345-3.3.patch`` (likely to have an issue with CHANGES.txt here: fix it ourselves, then git add CHANGES.txt) + #. ``git commit âamend`` + +On trunk: + #. ``git merge cassandra-3.3 -s ours`` + #. ``git apply -3 12345-trunk.patch`` (likely to have an issue with CHANGES.txt here: fix it ourselves, then git add CHANGES.txt) + #. ``git commit âamend`` + +On any branch: + #. ``git push origin cassandra-3.0 cassandra-3.3 trunk âatomic`` + +Same scenario, but a branch-based contribution: + +On cassandra-3.0: + #. ``git cherry-pick <sha-of-3.0-commit>`` (if we have a problem b/c of CHANGES.txt not merging anymore, we fix it ourselves, in place) + +On cassandra-3.3: + #. ``git merge cassandra-3.0 -s ours`` + #. ``git format-patch -1 <sha-of-3.3-commit>`` + #. ``git apply -3 <sha-of-3.3-commit>.patch`` (likely to have an issue with CHANGES.txt here: fix it ourselves, then git add CHANGES.txt) + #. ``git commit âamend`` + +On trunk: + #. ``git merge cassandra-3.3 -s ours`` + #. ``git format-patch -1 <sha-of-trunk-commit>`` + #. ``git apply -3 <sha-of-trunk-commit>.patch`` (likely to have an issue with CHANGES.txt here: fix it ourselves, then git add CHANGES.txt) + #. ``git commit âamend`` + +On any branch: + #. ``git push origin cassandra-3.0 cassandra-3.3 trunk âatomic`` + +.. tip:: + + Notes on git flags: + ``-3`` flag to am and apply will instruct git to perform a 3-way merge for you. If a conflict is detected, you can either resolve it manually or invoke git mergetool - for both am and apply. + + ``âatomic`` flag to git push does the obvious thing: pushes all or nothing. Without the flag, the command is equivalent to running git push once per each branch. This is nifty in case a race condition happens - you wonât push half the branches, blocking other committersâ progress while you are resolving the issue. + +.. tip:: + + The fastest way to get a patch from someoneâs commit in a branch on GH - if you donât have their repo in remotes - is to append .patch to the commit url, e.g. + curl -O https://github.com/apache/cassandra/commit/7374e9b5ab08c1f1e612bf72293ea14c959b0c3c.patch + http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/how_to_review.rst ---------------------------------------------------------------------- diff --git a/doc/source/development/how_to_review.rst b/doc/source/development/how_to_review.rst new file mode 100644 index 0000000..dc97743 --- /dev/null +++ b/doc/source/development/how_to_review.rst @@ -0,0 +1,71 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +Review Checklist +**************** + +When reviewing tickets in Apache JIRA, the following items should be covered as part of the review process: + +**General** + + * Does it conform to the :doc:`code_style` guidelines? + * Is there any redundant or duplicate code? + * Is the code as modular as possible? + * Can any singletons be avoided? + * Can any of the code be replaced with library functions? + * Are units of measurement used in the code consistent, both internally and with the rest of the ecosystem? + +**Error-Handling** + + * Are all data inputs and outputs checked (for the correct type, length, format, and range) and encoded? + * Where third-party utilities are used, are returning errors being caught? + * Are invalid parameter values handled? + * Are any Throwable/Exceptions passed to the JVMStabilityInspector? + * Are errors well-documented? Does the error message tell the user how to proceed? + * Do exceptions propagate to the appropriate level in the code? + +**Documentation** + + * Do comments exist and describe the intent of the code (the "why", not the "how")? + * Are javadocs added where appropriate? + * Is any unusual behavior or edge-case handling described? + * Are data structures and units of measurement explained? + * Is there any incomplete code? If so, should it be removed or flagged with a suitable marker like âTODOâ? + * Does the code self-document via clear naming, abstractions, and flow control? + * Have NEWS.txt, the cql3 docs, and the native protocol spec been updated if needed? + * Is the ticket tagged with "client-impacting" and "doc-impacting", where appropriate? + * Has lib/licences been updated for third-party libs? Are they Apache License compatible? + * Is the Component on the JIRA ticket set appropriately? + +**Testing** + + * Is the code testable? i.e. donât add too many or hide dependencies, unable to initialize objects, test frameworks can use methods etc. + * Do tests exist and are they comprehensive? + * Do unit tests actually test that the code is performing the intended functionality? + * Could any test code use common functionality (e.g. ccm, dtest, or CqlTester methods) or abstract it there for reuse? + * If the code may be affected by multi-node clusters, are there dtests? + * If the code may take a long time to test properly, are there CVH tests? + * Is the test passing on CI for all affected branches (up to trunk, if applicable)? Are there any regressions? + * If patch affects read/write path, did we test for performance regressions w/multiple workloads? + * If adding a new feature, were tests added and performed confirming it meets the expected SLA/use-case requirements for the feature? + +**Logging** + + * Are logging statements logged at the correct level? + * Are there logs in the critical path that could affect performance? + * Is there any log that could be added to communicate status or troubleshoot potential problems in this feature? + * Can any unnecessary logging statement be removed? + http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/ide.rst ---------------------------------------------------------------------- diff --git a/doc/source/development/ide.rst b/doc/source/development/ide.rst new file mode 100644 index 0000000..c52c11a --- /dev/null +++ b/doc/source/development/ide.rst @@ -0,0 +1,157 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +Building and IDE Integration +**************************** + +Building From Source +==================== + +Getting started with Cassandra and IntelliJ IDEA or Eclipse is simple, once you manage to build Cassandra from source using `Java 8 <http://www.oracle.com/technetwork/java/javase/downloads/index.html>`_, `Git <https://git-scm.com/>`_ and `Ant <http://ant.apache.org/>`_. + +The source code for Cassandra is shared through the central Apache Git repository and organized by different branches. You can access the code for the current development branch through git as follows:: + + git clone http://git-wip-us.apache.org/repos/asf/cassandra.git cassandra-trunk + +Other branches will point to different versions of Cassandra. Switching to a different branch requires checking out the branch by its name:: + + git checkout cassandra-3.0 + +You can get a list of available branches with ``git branch``. + +Finally build Cassandra using ant:: + + ant + +This may take a significant amount of time depending on whether artifacts have to be downloaded and the number of classes that need to be compiled. + +.. hint:: + + You can setup multiple working trees for different Cassandra versions from the same repository using `git-worktree <https://git-scm.com/docs/git-worktree>`_. + +Setting up Cassandra in IntelliJ IDEA +===================================== + +`IntelliJ IDEA <https://www.jetbrains.com/idea/>`_ by JetBrains is one of the most popular IDEs for Cassandra and Java development in general. The Community Edition is provided as a free download with all features needed to get started developing Cassandra. + +Setup Cassandra as a Project (C* 2.1 and newer) +----------------------------------------------- + +Since 2.1.5, there is a new ant target: ``generate-idea-files``. Please see our `wiki <https://wiki.apache.org/cassandra/RunningCassandraInIDEA>`_ for instructions for older Cassandra versions. + +Please clone and build Cassandra as described above and execute the following steps: + +1. Once Cassandra is built, generate the IDEA files using ant: + +:: + + ant generate-idea-files + +2. Start IDEA + +3. Open the IDEA project from the checked out Cassandra directory using the menu item Open in IDEA's File menu + +The project generated by the ant task ``generate-idea-files`` contains nearly everything you need to debug Cassandra and execute unit tests. + + * Run/debug defaults for JUnit + * Run/debug configuration for Cassandra daemon + * License header for Java source files + * Cassandra code style + * Inspections + +Setting up Cassandra in Eclipse +=============================== + +Eclipse is a popular open source IDE that can be used for Cassandra development. Various Eclipse environments are available from the `download page <https://www.eclipse.org/downloads/eclipse-packages/>`_. The following guide was created with "Eclipse IDE for Java Developers". + +These instructions were tested on Ubuntu 16.04 with Eclipse Neon (4.6) using Cassandra 2.1, 2.2 and 3.x. + +Project Settings +---------------- + +**It is important that you generate the Eclipse files with Ant before trying to set up the Eclipse project.** + + * Clone and build Cassandra as described above. + * Run ``ant generate-eclipse-files`` to create the Eclipse settings. + * Start Eclipse. + * Select ``File->Import->Existing Projects into Workspace->Select git directory``. + * Make sure "cassandra-trunk" is recognized and selected as a project (assuming you checked the code out into the folder cassandra-trunk as described above). + * Confirm "Finish" to have your project imported. + +You should now be able to find the project as part of the "Package Explorer" or "Project Explorer" without having Eclipse complain about any errors after building the project automatically. + +Unit Tests +---------- + +Unit tests can be run from Eclipse by simply right-clicking the class file or method and selecting ``Run As->JUnit Test``. Tests can be debugged this way as well by defining breakpoints (double-click line number) and selecting ``Debug As->JUnit Test``. + +Alternatively all unit tests can be run from the command line as described in :doc:`testing` + +Debugging Cassandra Using Eclipse +--------------------------------- + +There are two ways how to start and debug a local Cassandra instance with Eclipse. You can either start Cassandra just as you normally would by using the ``./bin/cassandra`` script and connect to the JVM through `remotely <https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/introclientissues005.html>`_ from Eclipse or start Cassandra from Eclipse right away. + +Starting Cassandra From Command Line +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + * Set environment variable to define remote debugging options for the JVM: + ``export JVM_EXTRA_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=1414"`` + * Start Cassandra by executing the ``./bin/cassandra`` + +Afterwards you should be able to connect to the running Cassandra process through the following steps: + +From the menu, select ``Run->Debug Configurations..`` + +.. image:: images/eclipse_debug0.png + +Create new remote application + +.. image:: images/eclipse_debug1.png + +Configure connection settings by specifying a name and port 1414 + +.. image:: images/eclipse_debug2.png + +Afterwards confirm "Debug" to connect to the JVM and start debugging Cassandra! + +Starting Cassandra From Eclipse +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Cassandra can also be started directly from Eclipse if you don't want to use the command line. + +From the menu, select ``Run->Run Configurations..`` + +.. image:: images/eclipse_debug3.png + +Create new application + +.. image:: images/eclipse_debug4.png + +Specify name, project and main class ``org.apache.cassandra.service.CassandraDaemon`` + +.. image:: images/eclipse_debug5.png + +Configure additional JVM specific parameters that will start Cassandra with some of the settings created by the regular startup script. Change heap related values as needed. + +:: + + -Xms1024M -Xmx1024M -Xmn220M -Xss256k -ea -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+UseCondCardMark -javaagent:./lib/jamm-0.3.0.jar -Djava.net.preferIPv4Stack=true + +.. image:: images/eclipse_debug6.png + +Now just confirm "Debug" and you should see the output of Cassandra starting up in the Eclipse console and should be able to set breakpoints and start debugging! + http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/images/eclipse_debug0.png ---------------------------------------------------------------------- diff --git a/doc/source/development/images/eclipse_debug0.png b/doc/source/development/images/eclipse_debug0.png new file mode 100644 index 0000000..e69de29 http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/images/eclipse_debug1.png ---------------------------------------------------------------------- diff --git a/doc/source/development/images/eclipse_debug1.png b/doc/source/development/images/eclipse_debug1.png new file mode 100644 index 0000000..e69de29 http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/images/eclipse_debug2.png ---------------------------------------------------------------------- diff --git a/doc/source/development/images/eclipse_debug2.png b/doc/source/development/images/eclipse_debug2.png new file mode 100644 index 0000000..e69de29 http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/images/eclipse_debug3.png ---------------------------------------------------------------------- diff --git a/doc/source/development/images/eclipse_debug3.png b/doc/source/development/images/eclipse_debug3.png new file mode 100644 index 0000000..e69de29 http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/images/eclipse_debug4.png ---------------------------------------------------------------------- diff --git a/doc/source/development/images/eclipse_debug4.png b/doc/source/development/images/eclipse_debug4.png new file mode 100644 index 0000000..e69de29 http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/images/eclipse_debug5.png ---------------------------------------------------------------------- diff --git a/doc/source/development/images/eclipse_debug5.png b/doc/source/development/images/eclipse_debug5.png new file mode 100644 index 0000000..e69de29 http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/images/eclipse_debug6.png ---------------------------------------------------------------------- diff --git a/doc/source/development/images/eclipse_debug6.png b/doc/source/development/images/eclipse_debug6.png new file mode 100644 index 0000000..e69de29 http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/index.rst ---------------------------------------------------------------------- diff --git a/doc/source/development/index.rst b/doc/source/development/index.rst new file mode 100644 index 0000000..aefc599 --- /dev/null +++ b/doc/source/development/index.rst @@ -0,0 +1,28 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +Cassandra Development +********************* + +.. toctree:: + :maxdepth: 2 + + ide + testing + patches + code_style + how_to_review + how_to_commit http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/patches.rst ---------------------------------------------------------------------- diff --git a/doc/source/development/patches.rst b/doc/source/development/patches.rst new file mode 100644 index 0000000..e3d968f --- /dev/null +++ b/doc/source/development/patches.rst @@ -0,0 +1,125 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +.. highlight:: none + +Contributing Code Changes +************************* + +Choosing What to Work on +======================== + +Submitted patches can include bug fixes, changes to the Java code base, improvements for tooling (both Java or Python), documentation, testing or any other changes that requires changing the code base. Although the process of contributing code is always the same, the amount of work and time it takes to get a patch accepted also depends on the kind of issue you're addressing. + +As a general rule of thumb: + * Major new features and significant changes to the code based will likely not going to be accepted without deeper discussion within the `developer community <http://cassandra.apache.org/community/>`_ + * Bug fixes take higher priority compared to features + * The extend to which tests are required depend on how likely your changes will effect the stability of Cassandra in production. Tooling changes requires fewer tests than storage engine changes. + * Less complex patches will be faster to review: consider breaking up an issue into individual tasks and contributions that can be reviewed separately + +.. hint:: + + Not sure what to work? Just pick an issue tagged with the `low hanging fruit label <https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+=+12310865+AND+labels+=+lhf+AND+status+!=+resolved>`_ in JIRA, which we use to flag issues that could turn out to be good starter tasks for beginners. + +Before You Start Coding +======================= + +Although contributions are highly appreciated, we do not guarantee that each contribution will become a part of Cassandra. Therefor it's generally a good idea to first get some feedback on the things you plan to work on, especially about any new features or major changes to the code base. You can reach out to other developers on the mailing list or IRC channel listed on our `community page <http://cassandra.apache.org/community/>`_. + +You should also + * Avoid redundant work by searching for already reported issues in `JIRA <https://issues.apache.org/jira/browse/CASSANDRA>`_ + * Create a new issue early in the process describing what you're working on - not just after finishing your patch + * Link related JIRA issues with your own ticket to provide a better context + * Update your ticket from time to time by giving feedback on your progress and link a GitHub WIP branch with your current code + * Ping people who you actively like to ask for advice on JIRA by `mentioning users <https://confluence.atlassian.com/conf54/confluence-user-s-guide/sharing-content/using-mentions>`_ + +There are also some fixed rules that you need to be aware: + * Patches will only be applied to branches by following the release model + * Code must be testable + * Code must follow the :doc:`code_style` convention + * Changes must not break compatibility between different Cassandra versions + * Contributions must be covered by the Apache License + +Choosing the Right Branches to Work on +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There are currently multiple Cassandra versions maintained in individual branches: + +======= ====== +Version Policy +======= ====== +3.x Tick-tock (see below) +3.0 Bug fixes only +2.2 Bug fixes only +2.1 Critical bug fixes only +======= ====== + +Corresponding branches in git are easy to recognize as they are named ``cassandra-<release>`` (e.g. ``cassandra-3.0``). The ``trunk`` branch is an exception, as it contains the most recent commits from all other branches and is used for creating new branches for future tick-tock releases. + +Tick-Tock Releases +"""""""""""""""""" + +New releases created as part of the `tick-tock release process <http://www.planetcassandra.org/blog/cassandra-2-2-3-0-and-beyond/>`_ will either focus on stability (odd version numbers) or introduce new features (even version numbers). Any code for new Cassandra features you should be based on the latest, unreleased 3.x branch with even version number or based on trunk. + +Bug Fixes +""""""""" + +Creating patches for bug fixes is a bit more complicated as this will depend on how many different versions of Cassandra are affected. In each case, the order for merging such changes will be ``cassandra-2.1`` -> ``cassandra-2.2`` -> ``cassandra-3.0`` -> ``cassandra-3.x`` -> ``trunk``. But don't worry, merging from 2.1 would be the worst case for bugs that affect all currently supported versions, which isn't very common. As a contributor, you're also not expected to provide a single patch for each version. What you need to do however is: + + * Be clear about which versions you could verify to be affected by the bug + * For 2.x: ask if a bug qualifies to be fixed in this release line, as this may be handled on case by case bases + * If possible, create a patch against the lowest version in the branches listed above (e.g. if you found the bug in 3.9 you should try to fix it already in 3.0) + * Test if the patch can be merged cleanly across branches in the direction listed above + * Be clear which branches may need attention by the committer or even create custom patches for those if you can + +Creating a Patch +================ + +So you've finished coding and the great moment arrives: it's time to submit your patch! + + 1. Create a branch for your changes if you haven't done already. Many contributors name their branches based on ticket number and Cassandra version, e.g. ``git checkout -b 12345-3.0`` + 2. Verify that you follow Cassandra's :doc:`code_style` + 3. Make sure all tests (including yours) pass using ant as described in :doc:`testing`. If you suspect a test failure is unrelated to your change, it may be useful to check the test's status by searching the issue tracker or looking at `CI <https://cassci.datastax.com/>`_ results for the relevant upstream version. Note that the full test suites take many hours to complete, so it is common to only run specific relevant tests locally before uploading a patch. Once a patch has been uploaded, the reviewer or committer can help setup CI jobs to run the full test suites. + 4. Consider going through the :doc:`how_to_review` for your code. This will help you to understand how others will consider your change for inclusion. + 5. Donât make the committer squash commits for you in the root branch either. Multiple commits are fine - and often preferable - during review stage, especially for incremental review, but once +1d, do either: + + a. Attach a patch to JIRA with a single squashed commit in it (per branch), or + b. Squash the commits in-place in your branches into one + + 6. Include a CHANGES.txt entry (put it at the top of the list), and format the commit message appropriately in your patch ending with the following statement on the last line: ``patch by X; reviewed by Y for CASSANDRA-ZZZZZ`` + 7. When you're happy with the result, create a patch: + + :: + + git add <any new or modified file> + git commit -m '<message>' + git format-patch HEAD~1 + mv <patch-file> <ticket-branchname.txt> (e.g. 12345-trunk.txt, 12345-3.0.txt) + + Alternatively, many contributors prefer to make their branch available on GitHub. In this case, fork the Cassandra repository on GitHub and push your branch: + + :: + + git push --set-upstream origin 12345-3.0 + + 8. To make life easier for your reviewer/committer, you may want to make sure your patch applies cleanly to later branches and create additional patches/branches for later Cassandra versions to which your original patch does not apply cleanly. That said, this is not critical, and you will receive feedback on your patch regardless. + 9. Attach the newly generated patch to the ticket/add a link to your branch and click "Submit Patch" at the top of the ticket. This will move the ticket into "Patch Available" status, indicating that your submission is ready for review. + 10. Wait for other developers or committers to review it and hopefully +1 the ticket (see :doc:`how_to_review`). If your change does not receive a +1, do not be discouraged. If possible, the reviewer will give suggestions to improve your patch or explain why it is not suitable. + 11. If the reviewer has given feedback to improve the patch, make the necessary changes and move the ticket into "Patch Available" once again. + +Once the review process is complete, you will receive a +1. Wait for a committer to commit it. Do not delete your branches immediately after theyâve been committed - keep them on GitHub for a while. Alternatively, attach a patch to JIRA for historical record. Itâs not that uncommon for a committer to mess up a merge. In case of that happening, access to the original code is required, or else youâll have to redo some of the work. + + http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/development/testing.rst ---------------------------------------------------------------------- diff --git a/doc/source/development/testing.rst b/doc/source/development/testing.rst new file mode 100644 index 0000000..eac336a --- /dev/null +++ b/doc/source/development/testing.rst @@ -0,0 +1,86 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +.. highlight:: none + +Testing +******* + +Creating tests is one of the most important and also most difficult parts of developing Cassandra. There are different ways to test your code depending on what you're working on. + + +Unit Testing +============ + +The most simple way to test code in Cassandra is probably by writing a unit test. Cassandra uses JUnit as a testing framework and test cases can be found in the ``test/unit`` directory. Ideally youâd be able to create a unit test for your implementation that would exclusively cover the class you created (the unit under test). Unfortunately this is not always possible and Cassandra doesnât have a very mock friendly code base. Often youâll find yourself in a situation where you have to make use of an embedded Cassandra instance that youâll be able to interact with in your test. If you want to make use of CQL in your test, you can simply extend CQLTester and use some of the convenient helper methods such as in the following example. + +.. code-block:: java + + @Test + public void testBatchAndList() throws Throwable + { + createTable("CREATE TABLE %s (k int PRIMARY KEY, l list<int>)"); + execute("BEGIN BATCH " + + "UPDATE %1$s SET l = l +[ 1 ] WHERE k = 0; " + + "UPDATE %1$s SET l = l + [ 2 ] WHERE k = 0; " + + "UPDATE %1$s SET l = l + [ 3 ] WHERE k = 0; " + + "APPLY BATCH"); + + assertRows(execute("SELECT l FROM %s WHERE k = 0"), + row(list(1, 2, 3))); + } + +Unit tests can be run from the command line using the ``ant test`` command, ``ant test -Dtest.name=<simple_classname>`` to execute a test suite or ``ant testsome -Dtest.name=<FQCN> -Dtest.methods=<testmethod1>[,testmethod2]`` for individual tests. For example, to run all test methods in the ``org.apache.cassandra.cql3.SimpleQueryTest`` class, you would run:: + + ant test -Dtest.name=SimpleQueryTest + +To run only the ``testStaticCompactTables()`` test method from that class, you would run:: + + ant testsome -Dtest.name=org.apache.cassandra.cql3.SimpleQueryTest -Dtest.methods=testStaticCompactTables + +Long running tests +------------------ + +Test that consume a significant amount of time during execution can be found in the ``test/long`` directory and executed as a regular JUnit test or standalone program. Except for the execution time, thereâs nothing really special about them. However, ant will execute tests under ``test/long`` only when using the ``ant long-test`` target. + +DTests +====== + +One way of doing integration or system testing at larger scale is by using `dtest <https://github.com/riptano/cassandra-dtest>`_, which stands for âCassandra Distributed Testsâ. The idea is to automatically setup Cassandra clusters using various configurations and simulate certain use cases you want to test. This is done using Python scripts and ``ccmlib`` from the `ccm <https://github.com/pcmanus/ccm>`_ project. Dtests will setup clusters using this library just as you do running ad-hoc ``ccm`` commands on your local machine. Afterwards dtests will use the `Python driver <http://datastax.github.io/python-driver/installation.html>`_ to interact with the nodes, manipulate the file system, analyze logs or mess with individual nodes. + +Using dtests helps us to prevent regression bugs by continually executing tests on the `CI server <http://cassci.datastax.com/>`_ against new patches. For frequent contributors, this Jenkins is set up to build branches from their GitHub repositories. It is likely that your reviewer will use this Jenkins instance to run tests for your patch. Read more on the motivation behind the CI server `here <http://www.datastax.com/dev/blog/cassandra-testing-improvements-for-developer-convenience-and-confidence>`_. + +The best way to learn how to write dtests is probably by reading the introduction "`How to Write a Dtest <http://www.datastax.com/dev/blog/how-to-write-a-dtest>`_" and by looking at existing, recently updated tests in the project. New tests must follow certain `style conventions <https://github.com/riptano/cassandra-dtest/blob/master/CONTRIBUTING.md>`_ that are being checked before accepting contributions. In contrast to Cassandra, dtest issues and pull-requests are managed on github, therefor you should make sure to link any created dtests in your Cassandra ticket and also refer to the ticket number in your dtest PR. + +Creating a good dtest can be tough, but it should not prevent you from submitting patches! Please ask in the corresponding JIRA ticket how to write a good dtest for the patch. In most cases a reviewer or committer will able to support you, and in some cases they may offer to write a dtest for you. + +Performance Testing +=================== + +Performance tests for Cassandra are a special breed of tests that are not part of the usual patch contribution process. In fact you can contribute tons of patches to Cassandra without ever running performance tests. They are important however when working on performance improvements, as such improvements must be measurable. + +Cassandra Stress Tool +--------------------- + +TODO: `CASSANDRA-12365 <https://issues.apache.org/jira/browse/CASSANDRA-12365>`_ + +cstar_perf +---------- + +Another tool available on github is `cstar_perf <https://github.com/datastax/cstar_perf>`_ that can be used for intensive performance testing in large clusters or locally. Please refer to the project page on how to set it up and how to use it. + + + http://git-wip-us.apache.org/repos/asf/cassandra/blob/94df2a9c/doc/source/index.rst ---------------------------------------------------------------------- diff --git a/doc/source/index.rst b/doc/source/index.rst index ec27f5a..562603d 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -34,6 +34,7 @@ Contents: operating/index tools/index troubleshooting/index + development/index faq/index bugs
