[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17633533#comment-17633533
]
Ekaterina Dimitrova commented on CASSANDRA-18001:
-------------------------------------------------
A few updates on my end... trying to be a good citizen it seems my try to do
good to the HIGHRES and reduce the parallelism so that people do not spin empty
containers for the new jobs didn't work. Manual verification of the file shows
things shifted wrong. Also, I hit the issue with CircleCI where commands are
squashed again and this makes it super hard for debugging.
Last response from CircleCI team on this issue:
{color:#de350b}After syncing with our team, I want to share that:{color}
{color:#de350b}[YAML is essentially a
specification|https://urldefense.com/v3/__https://yaml.org/__;!!PbtH5S7Ebw!b1SNWAq5oUmhmKoW44AWaB-eBX5wSa2T0dD6mpaTuIO6gYotfZnnuccv4Zc0vbn4bQ7ulVg9G_6Sz_k0lEhSMIY7oL_azCRsbw$],
and its implementation will depend on the libraries any tooling uses.{color}
{color:#de350b}[As stated in the
specification|https://urldefense.com/v3/__https://yaml.org/spec/1.2.2/*chapter-6-structural-productions__;Iw!!PbtH5S7Ebw!b1SNWAq5oUmhmKoW44AWaB-eBX5wSa2T0dD6mpaTuIO6gYotfZnnuccv4Zc0vbn4bQ7ulVg9G_6Sz_k0lEhSMIY7oL_w7U4BbA$],
"The amount of indentation is a presentation detail and must not be used to
convey content information."{color}
{color:#de350b}This means that as long as a YAML expression is still data-wise
equivalent, the formatted output of a YAML expression can depend on the library
itself.{color}
{color:#de350b}As such, certain libraries can choose to "compress" the
expressions, so long as the data is still the same data-wise.{color}
{color:#de350b}Such compressions can be beneficial, since the library may see
reducing the overall filesize being an important thing.{color}
{color:#de350b}In other words, there is really no hard-guarantees to preserving
a YAML expression's format.{color}
{color:#de350b}We think, *if* you really want to ensure the {{commands}} format
are always preserved / expressed in blocks, you would thus want to
post-process.{color}
{color:#de350b}I have set up an example here using the js-yaml Node
package:{color}
{color:#de350b}[https://replit.com/@kelvintaywl/YAMLprettifier?v=1|https://urldefense.com/v3/__https://replit.com/@kelvintaywl/YAMLprettifier?v=1__;!!PbtH5S7Ebw!b1SNWAq5oUmhmKoW44AWaB-eBX5wSa2T0dD6mpaTuIO6gYotfZnnuccv4Zc0vbn4bQ7ulVg9G_6Sz_k0lEhSMIY7oL-fJ6Nr3Q$]{color}
{color:#de350b}(If you run the repl code, you should see the post-processing
expanding the "compressed" command into a block){color}
*I do not think we can/should fix the whole world in one ticket and also that
trying that hides risks.*
{color:#172b4d}On the bright side MIDRES tests and files seem good to me.{color}
{color:#172b4d}*STEPS TAKEN TO UNBLOCK 4.1 OVER THE WEEKEND:* {color}
- 4.1 and trunk finished as there are various things that differ in branches
and few different bugs we caught... After that, when we are not pressed by the
4.1 release, I will fix 3.11, 4.0 and 3.0 patches over the next week.
* I fixed now the 4.1+ patches for HIGHRES
* I finished the cqlshlib
* My testing and an old ticket for JDK 11 improvements proved that the burn
tests in 4.1 and trunk (some of them) crash even with X-Large containers. I
verified in Jenkins and Butler that all the burn tests are run and they finish
successfully. SO I think those are not a blocker, we can use Jenkins
||Patch||LOWRES||MIDRES||HIGHRES||
|[4.1|https://github.com/ekaterinadimitrova2/cassandra/tree/18001-4.1-3]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-3]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-3-mid]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-3-high]|
|[trunk|https://github.com/ekaterinadimitrova2/cassandra/tree/18001-trunk-3]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-3]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2130/workflows/9df2d073-c26a-4750-82e4-8b6645f37f9c]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-3-high2]|
_*trunk still running - I will check it tomorrow morning_
Reminder about dtests large - _There are two tests that fail because there are
not enough resources to bring up 9 nodes. Spinning a 2X Large container for all
large tests because of 2 is not justified._
I can push the HIGHRES and pre-commit runs tomorrow if people are ok with the
MIDRES and the current flow (trying to save some resources)
> Add missing tests suites to CircleCI
> ------------------------------------
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
> Reporter: Ekaterina Dimitrova
> Assignee: Ekaterina Dimitrova
> Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
