[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[
https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456944#comment-16456944
]
Mark Miller commented on SOLR-11749:
Just as an example, your Junit tests would end up looking something like this:
{noformat}
public class TestSolrScriptsStuff1 extends Assert {
public TestSolrScriptsStuff1() {
}
@BeforeClass
public static void setUpBeforeClass() throws Exception {
String[] cmd = new String[]{"bash", "test-setup.sh"};
int exitVal = Util.runCmd(cmd).returnCode;
if (exitVal > 0) {
fail("Setup Failed!");
}
}
@AfterClass
public static void tearDownAfterClass() throws Exception {
String[] cmd = new String[] {"bash",
"test-teardown.sh"};
int exitVal = Util.runCmd(cmd).returnCode;
if (exitVal > 0) {
fail("Tear Down Failed!");
}
}
@Test
public void test1() throws Exception {
String[] cmd = new String[]{"bash", "test1.sh"};
Util.PbResult result = Util.runCmd(cmd);
assertEquals("Testing test1.sh failed", 0, result.returnCode);
}
@Test
public void test2() throws Exception {
String[] cmd = new String[]{"bash", "test2.sh"};
Util.PbResult result = Util.runCmd(cmd);
assertEquals("Testing test2.sh failed", 0, result.returnCode);
}
}{noformat}
> regression-test-like functionality for bin/solr*
>
>
> Key: SOLR-11749
> URL: https://issues.apache.org/jira/browse/SOLR-11749
> Project: Solr
> Issue Type: Wish
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Reporter: Christine Poerschke
>Assignee: Jason Gerlowski
>Priority: Minor
> Attachments: SOLR-11749.patch, SOLR-11749.patch, SOLR-11749.patch,
> SOLR-11749.patch, SOLR-11749.patch, test-output.txt
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[ https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456923#comment-16456923 ] Mark Miller commented on SOLR-11749: Oh yeah, and then you don't have to implement the test framework for windows as well. > regression-test-like functionality for bin/solr* > > > Key: SOLR-11749 > URL: https://issues.apache.org/jira/browse/SOLR-11749 > Project: Solr > Issue Type: Wish > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Reporter: Christine Poerschke >Assignee: Jason Gerlowski >Priority: Minor > Attachments: SOLR-11749.patch, SOLR-11749.patch, SOLR-11749.patch, > SOLR-11749.patch, SOLR-11749.patch, test-output.txt > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[ https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456922#comment-16456922 ] Mark Miller commented on SOLR-11749: This is great that you guys are addressing this! I wonder if trying to make ScriptUnit is the best approach though. It's been done a couple times before for bash at least, but I've always tried to avoid those projects. Just a thought, but I wonder if it would not be better to just drive these by junit? If they are very long we could make some or all of them @Nightly. What I kind of like is to make sure all of my bash scripts are setup to fail if any single command fails. Then a test gets its own bash scipt that just executes the test. If that returns non 0, we fail the junit test method. Now, if any command fails in the Solr script, the test is failed. If anything fails in the test itself, the test is failed. Now you can do really simple assert stuff like try and list a file, if it doesn't exist the test will fail. This gives you the same test reporting we get with our other tests, lets you do things like run the tests from an IDE, all of them or individually, and lets us treat the tests the same as our other tests for the most part with the same reporting. You can add an @assume to only run on the right OS for the script, or just an explicit check of the OS and call the right script. It also lets us avoid almost any scripting related to a script testing framework. Really, all you need to do is set bash up to be easily debuggable so that fails show the right info, which is just a few lines of bash. > regression-test-like functionality for bin/solr* > > > Key: SOLR-11749 > URL: https://issues.apache.org/jira/browse/SOLR-11749 > Project: Solr > Issue Type: Wish > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Reporter: Christine Poerschke >Assignee: Jason Gerlowski >Priority: Minor > Attachments: SOLR-11749.patch, SOLR-11749.patch, SOLR-11749.patch, > SOLR-11749.patch, SOLR-11749.patch, test-output.txt > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[ https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456503#comment-16456503 ] ASF subversion and git services commented on SOLR-11749: Commit b3f736d2653143e5b4fc8dfa48f4dfde93f0e299 in lucene-solr's branch refs/heads/branch_7x from [~gerlowskija] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b3f736d ] SOLR-11749: Fix logic errors in some assert funcs > regression-test-like functionality for bin/solr* > > > Key: SOLR-11749 > URL: https://issues.apache.org/jira/browse/SOLR-11749 > Project: Solr > Issue Type: Wish > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Reporter: Christine Poerschke >Assignee: Jason Gerlowski >Priority: Minor > Attachments: SOLR-11749.patch, SOLR-11749.patch, SOLR-11749.patch, > SOLR-11749.patch, SOLR-11749.patch, test-output.txt > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[ https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456495#comment-16456495 ] ASF subversion and git services commented on SOLR-11749: Commit 43c086a002b3488a6fdd6a71ce1879a2538cce4e in lucene-solr's branch refs/heads/master from [~gerlowskija] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=43c086a ] SOLR-11749: Fix logic errors in some assert funcs > regression-test-like functionality for bin/solr* > > > Key: SOLR-11749 > URL: https://issues.apache.org/jira/browse/SOLR-11749 > Project: Solr > Issue Type: Wish > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Reporter: Christine Poerschke >Assignee: Jason Gerlowski >Priority: Minor > Attachments: SOLR-11749.patch, SOLR-11749.patch, SOLR-11749.patch, > SOLR-11749.patch, SOLR-11749.patch, test-output.txt > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[
https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413046#comment-16413046
]
Jason Gerlowski commented on SOLR-11749:
Was thinking about next steps for this test suite, and there's a lot of
branches that I'd like this to go in.
- expand test coverage for other {{bin/solr}} tools (auth, zk, assert,
start/stop/restart).
- add Windows equivalent functionality
- minor test harness/runner improvements (add verbosity controls, etc.)
- performance improvements (currently the suite takes too long to run to make
it practical to run on normal builds).
- integrate with {{ant test}} target
I plan on writing up JIRA issues for each of these, and using this issue as a
parent/umbrella for the remaining functionality that I think is still
low-hanging fruit here. That should make it easy for others to pitch in too if
they feel the urge.
> regression-test-like functionality for bin/solr*
>
>
> Key: SOLR-11749
> URL: https://issues.apache.org/jira/browse/SOLR-11749
> Project: Solr
> Issue Type: Wish
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Reporter: Christine Poerschke
>Assignee: Jason Gerlowski
>Priority: Minor
> Attachments: SOLR-11749.patch, SOLR-11749.patch, SOLR-11749.patch,
> SOLR-11749.patch, SOLR-11749.patch, test-output.txt
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[ https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401361#comment-16401361 ] ASF subversion and git services commented on SOLR-11749: Commit bd83a339474f34c91e0f2e9a1057ef43ede645d7 in lucene-solr's branch refs/heads/branch_7x from [~gerlowskija] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bd83a33 ] SOLR-11749: Introduce bin/solr test suite > regression-test-like functionality for bin/solr* > > > Key: SOLR-11749 > URL: https://issues.apache.org/jira/browse/SOLR-11749 > Project: Solr > Issue Type: Wish > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Reporter: Christine Poerschke >Assignee: Jason Gerlowski >Priority: Minor > Attachments: SOLR-11749.patch, SOLR-11749.patch, SOLR-11749.patch, > SOLR-11749.patch, SOLR-11749.patch, test-output.txt > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[ https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401354#comment-16401354 ] ASF subversion and git services commented on SOLR-11749: Commit 2ca741d36a3078e7d7b03cb73176a1e99377eefc in lucene-solr's branch refs/heads/master from [~gerlowskija] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2ca741d ] SOLR-11749: Introduce bin/solr test suite > regression-test-like functionality for bin/solr* > > > Key: SOLR-11749 > URL: https://issues.apache.org/jira/browse/SOLR-11749 > Project: Solr > Issue Type: Wish > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Reporter: Christine Poerschke >Assignee: Jason Gerlowski >Priority: Minor > Attachments: SOLR-11749.patch, SOLR-11749.patch, SOLR-11749.patch, > SOLR-11749.patch, SOLR-11749.patch, test-output.txt > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[ https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394556#comment-16394556 ] Erick Erickson commented on SOLR-11749: --- +1 to check in the Unix version. Perhaps someone with access to Windows could pitch in... > regression-test-like functionality for bin/solr* > > > Key: SOLR-11749 > URL: https://issues.apache.org/jira/browse/SOLR-11749 > Project: Solr > Issue Type: Wish > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Reporter: Christine Poerschke >Assignee: Jason Gerlowski >Priority: Minor > Attachments: SOLR-11749.patch, SOLR-11749.patch, SOLR-11749.patch, > SOLR-11749.patch, SOLR-11749.patch, test-output.txt > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[
https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394548#comment-16394548
]
Jason Gerlowski commented on SOLR-11749:
*Changes*
- added the ability to run a single test suite {{-s}}, or a single specific
test {{-t}}
- added basic help test {{-h}}
- Added a README.md which gives a little more help for those looking to add
tests or understand how the test-runner/framework works.
I've had a lot of trouble making progress on the Windows version of this
script. Mostly because I only rarely have access to a Windows machine to make
and test changes. For that reason, I'm starting to wonder if it's worth
committing the *nix tests by themselves, and following up with the Windows
logic afterward. If I'm being honest, it'll be at least another few weeks
before I can hope to have the Windows side done. Committing the Linux tests in
the meantime would (in theory) allow others to add tests of their own, improve
the framework, or even port the logic to Windows on their own.
If no one objects to this approach, or has review comments, I hope to commit
this later this week. I'll probably ask for specific review of this on the
mailing list too, since bash has a lot of pitfalls that I may not have caught,
and I think this is a change others might be interested in too.
> regression-test-like functionality for bin/solr*
>
>
> Key: SOLR-11749
> URL: https://issues.apache.org/jira/browse/SOLR-11749
> Project: Solr
> Issue Type: Wish
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Reporter: Christine Poerschke
>Assignee: Jason Gerlowski
>Priority: Minor
> Attachments: SOLR-11749.patch, SOLR-11749.patch, SOLR-11749.patch,
> SOLR-11749.patch, SOLR-11749.patch, test-output.txt
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[
https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388679#comment-16388679
]
Jason Gerlowski commented on SOLR-11749:
Just noticed that my previous patch was broken. Due to some .gitignore rules I
didn't know about, some bash functions I'd pulled into a {{lib}} directory
under {{bin-test}} didn't get picked up when I wrote the patch. Fixed that and
reuploaded.
In other minor news, I'm making creeping progress on a batch version of this
functionality, but it's slow going.
> regression-test-like functionality for bin/solr*
>
>
> Key: SOLR-11749
> URL: https://issues.apache.org/jira/browse/SOLR-11749
> Project: Solr
> Issue Type: Wish
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Reporter: Christine Poerschke
>Assignee: Jason Gerlowski
>Priority: Minor
> Attachments: SOLR-11749.patch, SOLR-11749.patch, SOLR-11749.patch,
> SOLR-11749.patch, test-output.txt
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[
https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367468#comment-16367468
]
Jason Gerlowski commented on SOLR-11749:
I attached a patch above which adds enough tests to establish a pattern for how
these might be developed.
I tried my best to make the tests look as much as possible like the Java JUnit
tests everyone is familiar with.
- there are test suites (files named "test_*.sh" anywhere under {{bin-test}})
- inside suites, there are individual test cases (bash functions prefixed with
{{solr_test_}})
- there are @Before and @After analogues ({{solr_unit_test_before}} and
{{solr_unit_test_after}})
- there are @BeforeClass and @AfterClass analog [^test-output.txt] ues
({{solr_suite_before}} and {{solr_suite_after}})
The output, with the uploaded patch, is also attached.
> regression-test-like functionality for bin/solr*
>
>
> Key: SOLR-11749
> URL: https://issues.apache.org/jira/browse/SOLR-11749
> Project: Solr
> Issue Type: Wish
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Reporter: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-11749.patch, SOLR-11749.patch, test-output.txt
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[ https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362763#comment-16362763 ] Christine Poerschke commented on SOLR-11749: bq. ... bin/solr changes ... lack of tests has been a real burden. ... Yes, that was also what led me to starting the minimal patch. bq. ... if you don't mind, I'll add a few example tests to the minimal start you attached above, and see what we can learn from that. Yes, please! That was exactly the idea behind starting a minimal patch which can grow. > regression-test-like functionality for bin/solr* > > > Key: SOLR-11749 > URL: https://issues.apache.org/jira/browse/SOLR-11749 > Project: Solr > Issue Type: Wish > Security Level: Public(Default Security Level. Issues are Public) > Components: scripts and tools >Reporter: Christine Poerschke >Priority: Minor > Attachments: SOLR-11749.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11749) regression-test-like functionality for bin/solr*
[
https://issues.apache.org/jira/browse/SOLR-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359111#comment-16359111
]
Jason Gerlowski commented on SOLR-11749:
I'm a big fan of this as an idea. I've been toying with some bin/solr changes
recently, but the lack of tests has been a real burden. Conceivably if we get
a good suite of tests here, we could even hook them into {{ant test}} so it
runs as a part of ant command most people run.
The only downside is that having these tests in bash/batch means we'll need two
copies of the same test-logic (one in bash, one in Windows batch script). Not
sure of a way around that. Technically the logic could be in Java, but it'd be
full of a bunch of {{isWindows}}/{{isLinux}} conditionals, which wouldn't be
much better. Just thinking aloud here...
Anyways, if you don't mind, I'll add a few example tests to the minimal start
you attached above, and see what we can learn from that.
> regression-test-like functionality for bin/solr*
>
>
> Key: SOLR-11749
> URL: https://issues.apache.org/jira/browse/SOLR-11749
> Project: Solr
> Issue Type: Wish
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Reporter: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-11749.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
