Mikhail, For me is hard to differentiate between X509CertImplTest and X509CertImplITest. So I vote for inventing something different.
Why moving implementation-independent tests into a separate package (like org.apache.harmony.test.<tested_package>) is no longer discussed? BTW, in this case we can remove excessive Test suffix. With best regards, Alexei Fedotov, Intel Middleware Products Division >-----Original Message----- >From: George Harley [mailto:[EMAIL PROTECTED] >Sent: Thursday, May 18, 2006 4:34 PM >To: harmony-dev@incubator.apache.org >Subject: Re: [classlib] Layout of tests in crypto module > >Mikhail Loenko wrote: >> There are classes like X509CertImpl.java in this case impl test for it >> would be X509CertImplImplTest which does not look very good. >> >> How about ITest ? >> >> Thanks, >> Mikhail >> > >Hi Mikhail, > >Sure, that works for me. > >Best regards, >George > > >> 2006/5/18, George Harley <[EMAIL PROTECTED]>: >>> Hi Mikhail, >>> >>> That is a very good point and your suggestion of supplementing the class >>> or package name sounds like a very straightforward way around the >>> problem. Because there will be a number of tests that must be in an >>> identical package name to the type under test then it seems that the >>> differentiating mark needs to be added to the test class name. Not >>> really sure what this could be. Given that we are all settled on the >>> convention that a test type has the "Test" suffix, how about just add >>> "Impl" to that suffix for implementation tests giving us an "ImplTest" >>> suffix. >>> e.g. >>> >>> class under test : java.lang.SomeClassTest >>> implementation-independent test class : >>> o.a.h.module.tests.java.lang.SomeClassTest >>> Harmony-specific test class : >>> o.a.h.module.tests.java.lang.SomeClassImplTest >>> >>> >>> My 2 Euro cents... >>> George >>> >>> >>> >>> Mikhail Loenko wrote: >>> > That sounds very reasonable, but I have a problem: >>> > >>> > I tried to implement it and found that as far as we put all test >>> results >>> > into a single directory and generate a single report, we can't have >>> > different tests with the same name. >>> > >>> > For example we can't have impl and api tests of >>> > o.a.h.module.tests.java.lang.SomeClassTest >>> > Looks like we should put some differentiating mark to a class or >>> > package name. >>> > >>> > Thanks, >>> > Mikhail >>> > >>> > 2006/5/18, George Harley <[EMAIL PROTECTED]>: >>> >> Mikhail Loenko wrote: >>> >> > Hi George, >>> >> > >>> >> > I use ant to build and run the tests, so I'm likely unaware of some >>> >> > Eclipse >>> >> > problems. >>> >> > >>> >> > If we put classpath test classes to >>> >> > impl/java and api/java >>> >> > and bootclasspath ones to something like >>> >> > impl/java.injected and api/java.injected >>> >> > will it solve the problem you describe? >>> >> > >>> >> > Thanks, >>> >> > Mikhail >>> >> > >>> >> > >>> >> >>> >> Hi Mikhail, >>> >> >>> >> Yes, I think that compiling to separate bin folders would do the >>> trick ; >>> >> a bin for stuff that will go on the classpath and a bin for stuff >>> that >>> >> will be loaded on the bootclasspath. In order to reach that goal the >>> >> test source itself will AFAIK need to be laid out in a similar >>> manner. >>> >> That is, the directory tree will look something like the following >>> (may >>> >> need to be viewed in wide-screen): >>> >> >>> >> >>> >> src/test/api/java <--- implementation-independent tests >>> >> intended to be loaded by system class loader, compiled to bin >>> >> >>> >> src/test/api/java.injected <--- implementation-independent tests >>> >> intended to be loaded by boot class loader, compiled to bin.injected >>> >> >>> >> src/test/impl/java <--- Harmony-specific tests >>> intended to >>> >> be loaded by system class loader, compiled to bin >>> >> >>> >> src/test/impl/java.injected <--- Harmony-specific tests >>> intended to >>> >> be loaded by boot class loader, compiled to bin.injected >>> >> >>> >> >>> >> >>> >> Does that sound reasonable ? >>> >> >>> >> >>> >> Best regards, >>> >> George >>> >> >>> >> >>> >> > 2006/5/17, George Harley <[EMAIL PROTECTED]>: >>> >> >> Mikhail Loenko wrote: >>> >> >> > 2006/5/16, George Harley <[EMAIL PROTECTED]>: >>> >> >> >> Mikhail Loenko wrote: >>> >> >> >> > Hi George, see below >>> >> >> >> > >>> >> >> >> > 2006/5/16, George Harley <[EMAIL PROTECTED]>: >>> >> >> >> >> Hi Mikhail, >>> >> >> >> >> >>> >> >> >> >> I have a couple of minor comments about your proposal for a >>> >> test >>> >> >> >> >> layouts. I should have responded sooner, I know, but I have >>> >> >> suffered >>> >> >> >> >> from a number of hardware problems in the past few weeks >>> that >>> >> >> slowed >>> >> >> >> >> things down somewhat for me. Anyway, it's all great but it >>> >> would >>> >> >> >> be nice >>> >> >> >> >> to get answers to the following ... >>> >> >> >> >> >>> >> >> >> >> 1) The section on "Location of the tests in the directory >>> tree" >>> >> >> >> >> proposes <modulename>/src/tests/impl for Harmony specific >>> tests >>> >> >> and >>> >> >> >> >> <modulename>/src/tests/api for implementation-independent >>> >> tests. >>> >> >> >> Where >>> >> >> >> >> would tests go for Harmony API's that are not part of the >>> J2SE >>> >> >> >> spec but >>> >> >> >> >> are still accessible ? Strictly speaking they are API as >>> >> well as >>> >> >> >> being >>> >> >> >> >> specific to Harmony. >>> >> >> >> > >>> >> >> >> > The main idea is to separate tests that must pass on any >>> >> conformant >>> >> >> >> > implementation from the tests passing on Harmony only. >>> >> >> >> > >>> >> >> >> > When these are separated we can e.g. easily validate >>> >> >> "implementation- >>> >> >> >> > independent" tests by running them against RI. Actually I >>> >> would not >>> >> >> >> > like to >>> >> >> >> > start an endless "philosophical" discussion here about >>> what are >>> >> >> >> > unit vs. api vs. whatever tests. >>> >> >> >> >>> >> >> >> Me ? Start a philosophical discussion ? :-) >>> >> >> >> >>> >> >> >> >>> >> >> >> > >>> >> >> >> > "api" is just a name (suggested by Tim) and might be changed >>> >> to any >>> >> >> >> > unconfusing one. >>> >> >> >> > >>> >> >> >> > So, back to your question, those must go to 'impl' as far as >>> >> >> they fail >>> >> >> >> > on RI: >>> >> >> >> > "<modulename>/src/tests/impl - Harmony specific tests" >>> >> >> >> > >>> >> >> >> >> What about the location of tests designed to run on >>> >> >> >> >> the classpath and tests designed to run on the >>> bootclasspath ? >>> >> >> >> That does >>> >> >> >> >> not seem to be addressed in this section. When the tests are >>> >> run >>> >> >> >> how are >>> >> >> >> > >>> >> >> >> > Directory defines the root of the test suite, then the >>> package >>> >> >> goes, >>> >> >> >> > see "Package names for different types of the tests" section: >>> >> >> >> > "If the test is designed to be run from bootclasspath then >>> its >>> >> >> package >>> >> >> >> > is the same as the package of the class under test" >>> >> >> >> > >>> >> >> >> >>> >> >> >> Is it the case that all of the classes under a particular >>> source >>> >> >> folder >>> >> >> >> (say src/tests/api/java) get compiled to one output folder ? >>> >> >> > >>> >> >> > We have not discuss it yet. So, suggestions are welcome! :) >>> >> >> > >>> >> >> > Thanks, >>> >> >> > Mikhail >>> >> >> > >>> >> >> > >>> >> >> >>> >> >> Hi Mikhail, >>> >> >> >>> >> >> OK, great. I'm wondering about how the test classes intended >>> to be >>> >> >> loaded by the bootclasspath loader get distinguished from the >>> classes >>> >> >> that are intended to be loaded by the system loader. For the >>> sake of >>> >> >> discussion, imagine that all of the test classes under a given >>> source >>> >> >> folder (e.g. src/tests/api/java) get compiled to just one output >>> >> folder >>> >> >> (e.g. bin). At test runtime we only want the classes that are >>> "in the >>> >> >> same package as the class under test" to be on the bootclasspath - >>> >> but >>> >> >> how is this enforced ? If the tests are being run from an Ant >>> script >>> >> >> then I think that it is possible to do this using Ant's path-like >>> >> >> structures where wildcards can be specified among the classpath >>> and >>> >> >> bootclasspath elements. But how can an equivalent degree of >>> >> separation >>> >> >> be enforced if the tests are being run from an IDE such as >>> Eclipse ? >>> >> >> >>> >> >> I am not exactly an Eclipse "power-user" so it is possible >>> that I am >>> >> >> missing some piece of magic here but it appears to me that if I >>> >> want to >>> >> >> configure the classpath and bootclasspath for running a particular >>> >> set >>> >> >> of unit tests I can only work at the output folder level. That >>> is, it >>> >> >> can be configured that classes under a root folder of "bin" are on >>> >> the >>> >> >> classpath and that classes under the root folder "bin-other" are >>> >> >> similarly on the bootclasspath. But there is nothing available >>> >> that only >>> >> >> puts classes in package java.util.* under "bin" on the >>> >> bootclasspath and >>> >> >> everything else on the classpath. >>> >> >> >>> >> >> All of which makes me wonder if we should aim at compiling >>> classes >>> >> >> intended for the bootclasspath to one bin and classes intended for >>> >> the >>> >> >> classpath to another bin. It wouldn't really make much difference >>> >> to Ant >>> >> >> users but as far as I can tell it would offer more flexibility >>> to IDE >>> >> >> users who would then have the potential to run the tests in a way >>> >> that >>> >> >> was more familiar to them (runtime configuration, green bar etc). >>> >> >> >>> >> >> >>> >> >> Best regards, >>> >> >> George >>> >> >> >>> >> >> > >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >> the "bootclasspath" and "classpath" tests distinguished ? >>> >> >> Purely on >>> >> >> >> >> package name ? Did you ever see the append I wrote to the >>> >> list a >>> >> >> >> couple >>> >> >> >> >> of weeks ago on this topic ? [1] >>> >> >> >> > >>> >> >> >> > Yes, I've seen it. As you could notice we had more test >>> >> categories. >>> >> >> >> > They are described in the same thread. We might have >>> >> >> >> > "independent" tests running from bootclasspath and "specific" >>> >> ones >>> >> >> >> > running from classpath. >>> >> >> >> > >>> >> >> >> >> >>> >> >> >> >> 2) Still in the "Location of the tests in the directory >>> tree" >>> >> >> >> section, >>> >> >> >> >> shouldn't the suggested source folder names include >>> "java" in >>> >> >> there ? >>> >> >> >> >> e.g. <modulename>/src/tests/java/api ? What is wrong with >>> the >>> >> >> >> >> src/test/java (below here is Java code), src/test/resources >>> >> (below >>> >> >> >> here >>> >> >> >> >> is non-code test artefacts) convention ? I notice that at >>> >> >> present the >>> >> >> >> >> page does not include any mention of where test resources >>> would >>> >> >> go. >>> >> >> >> > >>> >> >> >> > Yes, you are right. It's missing. >>> >> >> >> > >>> >> >> >> > It was supposed to be under test "type", like: >>> >> >> >> > module/src/test/api/java >>> >> >> >> > module/src/test/api/resources >>> >> >> >> > >>> >> >> >> > >>> >> >> >> >> >>> >> >> >> >> 3) What does the sentence "Tests are not separated by >>> >> >> functionality >>> >> >> >> >> under test, e.g. tests against clone() methods are not >>> >> >> separated from >>> >> >> >> >> tests against equals() methods" actually mean ? >>> >> >> >> > >>> >> >> >> > That was to address Tim's concern: >>> >> >> >> > " > 1. Tests are separated on directory level by 'intention' >>> >> >> >> > >>> >> >> >> > I agree where the intention is broad (i.e. functional >>> tests vs. >>> >> >> >> > performance tests vs. stress tests) but I'm not convinced >>> that >>> >> >> we need >>> >> >> >> > to separate to a precise 'intent', at least I've never >>> been in a >>> >> >> >> > position where I've wished that my my serialization tests are >>> >> >> separate >>> >> >> >> > from my cloning tests or whatever." >>> >> >> >> > >>> >> >> >> > Feel free to reword if something is not clear in the proposal >>> >> >> >> > >>> >> >> >> > Thanks, >>> >> >> >> > Mikhail >>> >> >> >> > >>> >> >> >> > >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> Best regards, >>> >> >> >> >> George >>> >> >> >> >> >>> >> >> >> >> [1] >>> >> >> >> >> >>> >> >> >> >>> >> >> >>> >> >>> http://mail-archives.apache.org/mod_mbox/incubator-harmony- >dev/200604.mbox/[EMAIL PROTECTED] >>> >>> >> >>> >> >> >>> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> Mikhail Loenko wrote: >>> >> >> >> >> > Hello >>> >> >> >> >> > >>> >> >> >> >> > I would like to make some changes in the crypto module: >>> >> >> >> >> > >>> >> >> >> >> > - Separate implemetation-independent tests from >>> >> implementation >>> >> >> >> >> specific >>> >> >> >> >> > ones. >>> >> >> >> >> > >>> >> >> >> >> > - Fix layout according to the proposed scheme [1] >>> >> >> >> >> > >>> >> >> >> >> > Please let me know if you do any changes in that module >>> then >>> >> >> >> >> > I'll delay restruct. >>> >> >> >> >> > >>> >> >> >> >> > Thanks, >>> >> >> >> >> > Mikhail >>> >> >> >> >> > >>> >> >> >> >> > [1] >>> >> >> >> >> > >>> >> >> >> >> >>> >> >> >> >>> >> >> >>> >> >>> >http://incubator.apache.org/harmony/subcomponents/classlibrary/testing. html >>> >>> >> >>> >> >> >>> >> >> >> >>> >> >>> >> > > >--------------------------------------------------------------------- >Terms of use : http://incubator.apache.org/harmony/mailing.html >To unsubscribe, e-mail: [EMAIL PROTECTED] >For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
