Re: Javascript libs compliance degrading?

Mon, 25 Aug 2008 02:06:28 -0700

When you load up the reference test suite in the java sample container the result is: 
94 Passed
47 Failed
2 Warnings
22 Unverified
now 47 is a bit steep, especially considering that this uses java's json-rpc interface, so instruction ordering shouldn't be an issue here. 


A fair bit of those are silly errors (like the empty proxy string), or  
errors like "expected 'gadgets.Tab', got " 'gadgets.[object  
Object],<spam>'". and some errors that make no sense to me "[PPL005.1]  
Nonsupported Field - familyName: FAILED: (got 'Doe'), expected  
'undefined'" (name is supported, so why complain you got a familyName?)



However that doesn't account for all 47 errors, there's a few real  
ones in there too, and it's currently quite hard to separate the real  
failures from the ones that don't really matter.



I'm slightly concerned that with such a volume of errors (wether they  
are real errors or not), the tool looses it's usefulness. I mean if  
someone checks out shindig, implements the basic services and runs the  
test suite to see if they did that correctly .... How would one not  
completely familiar with the complete opensocial stack be able to  
diagnose what is or isn't the fault of their own code? A needle and  
haystack come to mind :)


        -- Chris

On Aug 25, 2008, at 4:58 AM, Cassie wrote:


Ahh - disclaimer for my last statement - I was only talking about the
opensocial related tests... I don't usually check the non-social ones
:)

- Cassie



On Sat, Aug 23, 2008 at 4:46 PM, Chris Chabot <[EMAIL PROTECTED]>  
wrote:

Don't suppose these are easy to fix on the javascript side?

gadgets.io.* TestSuite:


Description> Tests if we can get the proxy URL with given URL as  
proxy
[GIO101.0] gadgets.io.getProxyUrl(String) - With valid URL.: PASS:  
(got

'proxy?refresh=3600&url=http%3A%2F%2Fwww%2F~user')

[GIO101.1] gadgets.io.getProxyUrl(String) - With valid URL.: PASS:  
(got

'proxy?refresh=3600&url=http%3A%2F%2Fwww%2F~user')

[GIO102] [P2 ]:: gadgets.io.getProxyUrl() - With no parameters.:  
FAILED [458

ms]

Description> Tests if we can call getProxyUrl API without any  
parameter and

it returns the proxy URL with empty proxy
[GIO102.0] gadgets.io.getProxyUrl() - With no parameters.: PASS: (got
'proxy?refresh=3600&url=undefined')

[GIO102.1] gadgets.io.getProxyUrl() - With no parameters.: FAILED:  
(got

'proxy?refresh=3600&url=undefined'), expected 'url='

[GIO103] [P2 ]:: gadgets.io.getProxyUrl(null) - With null  
parameter.: FAILED

[462 ms]

Description> Tests if we can call getProxyUrl API with null as  
parameter and

it returns the proxy URL with empty proxy

[GIO103.0] gadgets.io.getProxyUrl(null) - With null parameter.:  
PASS: (got

'proxy?refresh=3600&url=null')

[GIO103.1] gadgets.io.getProxyUrl(null) - With null parameter.:  
FAILED: (got

'proxy?refresh=3600&url=null'), expected 'url='
[GIO104] [P2 ]:: gadgets.io.getProxyUrl(undefined) - With undefined
parameter.: FAILED [466 ms]
Description> Tests if we can call getProxyUrl API with undefined as
parameter and it returns the proxy URL with empty proxy

[GIO104.0] gadgets.io.getProxyUrl(undefined) - With undefined  
parameter.:

PASS: (got 'proxy?refresh=3600&url=undefined')

[GIO104.1] gadgets.io.getProxyUrl(undefined) - With undefined  
parameter.:

FAILED: (got 'proxy?refresh=3600&url=undefined'), expected 'url='

I'm just a sucker for seeing green boxes is all :)

      -- Chris

On Aug 22, 2008, at 11:04 PM, Cassie wrote:


I check the compliance tests regularly for the actual deployment of
Shindig

that I work on at work. We are failing more now only because the  
tests are
getting much more thorough. (The tests are also very active so  
sometimes

they have bugs too although it is usually our code that's wrong :)


I haven't found many issues with Shindig's actual js layer though  
- its

usually been in the server layer and most often in the service
implementations that are container specific.


The non-rpc based container definitely has some issues though  
because it

sending requests to the server in a json map format... which doesn't

preserve order. So, some of the compliance tests would fail simply  
because

they were fetching app data before it was updated and so forth.


So... hopefully someone out there can get a patch to switch the  
php to rpc

batching going :)

- Cassie



On Fri, Aug 22, 2008 at 12:28 PM, Dan Peterson  
<[EMAIL PROTECTED]>

wrote:


By the way, the docs for the compliance test suite are at:
http://code.google.com/p/opensocial-resources/wiki/ComplianceTests

-Dan

On Fri, Aug 22, 2008 at 12:22 PM, Louis Ryan <[EMAIL PROTECTED]>  
wrote:



Im seeing some similar issues. One thing I noticed is that  
lookingFor is

now

an Enum in JS but its not the Java datamodel. Im going to fix  
that one.


On Fri, Aug 22, 2008 at 11:54 AM, Chris Chabot <[EMAIL PROTECTED]>


wrote:



When running the compliance test suite:






http://opensocial-resources.googlecode.com/svn/tests/trunk/suites/0.7/compliance/reference/reference.xml


I get 28 failed on my live version of partuza+php shindig (


www.partuza.nl is



running a checkout that is about 1.5 weeks old), while the  
latest code

locally gives me 42 errors.


To rule out that it wasn't the php code, i updated just shindig/ 
php on


the



live server, and the error count didn't change, so it's  
probably some

shindig//features/* changes that cause this.

Anyone checking if our JS code is 'compliant' ? And/or working on


fixing


it? Seems right now it's only getting less so :)

    -- Chris
































					Reply via email to