Chris,
Thanks for trying the gadget. It uncovered couple wrong API use. I
fixed tehm in the gadget. Rightnow this is the summary for partuza.nl
Passed Failed Warnings Unverified Total
116 18 5 23 162
It's much cleaner now :-)
PPLX01 - throws exception : line 115 : peoplesuite.js
PPL004 - gives false for
opensocial.getEnvironment().supportsField(opensocial.Environment.ObjectType.ADDRESS,
"familyName"); but it also returns value which is contradictory. Test
expects true for any field which has value.
Opensocial spec 0.7 Definition :: SPEC-DEF-TEST - Need to fix the spec
gadgets.util.* Test Suite :: UTIL003 - container returns false for
opensocial-0.7, setPrefs etc.
AppData Test Suite :: APP010 -
AppData Test Suite :: APP103
AppData Test Suite :: APP302 -
AppData Test Suite :: APP008 -
in above cases it gives Internal server error in error message but
hadError is always false.
AppData Test Suite :: APP300 - it expects error because we are making call
req.add(req.newFetchPersonAppDataRequest('VIEWER_OWNER_ABC',
[this.id + '_dataKey1']), 'invalidFetchRequest');
'VIEWER_OWNER_ABC' is invalid id.
MakeRequest Test Suite :: SMKRT001 - data.validated = false
MakeRequest Test Suite :: MKRT005 - Json parse error
MakeRequest Test Suite :: MKRT011 - json parse error
gadget.Prefs Test Suite :: PREF010 - prefs are not honored for all the
cases below
gadget.Prefs Test Suite :: PREF020
gadget.Prefs Test Suite :: PREF030
gadget.Prefs Test Suite :: PREF040
gadget.Prefs Test Suite :: PREF050
I believe that gadgets spec is very little documented and requires
some brush up. But that's definitely not in the scope of compliance
gadget.
regards,
Shreyas
> - Show quoted text -
>
> On Mon, Aug 25, 2008 at 9:53 AM, Dan Peterson <[EMAIL PROTECTED]> wrote:
>> Hey folks,
>>
>> For suggestions to improve the compliance testing suite, I'd highly
>> encourage you to write to the spec list with feedback.
>>
>> -Dan
>> On Mon, Aug 25, 2008 at 2:16 AM, Chris Chabot <[EMAIL PROTECTED]> wrote:
>>
>>> Ps i also tried the archived old test gadget (that I'm kinda missing right
>>> now) at
>>>
>>> http://opensocial-resources.googlecode.com/svn/tests/trunk/archive/compliancetests.xml
>>>
>>> However that seems to error out now too (java + samplecontainer & on php
>>> shindig + partuza as well) on "request create activity" (x2), "update person
>>> app data" and "people field return types".
>>>
>>> In other words, I'm having a bit of a rough time testing stuff while trying
>>> to code up a json-rpc interface in php.. if everything is 'red', it's a good
>>> bit harder to verify, and i simply don't have the time right now to write my
>>> own gadget test code combined with the php code.
>>>
>>> I imagine containers currently working to implement shindig (and there's a
>>> few of those i know of) might run into these issues as well
>>>
>>> -- Chris
>>>
>>>
>>>
>>> On Aug 25, 2008, at 11:09 AM, Kevin Brown wrote:
>>>
>>> On Mon, Aug 25, 2008 at 2:05 AM, Chris Chabot <[EMAIL PROTECTED]> wrote:
>>>>
>>>> When you load up the reference test suite in the java sample container the
>>>>> result is:
>>>>> 94 Passed
>>>>> 47 Failed
>>>>> 2 Warnings
>>>>> 22 Unverified
>>>>>
>>>>> now 47 is a bit steep, especially considering that this uses java's
>>>>> json-rpc interface, so instruction ordering shouldn't be an issue here.
>>>>>
>>>>> A fair bit of those are silly errors (like the empty proxy string), or
>>>>> errors like "expected 'gadgets.Tab', got " 'gadgets.[object
>>>>> Object],<spam>'". and some errors that make no sense to me "[PPL005.1]
>>>>> Nonsupported Field - familyName: FAILED: (got 'Doe'), expected
>>>>> 'undefined'"
>>>>> (name is supported, so why complain you got a familyName?)
>>>>>
>>>>> However that doesn't account for all 47 errors, there's a few real ones
>>>>> in
>>>>> there too, and it's currently quite hard to separate the real failures
>>>>> from
>>>>> the ones that don't really matter.
>>>>>
>>>>> I'm slightly concerned that with such a volume of errors (wether they are
>>>>> real errors or not), the tool looses it's usefulness. I mean if someone
>>>>> checks out shindig, implements the basic services and runs the test suite
>>>>> to
>>>>> see if they did that correctly .... How would one not completely familiar
>>>>> with the complete opensocial stack be able to diagnose what is or isn't
>>>>> the
>>>>> fault of their own code? A needle and haystack come to mind :)
>>>>>
>>>>
>>>>
>>>> I agree, which is why we should talk to the people writing the compliance
>>>> gadget and get rid of the unnecessary stuff first, then we can look for
>>>> real
>>>> problems.
>>>>
>>>>
>>>>
>>>>>
>>>>> -- Chris
>>>>>
>>>>>
>>>>> On Aug 25, 2008, at 4:58 AM, Cassie wrote:
>>>>>
>>>>> Ahh - disclaimer for my last statement - I was only talking about the
>>>>>
>>>>>> opensocial related tests... I don't usually check the non-social ones
>>>>>> :)
>>>>>>
>>>>>> - Cassie
>>>>>>
>>>>>>
>>>>>> On Sat, Aug 23, 2008 at 4:46 PM, Chris Chabot <[EMAIL PROTECTED]>
>>>>>> wrote:
>>>>>>
>>>>>> Don't suppose these are easy to fix on the javascript side?
>>>>>>>
>>>>>>> gadgets.io.* TestSuite:
>>>>>>>
>>>>>>> Description> Tests if we can get the proxy URL with given URL as proxy
>>>>>>> [GIO101.0] gadgets.io.getProxyUrl(String) - With valid URL.: PASS: (got
>>>>>>> 'proxy?refresh=3600&url=http%3A%2F%2Fwww%2F~user')
>>>>>>> [GIO101.1] gadgets.io.getProxyUrl(String) - With valid URL.: PASS: (got
>>>>>>> 'proxy?refresh=3600&url=http%3A%2F%2Fwww%2F~user')
>>>>>>> [GIO102] [P2 ]:: gadgets.io.getProxyUrl() - With no parameters.: FAILED
>>>>>>> [458
>>>>>>> ms]
>>>>>>> Description> Tests if we can call getProxyUrl API without any parameter
>>>>>>> and
>>>>>>> it returns the proxy URL with empty proxy
>>>>>>> [GIO102.0] gadgets.io.getProxyUrl() - With no parameters.: PASS: (got
>>>>>>> 'proxy?refresh=3600&url=undefined')
>>>>>>> [GIO102.1] gadgets.io.getProxyUrl() - With no parameters.: FAILED: (got
>>>>>>> 'proxy?refresh=3600&url=undefined'), expected 'url='
>>>>>>> [GIO103] [P2 ]:: gadgets.io.getProxyUrl(null) - With null parameter.:
>>>>>>> FAILED
>>>>>>> [462 ms]
>>>>>>> Description> Tests if we can call getProxyUrl API with null as
>>>>>>> parameter
>>>>>>> and
>>>>>>> it returns the proxy URL with empty proxy
>>>>>>> [GIO103.0] gadgets.io.getProxyUrl(null) - With null parameter.: PASS:
>>>>>>> (got
>>>>>>> 'proxy?refresh=3600&url=null')
>>>>>>> [GIO103.1] gadgets.io.getProxyUrl(null) - With null parameter.: FAILED:
>>>>>>> (got
>>>>>>> 'proxy?refresh=3600&url=null'), expected 'url='
>>>>>>> [GIO104] [P2 ]:: gadgets.io.getProxyUrl(undefined) - With undefined
>>>>>>> parameter.: FAILED [466 ms]
>>>>>>> Description> Tests if we can call getProxyUrl API with undefined as
>>>>>>> parameter and it returns the proxy URL with empty proxy
>>>>>>> [GIO104.0] gadgets.io.getProxyUrl(undefined) - With undefined
>>>>>>> parameter.:
>>>>>>> PASS: (got 'proxy?refresh=3600&url=undefined')
>>>>>>> [GIO104.1] gadgets.io.getProxyUrl(undefined) - With undefined
>>>>>>> parameter.:
>>>>>>> FAILED: (got 'proxy?refresh=3600&url=undefined'), expected 'url='
>>>>>>>
>>>>>>> I'm just a sucker for seeing green boxes is all :)
>>>>>>>
>>>>>>> -- Chris
>>>>>>>
>>>>>>> On Aug 22, 2008, at 11:04 PM, Cassie wrote:
>>>>>>>
>>>>>>> I check the compliance tests regularly for the actual deployment of
>>>>>>>
>>>>>>>> Shindig
>>>>>>>> that I work on at work. We are failing more now only because the tests
>>>>>>>> are
>>>>>>>> getting much more thorough. (The tests are also very active so
>>>>>>>> sometimes
>>>>>>>> they have bugs too although it is usually our code that's wrong :)
>>>>>>>>
>>>>>>>> I haven't found many issues with Shindig's actual js layer though -
>>>>>>>> its
>>>>>>>> usually been in the server layer and most often in the service
>>>>>>>> implementations that are container specific.
>>>>>>>>
>>>>>>>> The non-rpc based container definitely has some issues though because
>>>>>>>> it
>>>>>>>> sending requests to the server in a json map format... which doesn't
>>>>>>>> preserve order. So, some of the compliance tests would fail simply
>>>>>>>> because
>>>>>>>> they were fetching app data before it was updated and so forth.
>>>>>>>>
>>>>>>>> So... hopefully someone out there can get a patch to switch the php to
>>>>>>>> rpc
>>>>>>>> batching going :)
>>>>>>>>
>>>>>>>> - Cassie
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Aug 22, 2008 at 12:28 PM, Dan Peterson <[EMAIL PROTECTED]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> By the way, the docs for the compliance test suite are at:
>>>>>>>>
>>>>>>>>> http://code.google.com/p/opensocial-resources/wiki/ComplianceTests
>>>>>>>>>
>>>>>>>>> -Dan
>>>>>>>>> On Fri, Aug 22, 2008 at 12:22 PM, Louis Ryan <[EMAIL PROTECTED]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Im seeing some similar issues. One thing I noticed is that lookingFor
>>>>>>>>>
>>>>>>>>>> is
>>>>>>>>>> now
>>>>>>>>>> an Enum in JS but its not the Java datamodel. Im going to fix that
>>>>>>>>>> one.
>>>>>>>>>>
>>>>>>>>>> On Fri, Aug 22, 2008 at 11:54 AM, Chris Chabot <[EMAIL PROTECTED]>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> When running the compliance test suite:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> http://opensocial-resources.googlecode.com/svn/tests/trunk/suites/0.7/compliance/reference/reference.xml
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> I get 28 failed on my live version of partuza+php shindig (
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> www.partuza.nl is
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> running a checkout that is about 1.5 weeks old), while the latest
>>>>>>>>>>> code
>>>>>>>>>>> locally gives me 42 errors.
>>>>>>>>>>>
>>>>>>>>>>> To rule out that it wasn't the php code, i updated just shindig/php
>>>>>>>>>>> on
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> live server, and the error count didn't change, so it's probably
>>>>>>>>>>> some
>>>>>>>>>>> shindig//features/* changes that cause this.
>>>>>>>>>>>
>>>>>>>>>>> Anyone checking if our JS code is 'compliant' ? And/or working on
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> fixing
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> it? Seems right now it's only getting less so :)
>>>>>>>>>>>
>>>>>>>>>>> -- Chris
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>
>>
>
>
>
> --
> regards,
> Shreyas
>
--
regards,
Shreyas
