Thanks for the great work Shreyas,
That really improves things quite a bit, 18 isn't a perfect score yet,
but I'm sure there's a few things on my side too that i need to fix;
Which will be a bit easier now! :)
-- Chris
On Aug 25, 2008, at 10:41 PM, Shreyas Desai wrote:
Chris,
Thanks for trying the gadget. It uncovered couple wrong API use. I
fixed tehm in the gadget. Rightnow this is the summary for partuza.nl
Passed Failed Warnings Unverified Total
116 18 5 23 162
It's much cleaner now :-)
PPLX01 - throws exception : line 115 : peoplesuite.js
PPL004 - gives false for
opensocial
.getEnvironment
().supportsField(opensocial.Environment.ObjectType.ADDRESS,
"familyName"); but it also returns value which is contradictory. Test
expects true for any field which has value.
Opensocial spec 0.7 Definition :: SPEC-DEF-TEST - Need to fix the spec
gadgets.util.* Test Suite :: UTIL003 - container returns false for
opensocial-0.7, setPrefs etc.
AppData Test Suite :: APP010 -
AppData Test Suite :: APP103
AppData Test Suite :: APP302 -
AppData Test Suite :: APP008 -
in above cases it gives Internal server error in error message but
hadError is always false.
AppData Test Suite :: APP300 - it expects error because we are
making call
req.add(req.newFetchPersonAppDataRequest('VIEWER_OWNER_ABC',
[this.id + '_dataKey1']), 'invalidFetchRequest');
'VIEWER_OWNER_ABC' is invalid id.
MakeRequest Test Suite :: SMKRT001 - data.validated = false
MakeRequest Test Suite :: MKRT005 - Json parse error
MakeRequest Test Suite :: MKRT011 - json parse error
gadget.Prefs Test Suite :: PREF010 - prefs are not honored for all the
cases below
gadget.Prefs Test Suite :: PREF020
gadget.Prefs Test Suite :: PREF030
gadget.Prefs Test Suite :: PREF040
gadget.Prefs Test Suite :: PREF050
I believe that gadgets spec is very little documented and requires
some brush up. But that's definitely not in the scope of compliance
gadget.
regards,
Shreyas
- Show quoted text -
On Mon, Aug 25, 2008 at 9:53 AM, Dan Peterson <[EMAIL PROTECTED]>
wrote:
Hey folks,
For suggestions to improve the compliance testing suite, I'd highly
encourage you to write to the spec list with feedback.
-Dan
On Mon, Aug 25, 2008 at 2:16 AM, Chris Chabot <[EMAIL PROTECTED]>
wrote:
Ps i also tried the archived old test gadget (that I'm kinda
missing right
now) at
http://opensocial-resources.googlecode.com/svn/tests/trunk/archive/compliancetests.xml
However that seems to error out now too (java + samplecontainer &
on php
shindig + partuza as well) on "request create activity" (x2),
"update person
app data" and "people field return types".
In other words, I'm having a bit of a rough time testing stuff
while trying
to code up a json-rpc interface in php.. if everything is 'red',
it's a good
bit harder to verify, and i simply don't have the time right now
to write my
own gadget test code combined with the php code.
I imagine containers currently working to implement shindig (and
there's a
few of those i know of) might run into these issues as well
-- Chris
On Aug 25, 2008, at 11:09 AM, Kevin Brown wrote:
On Mon, Aug 25, 2008 at 2:05 AM, Chris Chabot <[EMAIL PROTECTED]>
wrote:
When you load up the reference test suite in the java sample
container the
result is:
94 Passed
47 Failed
2 Warnings
22 Unverified
now 47 is a bit steep, especially considering that this uses
java's
json-rpc interface, so instruction ordering shouldn't be an
issue here.
A fair bit of those are silly errors (like the empty proxy
string), or
errors like "expected 'gadgets.Tab', got " 'gadgets.[object
Object],<spam>'". and some errors that make no sense to me
"[PPL005.1]
Nonsupported Field - familyName: FAILED: (got 'Doe'), expected
'undefined'"
(name is supported, so why complain you got a familyName?)
However that doesn't account for all 47 errors, there's a few
real ones
in
there too, and it's currently quite hard to separate the real
failures
from
the ones that don't really matter.
I'm slightly concerned that with such a volume of errors (wether
they are
real errors or not), the tool looses it's usefulness. I mean if
someone
checks out shindig, implements the basic services and runs the
test suite
to
see if they did that correctly .... How would one not completely
familiar
with the complete opensocial stack be able to diagnose what is
or isn't
the
fault of their own code? A needle and haystack come to mind :)
I agree, which is why we should talk to the people writing the
compliance
gadget and get rid of the unnecessary stuff first, then we can
look for
real
problems.
-- Chris
On Aug 25, 2008, at 4:58 AM, Cassie wrote:
Ahh - disclaimer for my last statement - I was only talking
about the
opensocial related tests... I don't usually check the non-
social ones
:)
- Cassie
On Sat, Aug 23, 2008 at 4:46 PM, Chris Chabot <[EMAIL PROTECTED]>
wrote:
Don't suppose these are easy to fix on the javascript side?
gadgets.io.* TestSuite:
Description> Tests if we can get the proxy URL with given URL
as proxy
[GIO101.0] gadgets.io.getProxyUrl(String) - With valid URL.:
PASS: (got
'proxy?refresh=3600&url=http%3A%2F%2Fwww%2F~user')
[GIO101.1] gadgets.io.getProxyUrl(String) - With valid URL.:
PASS: (got
'proxy?refresh=3600&url=http%3A%2F%2Fwww%2F~user')
[GIO102] [P2 ]:: gadgets.io.getProxyUrl() - With no
parameters.: FAILED
[458
ms]
Description> Tests if we can call getProxyUrl API without any
parameter
and
it returns the proxy URL with empty proxy
[GIO102.0] gadgets.io.getProxyUrl() - With no parameters.:
PASS: (got
'proxy?refresh=3600&url=undefined')
[GIO102.1] gadgets.io.getProxyUrl() - With no parameters.:
FAILED: (got
'proxy?refresh=3600&url=undefined'), expected 'url='
[GIO103] [P2 ]:: gadgets.io.getProxyUrl(null) - With null
parameter.:
FAILED
[462 ms]
Description> Tests if we can call getProxyUrl API with null as
parameter
and
it returns the proxy URL with empty proxy
[GIO103.0] gadgets.io.getProxyUrl(null) - With null
parameter.: PASS:
(got
'proxy?refresh=3600&url=null')
[GIO103.1] gadgets.io.getProxyUrl(null) - With null
parameter.: FAILED:
(got
'proxy?refresh=3600&url=null'), expected 'url='
[GIO104] [P2 ]:: gadgets.io.getProxyUrl(undefined) - With
undefined
parameter.: FAILED [466 ms]
Description> Tests if we can call getProxyUrl API with
undefined as
parameter and it returns the proxy URL with empty proxy
[GIO104.0] gadgets.io.getProxyUrl(undefined) - With undefined
parameter.:
PASS: (got 'proxy?refresh=3600&url=undefined')
[GIO104.1] gadgets.io.getProxyUrl(undefined) - With undefined
parameter.:
FAILED: (got 'proxy?refresh=3600&url=undefined'), expected
'url='
I'm just a sucker for seeing green boxes is all :)
-- Chris
On Aug 22, 2008, at 11:04 PM, Cassie wrote:
I check the compliance tests regularly for the actual
deployment of
Shindig
that I work on at work. We are failing more now only because
the tests
are
getting much more thorough. (The tests are also very active so
sometimes
they have bugs too although it is usually our code that's
wrong :)
I haven't found many issues with Shindig's actual js layer
though -
its
usually been in the server layer and most often in the service
implementations that are container specific.
The non-rpc based container definitely has some issues though
because
it
sending requests to the server in a json map format... which
doesn't
preserve order. So, some of the compliance tests would fail
simply
because
they were fetching app data before it was updated and so forth.
So... hopefully someone out there can get a patch to switch
the php to
rpc
batching going :)
- Cassie
On Fri, Aug 22, 2008 at 12:28 PM, Dan Peterson <[EMAIL PROTECTED]
>
wrote:
By the way, the docs for the compliance test suite are at:
http://code.google.com/p/opensocial-resources/wiki/ComplianceTests
-Dan
On Fri, Aug 22, 2008 at 12:22 PM, Louis Ryan
<[EMAIL PROTECTED]>
wrote:
Im seeing some similar issues. One thing I noticed is that
lookingFor
is
now
an Enum in JS but its not the Java datamodel. Im going to
fix that
one.
On Fri, Aug 22, 2008 at 11:54 AM, Chris Chabot <[EMAIL PROTECTED]
>
wrote:
When running the compliance test suite:
http://opensocial-resources.googlecode.com/svn/tests/trunk/suites/0.7/compliance/reference/reference.xml
I get 28 failed on my live version of partuza+php shindig (
www.partuza.nl is
running a checkout that is about 1.5 weeks old), while the
latest
code
locally gives me 42 errors.
To rule out that it wasn't the php code, i updated just
shindig/php
on
the
live server, and the error count didn't change, so it's
probably
some
shindig//features/* changes that cause this.
Anyone checking if our JS code is 'compliant' ? And/or
working on
fixing
it? Seems right now it's only getting less so :)
-- Chris
--
regards,
Shreyas
