Two thoughts: One thing that does work with other game/crowdsource applications (like some of the planethunter derived ones) is some kind of graphic interaction, rather than form filling. "Draw a box around a list of subsidiaries" or just "put an X where they start" is much more "fun" for people who like that sort of thing and would get you some data which you could feed into "game 2" where people who like form filling could get going.
Second, your blurb says your robot can guess things, but you don't know if they are right. That usually signals a game where you present partial (guessed) information for quality checking and/or upgrading, but in this as far as I can see we get nothing. If you _were_ able to present guessed information that might help (I expect you have thought of this and rejected it) but if you aren't going to, some expectation management in the lead might be useful. Francis 2013/10/4 Seb Bacon <[email protected]> > Thanks for the detailed feedback. > > Yes, there actually is a standard place to list subsidiaries - > "exhibit 21a". However, in pretty much all cases in this crowdsourcing > game, the reason they're in the game is because we couldn't scrape > them, and that's usually because there's no exhibit 21a. > > Your other points (varieties, I think, of "it's more complicated than > the form allows me to report") illustrate a real problem with > crowdsourcing like this. Do we add a button for "It explicitly states > there are no subsidiaries", and another for "It doesn't mention > subsidiaries"? How about a separate confidence level for jurisdiction > as well as name? > > Eventually we end up with a forest of buttons and options, and we > quickly end up with a game that only people who are detailed and > dedicated are going to be able to consider playing. It stops being a > game, and just becomes a "relationship editing tool" (which we are > developing at the same time, but trying to work out how we induct > people into using it -- volunteers / ideas welcome!) > > There should be a place for crowdsourcing for lower quality, higher > quantity data extraction, but the only way I can think of removing > these grey areas / cognitive blocks would be to decompose the task > into much smaller parts (maybe "draw a box around any sentence > mentioning subsidiaries", "click boxes mentioning subsidiaries and > jurisdictions" or something like that). Even then I wonder if a > crowdsourcing game is even going to much good at this kind of natural > language parsing. And it would take vast amounts of work to get right > for reducing returns. > > On the other hand, I hope we can tweak the current game so it's clear > that if you're not sure, or you hit any block at all, it's OK just to > continue. I think that should work OK, but haven't yet found a way to > make it really clear in the UI. > > Seb > > > On 3 October 2013 19:21, James McKinney <[email protected]> wrote: > > I continued playing until I found a document with relevant info near the > search term "subsidiar", and it seems there is no standard place to > disclose subsidiaries. For example, the following document has a single > phrase at the start of Part 1 mentioning the company's two wholly owned > subsidiaries (I didn't check the document to see if it discloses any > non-wholly-owned subsidiaries). > > > > > http://www.sec.gov/Archives/edgar/data/1349976/000114420413044838/v350454_10k.htm > > > > I've found a document where in Item 15 it lists "Active Subsidiaries of > the Registrant", which can then be searched for in the document to > determine ownership %. Another document calls it "List of Subsidiaries" in > Item 15. A strange thing is that a subsidiary StarTrak Systems, LLC was > sold to ORBCOMM Inc., yet it's also described as being a wholly-owned > subsidiary - my understanding is that the deal has not yet been closed. > > > > > http://www.sec.gov/Archives/edgar/data/98618/000009861813000011/k10_063013.htm > > > > The same document lists subsidiaries under a "Principles of > Consolidation" heading, in case that heading appears in other documents. > Similar headings in further documents include "Basis of Presentation" and > "Basis of Presentation and Consolidation". It may be worthwhile to compile > a list of these. "Percentage of Ownership" seems to be a popular column > heading. > > > > It may be worthwhile to add a field to the data collection form to > express the user's certainty as to whether all wholly-owned subsidiaries > mentioned in the document have been collected. For example, one document in > the "forward looking statements" before Part 1 explicitly says that the > company has five wholly-owned subsidiaries, which makes me certain that > I've found all five: > > > > > http://www.sec.gov/Archives/edgar/data/1425173/000101968713003726/novagen_10k-123112.htm > > > > Another document helpfully says "We do not have any subsidiaries." but > it seems most are not so helpful. If there is no mention of subsidiaries, > does that mean there are no subsidiaries? Should a button be added for > cases where no info can be found, or should I just click the "it's > complicated" button? In once case the list is described as "previously > filed", with no other mention of subsidiaries. > > > > Similarly, this document says it has subsidiaries, but doesn't name > them. Should a button be added to report these cases, or should I just > click the "it's complicated" button? > http://www.sec.gov/Archives/edgar/data/1435936/000143593613000153/f10k_20130531r5.htm > > > > Have you noticed cases where in one part of the document, a subsidiary > is described as wholly-owned, but is later described as not wholly-owned? > e.g. How much attention should be paid to verb tenses? > > > > Some documents don't give the jurisdiction of subsidiaries. What to do? > I can't submit the form without filling in the jurisdiction, but I would > expect that a partially filled form is better than no form? For example: > > > > > http://www.sec.gov/Archives/edgar/data/1020477/000135448813004921/robk_10k.htm > > > http://www.sec.gov/Archives/edgar/data/825322/000121390013005438/f10k2013_mphase.htm > > > http://www.sec.gov/Archives/edgar/data/896429/000118811213002827/t77452_10k.htm > > > > The instructions should perhaps warn that some documents mention the > subsidiaries of other companies, so it's important to check that you're > reporting about the right company. Most confusingly, the following document > starts talking about companies with entirely different names as though they > are the subject of the document: > > > > > http://www.sec.gov/Archives/edgar/data/1496741/000116169713000700/form_10-k.htm > > > > Lastly, I'd eliminate one click and put the "Step 1" content after the > other content on "Step 3". > > > > James > > > > On 2013-10-03, at 1:09 PM, James McKinney wrote: > > > >> Hi Seb, > >> > >> I've tried the game, but I'm getting very long (50+ page) documents > that don't seem to have any relevant information near search terms like > "subsidiar", "own", "wholly", etc. It may be the case that these documents > have no subsidiary information. However, I also notice that the two > documents I've looked at so far have identical tables of contents. Could > you provide instructions as to which section(s) within this (at least > somewhat) standard table of contents would contain the relevant > information, to avoid having to carefully read 50+ pages? > >> > >> Examples: > >> > >> > http://www.sec.gov/Archives/edgar/data/50292/000091431713000879/form10k-131780_ieh.htm > >> > http://www.sec.gov/Archives/edgar/data/874710/000114420413052878/v355226_10k.htm > >> > >> Thanks! > >> > >> James > >> > >> On 2013-10-03, at 7:39 AM, Seb Bacon wrote: > >> > >>> Hi all > >>> > >>> If you fancy helping turn unscrapeable info about corporate control > >>> networks into open data, consider having a go at our > >>> fun-for-all-the-family game! > >>> > >>> Background: > >>> > http://blog.opencorporates.com/2013/10/02/help-uncover-corporate-networks/ > >>> > >>> Game: > >>> http://opencorporates.com/games/secfilings > >>> > >>> We know it's a bit rough around the edges, but our goal with this > >>> iteration was to make it good enough that 100 people could > >>> successfully use it to add relationships. If you can give it a go, > >>> ask your partner, your children, your grandparents, that would be > >>> great! > >>> > >>> Feedback to [email protected] please > >>> > >>> Thanks > >>> > >>> Seb > >>> > >>> > >>> -- > >>> skype: seb.bacon > >>> mobile: 07790 939224 > >>> land: 01531 671074 > >>> > >>> _______________________________________________ > >>> developers-public mailing list > >>> [email protected] > >>> > https://secure.mysociety.org/admin/lists/mailman/listinfo/developers-public > >>> > >>> Unsubscribe: > https://secure.mysociety.org/admin/lists/mailman/options/developers-public/james%40opennorth.ca > >> > > > > > > -- > skype: seb.bacon > mobile: 07790 939224 > land: 01531 671074 > > _______________________________________________ > developers-public mailing list > [email protected] > https://secure.mysociety.org/admin/lists/mailman/listinfo/developers-public > > Unsubscribe: > https://secure.mysociety.org/admin/lists/mailman/options/developers-public/fjmd1a%40gmail.com > -- Francis Davey
_______________________________________________ developers-public mailing list [email protected] https://secure.mysociety.org/admin/lists/mailman/listinfo/developers-public Unsubscribe: https://secure.mysociety.org/admin/lists/mailman/options/developers-public/archive%40mail-archive.com
