Send Link mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://mailman.anu.edu.au/mailman/listinfo/link
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Link digest..."
Today's Topics:
1. SAI Generated Instructions (Tom Worthington)
2. Re: SAI Generated Instructions (Roger Clarke)
3. Re: SAI Generated Instructions (Tom Worthington)
4. Re: SAI Generated Instructions (Roger Clarke)
----------------------------------------------------------------------
Message: 1
Date: Tue, 06 May 2025 19:13:30 +1000
From: "Tom Worthington" <[email protected]>
To: [email protected]
Subject: [LINK] SAI Generated Instructions
Message-ID: <[email protected]>
Content-Type: text/plain
Speaking at the Australian Computer Society Canberra Branch this evening, Harry
Hoang from Tailored Accounts Trude, asked his staff to use two AI systems to
write a step by step process to apply for a complicated government tax
deduction. Both provided an accurate useful guide. He emphasized these systems
did not need the back and forth of Chat GPT: one request gave a useful answer.
Genspark and Felo were the AI tools.
https://blog.highereducationwhisperer.com/2025/05/ai-generated-instructions.html
--
Tom Worthington
------------------------------
Message: 2
Date: Tue, 6 May 2025 21:03:00 +1000
From: Roger Clarke <[email protected]>
To: [email protected]
Subject: Re: [LINK] SAI Generated Instructions
Message-ID: <[email protected]>
Content-Type: text/plain; charset=UTF-8; format=flowed
On 6/5/2025 19:13, Tom Worthington wrote:
> Speaking at the Australian Computer Society Canberra Branch this evening,
> Harry Hoang from Tailored Accounts Trude, asked his staff to use two AI
> systems to write a step by step process to apply for a complicated government
> tax deduction. Both provided an accurate useful guide. He emphasized these
> systems did not need the back and forth of Chat GPT: one request gave a
> useful answer. Genspark and Felo were the AI tools.
> https://blog.highereducationwhisperer.com/2025/05/ai-generated-instructions.html
I'm a pedant. (No kidding?). So I have to ask.
Testing theory (and there is some) says you need to specify the 'right'
answer (grey zones permitted) *in advance of* evaluating the output.
Did someone apply that criterion?
You've covered yourself with "[it] gave a useful answer".
Me too, with weasel words like:
"[GenAI produces] synthetic text that is likely to appear authoritative,
authentic and convincing to the person who asked the question"
and
" ... the promptness, and the contrived fluidity and authoritativeness
of expression of the responses, may in many cases cause the individual
to suspend their disbelief, and uncautiously embrace the responses as
being reliable".
But a retrospective 'well, I don't think I'd have expected that' is very
closely related to genuinely believing that the pea just *has* to be
under *that* thimble.
The sceptic reckons that the real magic in GenAI is that being conned no
longer depends on the skills of a con-man. Stochastic parrots rule.
Quotes from:
Principles for the Responsible Application of Generative AI
Forthcoming in Computer Law & Security Review
https://rogerclarke.com/EC/RGAI-C.html
Please tear it to shreds.
I need to present it shortly, and forewarned is forearmed.
--
Roger Clarke mailto:[email protected]
T: +61 2 6288 6916 http://www.xamax.com.au http://www.rogerclarke.com
Xamax Consultancy Pty Ltd 78 Sidaway St, Chapman ACT 2611 AUSTRALIA
Visiting Professorial Fellow UNSW Law & Justice
Visiting Professor in Computer Science Australian National University
------------------------------
Message: 3
Date: Wed, 7 May 2025 09:32:01 +1000
From: Tom Worthington <[email protected]>
To: [email protected]
Subject: Re: [LINK] SAI Generated Instructions
Message-ID: <[email protected]>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
On 5/6/25 21:03, Roger Clarke wrote:
> ... you need to specify the 'right'
> answer ... *in advance of* evaluating the output. ...
The test was evaluation by an expert in the subject matter area, in this
case a tax accountant. They concluded the output was accurate and
suitable for presenting to clients.
> You've covered yourself with "[it] gave a useful answer".
Isn't useful all we need from AI?
--
Tom Worthington http://www.tomw.net.au
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 665 bytes
Desc: OpenPGP digital signature
URL:
<https://mailman.anu.edu.au/pipermail/link/attachments/20250507/1dfb3fee/attachment-0001.sig>
------------------------------
Message: 4
Date: Wed, 7 May 2025 10:19:04 +1000
From: Roger Clarke <[email protected]>
To: [email protected]
Subject: Re: [LINK] SAI Generated Instructions
Message-ID: <[email protected]>
Content-Type: text/plain; charset=UTF-8; format=flowed
> On 5/6/25 21:03, Roger Clarke wrote:
>> ... you need to specify the 'right' answer ... *in advance of*
>> evaluating the output. ...
On 7/5/2025 09:32, Tom Worthington wrote:
> The test was evaluation by an expert in the subject matter area, in this
> case a tax accountant. They concluded the output was accurate and
> suitable for presenting to clients.
Not bad, but not compliant with the principle ...
>> You've covered yourself with "[it] gave a useful answer".
>
> Isn't useful all we need from AI?
Nope.
Useful = "able to be used for a practical purpose".
There's no element of reliability in that property.
Without data quality factors being satisfied (appropriate association,
accuracy, timeliness, etc. - a dozen factors in all), mere usefulness
(which I'd equate to 'apparent relevance') isn't enough.
http://www.rogerclarke.com/ID/FDIS.html#Q
--
Roger Clarke mailto:[email protected]
T: +61 2 6288 6916 http://www.xamax.com.au http://www.rogerclarke.com
Xamax Consultancy Pty Ltd 78 Sidaway St, Chapman ACT 2611 AUSTRALIA
Visiting Professorial Fellow UNSW Law & Justice
Visiting Professor in Computer Science Australian National University
------------------------------
Subject: Digest Footer
_______________________________________________
Link mailing list
[email protected]
https://mailman.anu.edu.au/mailman/listinfo/link
------------------------------
End of Link Digest, Vol 390, Issue 5
************************************
