[Wikimedia-l] Re: ChatGPT as a reliable source
FYI, yesterday I stumbled upon Perplexity <https://www.perplexity.ai/>, an AI that cites its sources for its answers. After a couple tests, I'm not convinced on how tight the connection is between the generated text and the sources, but they seem at least to broadly support the claims. On Thu, May 18, 2023 at 8:30 AM Peter Southwood < peter.southw...@telkomsa.net> wrote: > It depends on how much you know about the topic, Both methods have their > advantages. > > Cheers, > > Peter > > > > *From:* Todd Allen [mailto:toddmal...@gmail.com] > *Sent:* 17 May 2023 20:10 > *To:* Wikimedia Mailing List > *Subject:* [Wikimedia-l] Re: ChatGPT as a reliable source > > > > Though, this does run the risk of encouraging people to take the > "backwards" approach to writing an article--writing some stuff, and then > (hopefully at least) trying to come up with sources for it. > > > > The much superior approach is to locate the available sources first, and > then to develop the article based upon what those sources say. > > > > Todd > > > > On Wed, May 17, 2023 at 12:06 PM Samuel Klein wrote: > > > > First: Wikipedia style for dense inline citations is one of the most > granular and articulate around, so we're pushing the boundaries in some > cases of research norms for clarity in sourcing. That's great; also means > sometimes we are considering nuances that may be new. > > > > Second: We're approaching a topic close to my heart, which is > distinguishing reference-sources from process-sources. Right now we often > capture process sources (for an edit) in the edit summary, and this is not > visible anywhere on the resulting article. Translations via a translate > tool; updates by a script that does a particular class of work (like > spelling or grammer checking); applying a detailed diff that was > workshopped on some other page. An even better interface might allow for > that detail to be visible to readers of the article [w/o traversing the > edit history], and linked to the sections/paragraphs/sentences affected. > > > > I think any generative tools used to rewrite a section or article, or to > produce a sibling version for a different reading-level, or to generate a > timeline or other visualization that is then embedded in the article, > should all be cited somehow. To Jimbo's point, that doesn't belong in a > References section as we currently have them. But I'd like to see us > develop a way to capture these process notes in a more legible way, so > readers can discover them without browsing the revision history. > > > > People using generative tools to draft new material should find reliable > sources for every claim in that material, much more densely than you would > when summarizing a series of sources yourself. > > However, as we approach models that can discover sources and check facts, > a combination of those with current generative tools could produce things > closer to what we'd consider acceptable drafts, and at scale could generate > reference works in languages that lack them. I suggest a separate project > for those as the best way to explore the implications of being able to do > this at scale, and should capture the full model/tuning/prompt details of > how each edit was generated. Such an automatically-updated resource would > not be a good reliable source, just as we avoid citing any tertiary > sources, but could be a research tool for WP editors and modelers alike. > > > > SJ > > > > On Wed, May 17, 2023 at 9:27 AM Jimmy Wales > wrote: > > One way I think we can approach this is to think of it as being the latest > in this progression: > > spellchecker -> grammar checker -> text generation support > > We wouldn't have any sort of footnote or indication of any kind that a > spellchecker or grammar checker was > used by an editor, it's just built-in to many writing tools. Similarly, > if writing a short prompt to generate a longer > text is used, then we have no reason to cite that. > > What we do have, though, is a responsibility to check the output. > Spellcheckers can be wrong (suggesting the correct > spelling of the wrong word for example). Grammar checkers can be wrong > (trying to correct the grammar of a direct quote > for example). Generative AI models can be wrong - often simply making > things up out of thin air that sound plausible. > > If someone uses a generative AI to help them write some text, that's not a > big deal. If they upload text without checking > the facts and citing a real source, that's very bad. > > > > On 2023-05-17 11:51, The Cunctator wrote: > > Again at no point should even an improve
[Wikimedia-l] Re: ChatGPT as a reliable source
(Apologies, accidentally deleted, content recovered) From: Jimmy Wales To: wikimedia-l@lists.wikimedia.org Cc: Bcc: Date: Thu, 18 May 2023 14:48:06 +0100 Subject: Re: [Wikimedia-l] Re: ChatGPT as a reliable source On 2023-05-17 19:05, Samuel Klein wrote: > > I think any generative tools used to rewrite a section or article, or > to produce a sibling version for a different reading-level, or to > generate a timeline or other visualization that is then embedded in > the article, should all be cited somehow. While I don't have anything against that, obviously, I'm not really convinced that we need to do this. I suppose it depends on the context. If a non-native speaker of a language uses a spell checker, we don't ask them to even mention it. If they use a more sophisticated grammar tool to help them with some nuance of that language, we don't ask them to even mention it. If they use an AI tool to evaluate and edit their paragraph for tone? What if they use an AI tool to compare the text they are writing with the source being cited, to see if the AI notices any discrepancies? I feel that those last two use cases are going to be ubiquitous within a couple of years, possibly even embedded in browsers or browser extensions. > People using generative tools to draft new material should find > reliable sources for every claim in that material, much more densely > than you would when summarizing a series of sources yourself. This is definitely true. ___ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/YZI7OA322C5PBICZ373NXCET6HUS33XW/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org
[Wikimedia-l] Re: ChatGPT as a reliable source
It depends on how much you know about the topic, Both methods have their advantages. Cheers, Peter From: Todd Allen [mailto:toddmal...@gmail.com] Sent: 17 May 2023 20:10 To: Wikimedia Mailing List Subject: [Wikimedia-l] Re: ChatGPT as a reliable source Though, this does run the risk of encouraging people to take the "backwards" approach to writing an article--writing some stuff, and then (hopefully at least) trying to come up with sources for it. The much superior approach is to locate the available sources first, and then to develop the article based upon what those sources say. Todd On Wed, May 17, 2023 at 12:06 PM Samuel Klein wrote: First: Wikipedia style for dense inline citations is one of the most granular and articulate around, so we're pushing the boundaries in some cases of research norms for clarity in sourcing. That's great; also means sometimes we are considering nuances that may be new. Second: We're approaching a topic close to my heart, which is distinguishing reference-sources from process-sources. Right now we often capture process sources (for an edit) in the edit summary, and this is not visible anywhere on the resulting article. Translations via a translate tool; updates by a script that does a particular class of work (like spelling or grammer checking); applying a detailed diff that was workshopped on some other page. An even better interface might allow for that detail to be visible to readers of the article [w/o traversing the edit history], and linked to the sections/paragraphs/sentences affected. I think any generative tools used to rewrite a section or article, or to produce a sibling version for a different reading-level, or to generate a timeline or other visualization that is then embedded in the article, should all be cited somehow. To Jimbo's point, that doesn't belong in a References section as we currently have them. But I'd like to see us develop a way to capture these process notes in a more legible way, so readers can discover them without browsing the revision history. People using generative tools to draft new material should find reliable sources for every claim in that material, much more densely than you would when summarizing a series of sources yourself. However, as we approach models that can discover sources and check facts, a combination of those with current generative tools could produce things closer to what we'd consider acceptable drafts, and at scale could generate reference works in languages that lack them. I suggest a separate project for those as the best way to explore the implications of being able to do this at scale, and should capture the full model/tuning/prompt details of how each edit was generated. Such an automatically-updated resource would not be a good reliable source, just as we avoid citing any tertiary sources, but could be a research tool for WP editors and modelers alike. SJ On Wed, May 17, 2023 at 9:27 AM Jimmy Wales wrote: One way I think we can approach this is to think of it as being the latest in this progression: spellchecker -> grammar checker -> text generation support We wouldn't have any sort of footnote or indication of any kind that a spellchecker or grammar checker was used by an editor, it's just built-in to many writing tools. Similarly, if writing a short prompt to generate a longer text is used, then we have no reason to cite that. What we do have, though, is a responsibility to check the output. Spellcheckers can be wrong (suggesting the correct spelling of the wrong word for example). Grammar checkers can be wrong (trying to correct the grammar of a direct quote for example). Generative AI models can be wrong - often simply making things up out of thin air that sound plausible. If someone uses a generative AI to help them write some text, that's not a big deal. If they upload text without checking the facts and citing a real source, that's very bad. On 2023-05-17 11:51, The Cunctator wrote: Again at no point should even an improved version be considered a source; at best it would be a research or editing tool. On Wed, May 17, 2023, 4:40 AM Lane Chance wrote: Keep in mind how fast these tools change. ChatGPT, Bard and competitors understand well the issues with lack of sources, and Bard does sometimes put a suitable source in a footnote, even if it (somewhat disappointingly) just links to wikipedia. There's likely to be a variation soon that does a decent job of providing references, and at that point the role of these tools moves beyond being an amusement to a far more credible research tool. So, these long discussions about impact on open knowledge are quite likely to have to run again in 2024... On Wed, 17 May 2023 at 09:24, Kiril Simeonovski wrote: > > Thank you everyone for your input. > > Your considerations are very similar to mine, and they give a cle
[Wikimedia-l] Re: ChatGPT as a reliable source
On Wed, 17 May 2023 at 08:08, Kiril Simeonovski wrote: > Can ChatGPT be used as a reliable source and, if yes, how would the citation > look like? I asked Google Bard "who is Andy Mabbett" the other day. In three short paragraphs, there were five serious mistakes, including the wrong year of birth; the wrong subject for my degree, and the completely false: He is best known for his work on the Wikipedia article about Pink Floyd, which he has been the primary editor of since 2007. -- Andy Mabbett @pigsonthewing http://pigsonthewing.org.uk ___ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/4DIIPU36MOBYWFRHT2H7CFCTVDT4BXRZ/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org
[Wikimedia-l] Re: ChatGPT as a reliable source
Agreed. The editor is responsible for their edits, if they fail to provide suitable sourcing, misrepresent the cited source, plagiarise or infringe on copyright that is on them. Do it too often and they get banned. We don’t need to know how they composed the content, so we shouldn’t care. Competence is required. Using a different tool just needs a slightly different competence. Like a chainsaw instead of an axe Cheers, Peter. From: Denny Vrandečić [mailto:vrande...@gmail.com] Sent: 18 May 2023 01:35 To: Wikimedia Mailing List Subject: [Wikimedia-l] Re: ChatGPT as a reliable source I think Jimmy's proposal is spot on. A generative AI is a tool, and whoever makes the edit is fully responsible for the edit, no matter whether the text was written by the person or with the help of a generative tool. This has the potential to open us for people who are not good at formulating, or who are not confident about their writing. As long as they completely take responsibility for the written text, all is fine. This is similar to the approach the ACM has taken for AI generated text. They decided that a generative model cannot be a co-author as it lacks the ability to be morally responsible for the text. Second, anything that you publish under your name is your responsibility. On Wed, May 17, 2023 at 11:11 AM Todd Allen wrote: Though, this does run the risk of encouraging people to take the "backwards" approach to writing an article--writing some stuff, and then (hopefully at least) trying to come up with sources for it. The much superior approach is to locate the available sources first, and then to develop the article based upon what those sources say. Todd On Wed, May 17, 2023 at 12:06 PM Samuel Klein wrote: First: Wikipedia style for dense inline citations is one of the most granular and articulate around, so we're pushing the boundaries in some cases of research norms for clarity in sourcing. That's great; also means sometimes we are considering nuances that may be new. Second: We're approaching a topic close to my heart, which is distinguishing reference-sources from process-sources. Right now we often capture process sources (for an edit) in the edit summary, and this is not visible anywhere on the resulting article. Translations via a translate tool; updates by a script that does a particular class of work (like spelling or grammer checking); applying a detailed diff that was workshopped on some other page. An even better interface might allow for that detail to be visible to readers of the article [w/o traversing the edit history], and linked to the sections/paragraphs/sentences affected. I think any generative tools used to rewrite a section or article, or to produce a sibling version for a different reading-level, or to generate a timeline or other visualization that is then embedded in the article, should all be cited somehow. To Jimbo's point, that doesn't belong in a References section as we currently have them. But I'd like to see us develop a way to capture these process notes in a more legible way, so readers can discover them without browsing the revision history. People using generative tools to draft new material should find reliable sources for every claim in that material, much more densely than you would when summarizing a series of sources yourself. However, as we approach models that can discover sources and check facts, a combination of those with current generative tools could produce things closer to what we'd consider acceptable drafts, and at scale could generate reference works in languages that lack them. I suggest a separate project for those as the best way to explore the implications of being able to do this at scale, and should capture the full model/tuning/prompt details of how each edit was generated. Such an automatically-updated resource would not be a good reliable source, just as we avoid citing any tertiary sources, but could be a research tool for WP editors and modelers alike. SJ On Wed, May 17, 2023 at 9:27 AM Jimmy Wales wrote: One way I think we can approach this is to think of it as being the latest in this progression: spellchecker -> grammar checker -> text generation support We wouldn't have any sort of footnote or indication of any kind that a spellchecker or grammar checker was used by an editor, it's just built-in to many writing tools. Similarly, if writing a short prompt to generate a longer text is used, then we have no reason to cite that. What we do have, though, is a responsibility to check the output. Spellcheckers can be wrong (suggesting the correct spelling of the wrong word for example). Grammar checkers can be wrong (trying to correct the grammar of a direct quote for example). Generative AI models can be wrong - often simply making things up out of thin air that sound plausible. If someone uses a gener
[Wikimedia-l] Re: ChatGPT as a reliable source
Dear Wikimedians,I would like to share with you the editorial (written in French) of an academic magazine related to ChatGPT. Through what it says and quotes made in particular, I think that WE Wikimedians, just need to rethink the way we share knowledge and information through our different projects (platforms) in order to achieve our goals set for the year 2030 : To be the essential infrastructure of the free knowledge ecosystem... In bold the relevant points of the editorial. Thank you and have a nice day __ ChatGPTIt can do everything: write an essay, summarise a lecture, solve equations and answer complex questions fluently. And it can do it all so quickly. Its name? ChatGPT. This "conversational agent" based on artificial intelligence, which appeared in November 2022, is delighting students and giving teachers cold sweats because, even if it is not free of clumsiness or inaccuracies, it is revolutionising evaluation methods. So what to do? It's hard to ignore it. Censor it? According to Frédéric Schoenaers, Vice-President of Education, it is preferable to integrate it into the range of teaching practices while taking into account its high potential. "It's an opportunity to change our expectations of students, to think about how we test their knowledge." If ChatGPT can pass a test, then modify the test. Without stress. After all, the calculator hasn't replaced the maths lesson._ Le Quinzième Jour - Quadrimestriel de l'ULiège - May-August 2023 - Number 285 - ISSN : 2593-5984 - www.uliege.be/LQJGeorges Fodouop IT Manager - Wikimedia TrainerCo-founder Wikimedia CameroonUser : Geugeor Le mercredi 17 mai 2023 à 09:09:28 UTC+2, Kiril Simeonovski a écrit : Dear Wikimedians, Two days ago, a participant in one of our edit-a-thons consulted ChatGPT when writing an article on the Macedonian Wikipedia that did not exist on any other language edition. ChatGPT provided some output, but the problem was how to cite it. The community on the Macedonian Wikipedia has not yet had a discussion on this matter and we do not have any guidelines. So, my main questions are the following: * Can ChatGPT be used as a reliable source and, if yes, how would the citation look like? * Are there any ongoing community discussions on introducing guidelines? My personal opinion is that ChatGPT should be avoided as a reliable source, and only the original source where the algorithm gets the information from should be used. Best regards,Kiril___ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/WMGIBNPN5JNJGUOCLWFCCPD7EL5YN6KU/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org ___ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/HD2OI6C4B6U2PORP4SAXI7KHPTKSVY6P/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org
[Wikimedia-l] Re: ChatGPT as a reliable source
I think Jimmy's proposal is spot on. A generative AI is a tool, and whoever makes the edit is fully responsible for the edit, no matter whether the text was written by the person or with the help of a generative tool. This has the potential to open us for people who are not good at formulating, or who are not confident about their writing. As long as they completely take responsibility for the written text, all is fine. This is similar to the approach the ACM has taken for AI generated text. They decided that a generative model cannot be a co-author as it lacks the ability to be morally responsible for the text. Second, anything that you publish under your name is your responsibility. On Wed, May 17, 2023 at 11:11 AM Todd Allen wrote: > Though, this does run the risk of encouraging people to take the > "backwards" approach to writing an article--writing some stuff, and then > (hopefully at least) trying to come up with sources for it. > > The much superior approach is to locate the available sources first, and > then to develop the article based upon what those sources say. > > Todd > > On Wed, May 17, 2023 at 12:06 PM Samuel Klein wrote: > >> >> First: Wikipedia style for dense inline citations is one of the most >> granular and articulate around, so we're pushing the boundaries in some >> cases of research norms for clarity in sourcing. That's great; also means >> sometimes we are considering nuances that may be new. >> >> Second: We're approaching a topic close to my heart, which is >> distinguishing reference-sources from process-sources. Right now we often >> capture process sources (for an edit) in the edit summary, and this is not >> visible anywhere on the resulting article. Translations via a translate >> tool; updates by a script that does a particular class of work (like >> spelling or grammer checking); applying a detailed diff that was >> workshopped on some other page. An even better interface might allow for >> that detail to be visible to readers of the article [w/o traversing the >> edit history], and linked to the sections/paragraphs/sentences affected. >> >> I think any generative tools used to rewrite a section or article, or to >> produce a sibling version for a different reading-level, or to generate a >> timeline or other visualization that is then embedded in the article, >> should all be cited somehow. To Jimbo's point, that doesn't belong in a >> References section as we currently have them. But I'd like to see us >> develop a way to capture these process notes in a more legible way, so >> readers can discover them without browsing the revision history. >> >> People using generative tools to draft new material should find reliable >> sources for every claim in that material, much more densely than you would >> when summarizing a series of sources yourself. >> However, as we approach models that can discover sources and check facts, >> a combination of those with current generative tools could produce things >> closer to what we'd consider acceptable drafts, and at scale could generate >> reference works in languages that lack them. I suggest a separate project >> for those as the best way to explore the implications of being able to do >> this at scale, and should capture the full model/tuning/prompt details of >> how each edit was generated. Such an automatically-updated resource would >> not be a good reliable source, just as we avoid citing any tertiary >> sources, but could be a research tool for WP editors and modelers alike. >> >> SJ >> >> On Wed, May 17, 2023 at 9:27 AM Jimmy Wales >> wrote: >> >>> One way I think we can approach this is to think of it as being the >>> latest in this progression: >>> >>> spellchecker -> grammar checker -> text generation support >>> >>> We wouldn't have any sort of footnote or indication of any kind that a >>> spellchecker or grammar checker was >>> used by an editor, it's just built-in to many writing tools. Similarly, >>> if writing a short prompt to generate a longer >>> text is used, then we have no reason to cite that. >>> >>> What we do have, though, is a responsibility to check the output. >>> Spellcheckers can be wrong (suggesting the correct >>> spelling of the wrong word for example). Grammar checkers can be wrong >>> (trying to correct the grammar of a direct quote >>> for example). Generative AI models can be wrong - often simply making >>> things up out of thin air that sound plausible. >>> >>> If someone uses a generative AI to help them write some text, that's not >>> a big deal. If they upload text without checking >>> the facts and citing a real source, that's very bad. >>> >>> >>> On 2023-05-17 11:51, The Cunctator wrote: >>> >>> Again at no point should even an improved version be considered a >>> source; at best it would be a research or editing tool. >>> >>> On Wed, May 17, 2023, 4:40 AM Lane Chance wrote: >>> Keep in mind how fast these tools change. ChatGPT, Bard and competitors
[Wikimedia-l] Re: ChatGPT as a reliable source
Though, this does run the risk of encouraging people to take the "backwards" approach to writing an article--writing some stuff, and then (hopefully at least) trying to come up with sources for it. The much superior approach is to locate the available sources first, and then to develop the article based upon what those sources say. Todd On Wed, May 17, 2023 at 12:06 PM Samuel Klein wrote: > > First: Wikipedia style for dense inline citations is one of the most > granular and articulate around, so we're pushing the boundaries in some > cases of research norms for clarity in sourcing. That's great; also means > sometimes we are considering nuances that may be new. > > Second: We're approaching a topic close to my heart, which is > distinguishing reference-sources from process-sources. Right now we often > capture process sources (for an edit) in the edit summary, and this is not > visible anywhere on the resulting article. Translations via a translate > tool; updates by a script that does a particular class of work (like > spelling or grammer checking); applying a detailed diff that was > workshopped on some other page. An even better interface might allow for > that detail to be visible to readers of the article [w/o traversing the > edit history], and linked to the sections/paragraphs/sentences affected. > > I think any generative tools used to rewrite a section or article, or to > produce a sibling version for a different reading-level, or to generate a > timeline or other visualization that is then embedded in the article, > should all be cited somehow. To Jimbo's point, that doesn't belong in a > References section as we currently have them. But I'd like to see us > develop a way to capture these process notes in a more legible way, so > readers can discover them without browsing the revision history. > > People using generative tools to draft new material should find reliable > sources for every claim in that material, much more densely than you would > when summarizing a series of sources yourself. > However, as we approach models that can discover sources and check facts, > a combination of those with current generative tools could produce things > closer to what we'd consider acceptable drafts, and at scale could generate > reference works in languages that lack them. I suggest a separate project > for those as the best way to explore the implications of being able to do > this at scale, and should capture the full model/tuning/prompt details of > how each edit was generated. Such an automatically-updated resource would > not be a good reliable source, just as we avoid citing any tertiary > sources, but could be a research tool for WP editors and modelers alike. > > SJ > > On Wed, May 17, 2023 at 9:27 AM Jimmy Wales > wrote: > >> One way I think we can approach this is to think of it as being the >> latest in this progression: >> >> spellchecker -> grammar checker -> text generation support >> >> We wouldn't have any sort of footnote or indication of any kind that a >> spellchecker or grammar checker was >> used by an editor, it's just built-in to many writing tools. Similarly, >> if writing a short prompt to generate a longer >> text is used, then we have no reason to cite that. >> >> What we do have, though, is a responsibility to check the output. >> Spellcheckers can be wrong (suggesting the correct >> spelling of the wrong word for example). Grammar checkers can be wrong >> (trying to correct the grammar of a direct quote >> for example). Generative AI models can be wrong - often simply making >> things up out of thin air that sound plausible. >> >> If someone uses a generative AI to help them write some text, that's not >> a big deal. If they upload text without checking >> the facts and citing a real source, that's very bad. >> >> >> On 2023-05-17 11:51, The Cunctator wrote: >> >> Again at no point should even an improved version be considered a source; >> at best it would be a research or editing tool. >> >> On Wed, May 17, 2023, 4:40 AM Lane Chance wrote: >> >>> Keep in mind how fast these tools change. ChatGPT, Bard and >>> competitors understand well the issues with lack of sources, and Bard >>> does sometimes put a suitable source in a footnote, even if it >>> (somewhat disappointingly) just links to wikipedia. There's likely to >>> be a variation soon that does a decent job of providing references, >>> and at that point the role of these tools moves beyond being an >>> amusement to a far more credible research tool. >>> >>> So, these long discussions about impact on open knowledge are quite >>> likely to have to run again in 2024... >>> >>> On Wed, 17 May 2023 at 09:24, Kiril Simeonovski >>> wrote: >>> > >>> > Thank you everyone for your input. >>> > >>> > Your considerations are very similar to mine, and they give a clear >>> direction towards what the guidelines regarding the use of ChatGPT should >>> point to. >>> > >>> > Best regards, >>> > Kiril >>> > >>> > On
[Wikimedia-l] Re: ChatGPT as a reliable source
First: Wikipedia style for dense inline citations is one of the most granular and articulate around, so we're pushing the boundaries in some cases of research norms for clarity in sourcing. That's great; also means sometimes we are considering nuances that may be new. Second: We're approaching a topic close to my heart, which is distinguishing reference-sources from process-sources. Right now we often capture process sources (for an edit) in the edit summary, and this is not visible anywhere on the resulting article. Translations via a translate tool; updates by a script that does a particular class of work (like spelling or grammer checking); applying a detailed diff that was workshopped on some other page. An even better interface might allow for that detail to be visible to readers of the article [w/o traversing the edit history], and linked to the sections/paragraphs/sentences affected. I think any generative tools used to rewrite a section or article, or to produce a sibling version for a different reading-level, or to generate a timeline or other visualization that is then embedded in the article, should all be cited somehow. To Jimbo's point, that doesn't belong in a References section as we currently have them. But I'd like to see us develop a way to capture these process notes in a more legible way, so readers can discover them without browsing the revision history. People using generative tools to draft new material should find reliable sources for every claim in that material, much more densely than you would when summarizing a series of sources yourself. However, as we approach models that can discover sources and check facts, a combination of those with current generative tools could produce things closer to what we'd consider acceptable drafts, and at scale could generate reference works in languages that lack them. I suggest a separate project for those as the best way to explore the implications of being able to do this at scale, and should capture the full model/tuning/prompt details of how each edit was generated. Such an automatically-updated resource would not be a good reliable source, just as we avoid citing any tertiary sources, but could be a research tool for WP editors and modelers alike. SJ On Wed, May 17, 2023 at 9:27 AM Jimmy Wales wrote: > One way I think we can approach this is to think of it as being the latest > in this progression: > > spellchecker -> grammar checker -> text generation support > > We wouldn't have any sort of footnote or indication of any kind that a > spellchecker or grammar checker was > used by an editor, it's just built-in to many writing tools. Similarly, > if writing a short prompt to generate a longer > text is used, then we have no reason to cite that. > > What we do have, though, is a responsibility to check the output. > Spellcheckers can be wrong (suggesting the correct > spelling of the wrong word for example). Grammar checkers can be wrong > (trying to correct the grammar of a direct quote > for example). Generative AI models can be wrong - often simply making > things up out of thin air that sound plausible. > > If someone uses a generative AI to help them write some text, that's not a > big deal. If they upload text without checking > the facts and citing a real source, that's very bad. > > > On 2023-05-17 11:51, The Cunctator wrote: > > Again at no point should even an improved version be considered a source; > at best it would be a research or editing tool. > > On Wed, May 17, 2023, 4:40 AM Lane Chance wrote: > >> Keep in mind how fast these tools change. ChatGPT, Bard and >> competitors understand well the issues with lack of sources, and Bard >> does sometimes put a suitable source in a footnote, even if it >> (somewhat disappointingly) just links to wikipedia. There's likely to >> be a variation soon that does a decent job of providing references, >> and at that point the role of these tools moves beyond being an >> amusement to a far more credible research tool. >> >> So, these long discussions about impact on open knowledge are quite >> likely to have to run again in 2024... >> >> On Wed, 17 May 2023 at 09:24, Kiril Simeonovski >> wrote: >> > >> > Thank you everyone for your input. >> > >> > Your considerations are very similar to mine, and they give a clear >> direction towards what the guidelines regarding the use of ChatGPT should >> point to. >> > >> > Best regards, >> > Kiril >> > >> > On Wed, 17 May 2023 at 10:11, Ilario valdelli >> wrote: >> >> >> >> Define "reliable source". >> >> >> >> A source is reliable if can be consulted by other people than the >> editor >> >> to check the content. >> >> >> >> Is this possible with ChatGPT? No, becaue if you address the same >> >> question to CHatGPT, you will have a different answer. >> >> >> >> In this case how the people verificaying the information can check that >> >> the editor did not invent the result? >> >> >> >> Kind regards >> >> >> >> On 17/05/2023
[Wikimedia-l] Re: ChatGPT as a reliable source
One way I think we can approach this is to think of it as being the latest in this progression: spellchecker -> grammar checker -> text generation support We wouldn't have any sort of footnote or indication of any kind that a spellchecker or grammar checker was used by an editor, it's just built-in to many writing tools. Similarly, if writing a short prompt to generate a longer text is used, then we have no reason to cite that. What we do have, though, is a responsibility to check the output. Spellcheckers can be wrong (suggesting the correct spelling of the wrong word for example). Grammar checkers can be wrong (trying to correct the grammar of a direct quote for example). Generative AI models can be wrong - often simply making things up out of thin air that sound plausible. If someone uses a generative AI to help them write some text, that's not a big deal. If they upload text without checking the facts and citing a real source, that's very bad. On 2023-05-17 11:51, The Cunctator wrote: Again at no point should even an improved version be considered a source; at best it would be a research or editing tool. On Wed, May 17, 2023, 4:40 AM Lane Chance wrote: Keep in mind how fast these tools change. ChatGPT, Bard and competitors understand well the issues with lack of sources, and Bard does sometimes put a suitable source in a footnote, even if it (somewhat disappointingly) just links to wikipedia. There's likely to be a variation soon that does a decent job of providing references, and at that point the role of these tools moves beyond being an amusement to a far more credible research tool. So, these long discussions about impact on open knowledge are quite likely to have to run again in 2024... On Wed, 17 May 2023 at 09:24, Kiril Simeonovski wrote: > > Thank you everyone for your input. > > Your considerations are very similar to mine, and they give a clear direction towards what the guidelines regarding the use of ChatGPT should point to. > > Best regards, > Kiril > > On Wed, 17 May 2023 at 10:11, Ilario valdelli wrote: >> >> Define "reliable source". >> >> A source is reliable if can be consulted by other people than the editor >> to check the content. >> >> Is this possible with ChatGPT? No, becaue if you address the same >> question to CHatGPT, you will have a different answer. >> >> In this case how the people verificaying the information can check that >> the editor did not invent the result? >> >> Kind regards >> >> On 17/05/2023 09:08, Kiril Simeonovski wrote: >> > Dear Wikimedians, >> > >> > Two days ago, a participant in one of our edit-a-thons consulted >> > ChatGPT when writing an article on the Macedonian Wikipedia that did >> > not exist on any other language edition. ChatGPT provided some output, >> > but the problem was how to cite it. >> > >> > The community on the Macedonian Wikipedia has not yet had a discussion >> > on this matter and we do not have any guidelines. So, my main >> > questions are the following: >> > >> > * Can ChatGPT be used as a reliable source and, if yes, how would the >> > citation look like? >> > >> > * Are there any ongoing community discussions on introducing guidelines? >> > >> > My personal opinion is that ChatGPT should be avoided as a reliable >> > source, and only the original source where the algorithm gets the >> > information from should be used. >> > >> > Best regards, >> > Kiril >> > >> > ___ >> > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l >> > Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/WMGIBNPN5JNJGUOCLWFCCPD7EL5YN6KU/ >> > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org >> >> -- >> Ilario Valdelli >> Wikimedia CH >> Verein zur Förderung Freien Wissens >> Association pour l’avancement des connaissances libre >> Associazione per il sostegno alla conoscenza libera >> Switzerland - 8008 Zürich >> Wikipedia: Ilario >> Skype: valdelli >> Tel: +41764821371 >> http://www.wikimedia.ch >> > ___ > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l > Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/4L4K2BUD3YYTAKN6JPHVSSVGOFHW5AKG/ > To unsubscribe send an
[Wikimedia-l] Re: ChatGPT as a reliable source
It's quite interesting how these models ended up being so illiterate and dumb on source reading and interpretation, while so creative and plausible at the same time. I'm sure there's a reason for this, can somebody please point to a link to a place where this is discussed, if you know it? Thanks, Paulo David Gerard escreveu no dia quarta, 17/05/2023 à(s) 13:12: > Note that quite often it just *makes up* a plausible-looking source. > Because AI text generators just make up plausible text, not accurate > text. > > On Wed, 17 May 2023 at 09:40, Lane Chance wrote: > > > > Keep in mind how fast these tools change. ChatGPT, Bard and > > competitors understand well the issues with lack of sources, and Bard > > does sometimes put a suitable source in a footnote, even if it > > (somewhat disappointingly) just links to wikipedia. There's likely to > > be a variation soon that does a decent job of providing references, > > and at that point the role of these tools moves beyond being an > > amusement to a far more credible research tool. > > > > So, these long discussions about impact on open knowledge are quite > > likely to have to run again in 2024... > > > > On Wed, 17 May 2023 at 09:24, Kiril Simeonovski > > wrote: > > > > > > Thank you everyone for your input. > > > > > > Your considerations are very similar to mine, and they give a clear > direction towards what the guidelines regarding the use of ChatGPT should > point to. > > > > > > Best regards, > > > Kiril > > > > > > On Wed, 17 May 2023 at 10:11, Ilario valdelli > wrote: > > >> > > >> Define "reliable source". > > >> > > >> A source is reliable if can be consulted by other people than the > editor > > >> to check the content. > > >> > > >> Is this possible with ChatGPT? No, becaue if you address the same > > >> question to CHatGPT, you will have a different answer. > > >> > > >> In this case how the people verificaying the information can check > that > > >> the editor did not invent the result? > > >> > > >> Kind regards > > >> > > >> On 17/05/2023 09:08, Kiril Simeonovski wrote: > > >> > Dear Wikimedians, > > >> > > > >> > Two days ago, a participant in one of our edit-a-thons consulted > > >> > ChatGPT when writing an article on the Macedonian Wikipedia that did > > >> > not exist on any other language edition. ChatGPT provided some > output, > > >> > but the problem was how to cite it. > > >> > > > >> > The community on the Macedonian Wikipedia has not yet had a > discussion > > >> > on this matter and we do not have any guidelines. So, my main > > >> > questions are the following: > > >> > > > >> > * Can ChatGPT be used as a reliable source and, if yes, how would > the > > >> > citation look like? > > >> > > > >> > * Are there any ongoing community discussions on introducing > guidelines? > > >> > > > >> > My personal opinion is that ChatGPT should be avoided as a reliable > > >> > source, and only the original source where the algorithm gets the > > >> > information from should be used. > > >> > > > >> > Best regards, > > >> > Kiril > > >> > > > >> > ___ > > >> > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, > guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > and https://meta.wikimedia.org/wiki/Wikimedia-l > > >> > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/WMGIBNPN5JNJGUOCLWFCCPD7EL5YN6KU/ > > >> > To unsubscribe send an email to > wikimedia-l-le...@lists.wikimedia.org > > >> > > >> -- > > >> Ilario Valdelli > > >> Wikimedia CH > > >> Verein zur Förderung Freien Wissens > > >> Association pour l’avancement des connaissances libre > > >> Associazione per il sostegno alla conoscenza libera > > >> Switzerland - 8008 Zürich > > >> Wikipedia: Ilario > > >> Skype: valdelli > > >> Tel: +41764821371 > > >> http://www.wikimedia.ch > > >> > > > ___ > > > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, > guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > and https://meta.wikimedia.org/wiki/Wikimedia-l > > > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/4L4K2BUD3YYTAKN6JPHVSSVGOFHW5AKG/ > > > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org > > ___ > > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines > at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/DNOFFTF2DECPFETILCWBOVT5AD63R3UH/ > > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org > ___ > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines > at:
[Wikimedia-l] Re: ChatGPT as a reliable source
As has been pointed out above, we have the hallucination issues, because AIs/LLMs deal in language and how probable a phrase seems to be, rather than in facts. Beyond the hallucination issues, we have the fact that their answers can't be accessed by other editors. Beyond the fact that their answers aren't published sources, even in a scenario where they reliably could present information, they would relay what had been found elsewhere. But is this so different from what we're used to? If I want to use information from a Wikipedia article elsewhere in the encyclopedia, I don't cite said article; I go to the sources. If I can't figure out where the information is coming from, I don't use it. If you can see where the information is coming from (not possible specifically in the normal ChatGPT experience at this time, as this requires that the tool is used to retrieve information from a specific place rather than find probable phrases), read that and cite it, if it's a reliable source. If you can't see where the information is coming from, it can't be used. //Johan Jönsson -- Den ons 17 maj 2023 kl 12:52 skrev The Cunctator : > Again at no point should even an improved version be considered a source; > at best it would be a research or editing tool. > > On Wed, May 17, 2023, 4:40 AM Lane Chance wrote: > >> Keep in mind how fast these tools change. ChatGPT, Bard and >> competitors understand well the issues with lack of sources, and Bard >> does sometimes put a suitable source in a footnote, even if it >> (somewhat disappointingly) just links to wikipedia. There's likely to >> be a variation soon that does a decent job of providing references, >> and at that point the role of these tools moves beyond being an >> amusement to a far more credible research tool. >> >> So, these long discussions about impact on open knowledge are quite >> likely to have to run again in 2024... >> >> On Wed, 17 May 2023 at 09:24, Kiril Simeonovski >> wrote: >> > >> > Thank you everyone for your input. >> > >> > Your considerations are very similar to mine, and they give a clear >> direction towards what the guidelines regarding the use of ChatGPT should >> point to. >> > >> > Best regards, >> > Kiril >> > >> > On Wed, 17 May 2023 at 10:11, Ilario valdelli >> wrote: >> >> >> >> Define "reliable source". >> >> >> >> A source is reliable if can be consulted by other people than the >> editor >> >> to check the content. >> >> >> >> Is this possible with ChatGPT? No, becaue if you address the same >> >> question to CHatGPT, you will have a different answer. >> >> >> >> In this case how the people verificaying the information can check that >> >> the editor did not invent the result? >> >> >> >> Kind regards >> >> >> >> On 17/05/2023 09:08, Kiril Simeonovski wrote: >> >> > Dear Wikimedians, >> >> > >> >> > Two days ago, a participant in one of our edit-a-thons consulted >> >> > ChatGPT when writing an article on the Macedonian Wikipedia that did >> >> > not exist on any other language edition. ChatGPT provided some >> output, >> >> > but the problem was how to cite it. >> >> > >> >> > The community on the Macedonian Wikipedia has not yet had a >> discussion >> >> > on this matter and we do not have any guidelines. So, my main >> >> > questions are the following: >> >> > >> >> > * Can ChatGPT be used as a reliable source and, if yes, how would the >> >> > citation look like? >> >> > >> >> > * Are there any ongoing community discussions on introducing >> guidelines? >> >> > >> >> > My personal opinion is that ChatGPT should be avoided as a reliable >> >> > source, and only the original source where the algorithm gets the >> >> > information from should be used. >> >> > >> >> > Best regards, >> >> > Kiril >> >> > >> >> > ___ >> >> > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, >> guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines >> and https://meta.wikimedia.org/wiki/Wikimedia-l >> >> > Public archives at >> https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/WMGIBNPN5JNJGUOCLWFCCPD7EL5YN6KU/ >> >> > To unsubscribe send an email to >> wikimedia-l-le...@lists.wikimedia.org >> >> >> >> -- >> >> Ilario Valdelli >> >> Wikimedia CH >> >> Verein zur Förderung Freien Wissens >> >> Association pour l’avancement des connaissances libre >> >> Associazione per il sostegno alla conoscenza libera >> >> Switzerland - 8008 Zürich >> >> Wikipedia: Ilario >> >> Skype: valdelli >> >> Tel: +41764821371 >> >> http://www.wikimedia.ch >> >> >> > ___ >> > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, >> guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines >> and https://meta.wikimedia.org/wiki/Wikimedia-l >> > Public archives at >> https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/4L4K2BUD3YYTAKN6JPHVSSVGOFHW5AKG/ >> > To unsubscribe
[Wikimedia-l] Re: ChatGPT as a reliable source
Note that quite often it just *makes up* a plausible-looking source. Because AI text generators just make up plausible text, not accurate text. On Wed, 17 May 2023 at 09:40, Lane Chance wrote: > > Keep in mind how fast these tools change. ChatGPT, Bard and > competitors understand well the issues with lack of sources, and Bard > does sometimes put a suitable source in a footnote, even if it > (somewhat disappointingly) just links to wikipedia. There's likely to > be a variation soon that does a decent job of providing references, > and at that point the role of these tools moves beyond being an > amusement to a far more credible research tool. > > So, these long discussions about impact on open knowledge are quite > likely to have to run again in 2024... > > On Wed, 17 May 2023 at 09:24, Kiril Simeonovski > wrote: > > > > Thank you everyone for your input. > > > > Your considerations are very similar to mine, and they give a clear > > direction towards what the guidelines regarding the use of ChatGPT should > > point to. > > > > Best regards, > > Kiril > > > > On Wed, 17 May 2023 at 10:11, Ilario valdelli wrote: > >> > >> Define "reliable source". > >> > >> A source is reliable if can be consulted by other people than the editor > >> to check the content. > >> > >> Is this possible with ChatGPT? No, becaue if you address the same > >> question to CHatGPT, you will have a different answer. > >> > >> In this case how the people verificaying the information can check that > >> the editor did not invent the result? > >> > >> Kind regards > >> > >> On 17/05/2023 09:08, Kiril Simeonovski wrote: > >> > Dear Wikimedians, > >> > > >> > Two days ago, a participant in one of our edit-a-thons consulted > >> > ChatGPT when writing an article on the Macedonian Wikipedia that did > >> > not exist on any other language edition. ChatGPT provided some output, > >> > but the problem was how to cite it. > >> > > >> > The community on the Macedonian Wikipedia has not yet had a discussion > >> > on this matter and we do not have any guidelines. So, my main > >> > questions are the following: > >> > > >> > * Can ChatGPT be used as a reliable source and, if yes, how would the > >> > citation look like? > >> > > >> > * Are there any ongoing community discussions on introducing guidelines? > >> > > >> > My personal opinion is that ChatGPT should be avoided as a reliable > >> > source, and only the original source where the algorithm gets the > >> > information from should be used. > >> > > >> > Best regards, > >> > Kiril > >> > > >> > ___ > >> > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines > >> > at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > >> > https://meta.wikimedia.org/wiki/Wikimedia-l > >> > Public archives at > >> > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/WMGIBNPN5JNJGUOCLWFCCPD7EL5YN6KU/ > >> > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org > >> > >> -- > >> Ilario Valdelli > >> Wikimedia CH > >> Verein zur Förderung Freien Wissens > >> Association pour l’avancement des connaissances libre > >> Associazione per il sostegno alla conoscenza libera > >> Switzerland - 8008 Zürich > >> Wikipedia: Ilario > >> Skype: valdelli > >> Tel: +41764821371 > >> http://www.wikimedia.ch > >> > > ___ > > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > > https://meta.wikimedia.org/wiki/Wikimedia-l > > Public archives at > > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/4L4K2BUD3YYTAKN6JPHVSSVGOFHW5AKG/ > > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org > ___ > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/DNOFFTF2DECPFETILCWBOVT5AD63R3UH/ > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org ___ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/O75PUVCXLCVELZW7VMQY4KR65M6I5BYK/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org
[Wikimedia-l] Re: ChatGPT as a reliable source
Again at no point should even an improved version be considered a source; at best it would be a research or editing tool. On Wed, May 17, 2023, 4:40 AM Lane Chance wrote: > Keep in mind how fast these tools change. ChatGPT, Bard and > competitors understand well the issues with lack of sources, and Bard > does sometimes put a suitable source in a footnote, even if it > (somewhat disappointingly) just links to wikipedia. There's likely to > be a variation soon that does a decent job of providing references, > and at that point the role of these tools moves beyond being an > amusement to a far more credible research tool. > > So, these long discussions about impact on open knowledge are quite > likely to have to run again in 2024... > > On Wed, 17 May 2023 at 09:24, Kiril Simeonovski > wrote: > > > > Thank you everyone for your input. > > > > Your considerations are very similar to mine, and they give a clear > direction towards what the guidelines regarding the use of ChatGPT should > point to. > > > > Best regards, > > Kiril > > > > On Wed, 17 May 2023 at 10:11, Ilario valdelli > wrote: > >> > >> Define "reliable source". > >> > >> A source is reliable if can be consulted by other people than the editor > >> to check the content. > >> > >> Is this possible with ChatGPT? No, becaue if you address the same > >> question to CHatGPT, you will have a different answer. > >> > >> In this case how the people verificaying the information can check that > >> the editor did not invent the result? > >> > >> Kind regards > >> > >> On 17/05/2023 09:08, Kiril Simeonovski wrote: > >> > Dear Wikimedians, > >> > > >> > Two days ago, a participant in one of our edit-a-thons consulted > >> > ChatGPT when writing an article on the Macedonian Wikipedia that did > >> > not exist on any other language edition. ChatGPT provided some output, > >> > but the problem was how to cite it. > >> > > >> > The community on the Macedonian Wikipedia has not yet had a discussion > >> > on this matter and we do not have any guidelines. So, my main > >> > questions are the following: > >> > > >> > * Can ChatGPT be used as a reliable source and, if yes, how would the > >> > citation look like? > >> > > >> > * Are there any ongoing community discussions on introducing > guidelines? > >> > > >> > My personal opinion is that ChatGPT should be avoided as a reliable > >> > source, and only the original source where the algorithm gets the > >> > information from should be used. > >> > > >> > Best regards, > >> > Kiril > >> > > >> > ___ > >> > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, > guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > and https://meta.wikimedia.org/wiki/Wikimedia-l > >> > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/WMGIBNPN5JNJGUOCLWFCCPD7EL5YN6KU/ > >> > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org > >> > >> -- > >> Ilario Valdelli > >> Wikimedia CH > >> Verein zur Förderung Freien Wissens > >> Association pour l’avancement des connaissances libre > >> Associazione per il sostegno alla conoscenza libera > >> Switzerland - 8008 Zürich > >> Wikipedia: Ilario > >> Skype: valdelli > >> Tel: +41764821371 > >> http://www.wikimedia.ch > >> > > ___ > > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines > at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/4L4K2BUD3YYTAKN6JPHVSSVGOFHW5AKG/ > > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org > ___ > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines > at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/DNOFFTF2DECPFETILCWBOVT5AD63R3UH/ > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org ___ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/LNUPXC7OM56CNFNE3JHHSYR7KTNBCLIM/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org
[Wikimedia-l] Re: ChatGPT as a reliable source
Keep in mind how fast these tools change. ChatGPT, Bard and competitors understand well the issues with lack of sources, and Bard does sometimes put a suitable source in a footnote, even if it (somewhat disappointingly) just links to wikipedia. There's likely to be a variation soon that does a decent job of providing references, and at that point the role of these tools moves beyond being an amusement to a far more credible research tool. So, these long discussions about impact on open knowledge are quite likely to have to run again in 2024... On Wed, 17 May 2023 at 09:24, Kiril Simeonovski wrote: > > Thank you everyone for your input. > > Your considerations are very similar to mine, and they give a clear direction > towards what the guidelines regarding the use of ChatGPT should point to. > > Best regards, > Kiril > > On Wed, 17 May 2023 at 10:11, Ilario valdelli wrote: >> >> Define "reliable source". >> >> A source is reliable if can be consulted by other people than the editor >> to check the content. >> >> Is this possible with ChatGPT? No, becaue if you address the same >> question to CHatGPT, you will have a different answer. >> >> In this case how the people verificaying the information can check that >> the editor did not invent the result? >> >> Kind regards >> >> On 17/05/2023 09:08, Kiril Simeonovski wrote: >> > Dear Wikimedians, >> > >> > Two days ago, a participant in one of our edit-a-thons consulted >> > ChatGPT when writing an article on the Macedonian Wikipedia that did >> > not exist on any other language edition. ChatGPT provided some output, >> > but the problem was how to cite it. >> > >> > The community on the Macedonian Wikipedia has not yet had a discussion >> > on this matter and we do not have any guidelines. So, my main >> > questions are the following: >> > >> > * Can ChatGPT be used as a reliable source and, if yes, how would the >> > citation look like? >> > >> > * Are there any ongoing community discussions on introducing guidelines? >> > >> > My personal opinion is that ChatGPT should be avoided as a reliable >> > source, and only the original source where the algorithm gets the >> > information from should be used. >> > >> > Best regards, >> > Kiril >> > >> > ___ >> > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines >> > at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and >> > https://meta.wikimedia.org/wiki/Wikimedia-l >> > Public archives at >> > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/WMGIBNPN5JNJGUOCLWFCCPD7EL5YN6KU/ >> > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org >> >> -- >> Ilario Valdelli >> Wikimedia CH >> Verein zur Förderung Freien Wissens >> Association pour l’avancement des connaissances libre >> Associazione per il sostegno alla conoscenza libera >> Switzerland - 8008 Zürich >> Wikipedia: Ilario >> Skype: valdelli >> Tel: +41764821371 >> http://www.wikimedia.ch >> > ___ > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/4L4K2BUD3YYTAKN6JPHVSSVGOFHW5AKG/ > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org ___ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/DNOFFTF2DECPFETILCWBOVT5AD63R3UH/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org
[Wikimedia-l] Re: ChatGPT as a reliable source
Thank you everyone for your input. Your considerations are very similar to mine, and they give a clear direction towards what the guidelines regarding the use of ChatGPT should point to. Best regards, Kiril On Wed, 17 May 2023 at 10:11, Ilario valdelli wrote: > Define "reliable source". > > A source is reliable if can be consulted by other people than the editor > to check the content. > > Is this possible with ChatGPT? No, becaue if you address the same > question to CHatGPT, you will have a different answer. > > In this case how the people verificaying the information can check that > the editor did not invent the result? > > Kind regards > > On 17/05/2023 09:08, Kiril Simeonovski wrote: > > Dear Wikimedians, > > > > Two days ago, a participant in one of our edit-a-thons consulted > > ChatGPT when writing an article on the Macedonian Wikipedia that did > > not exist on any other language edition. ChatGPT provided some output, > > but the problem was how to cite it. > > > > The community on the Macedonian Wikipedia has not yet had a discussion > > on this matter and we do not have any guidelines. So, my main > > questions are the following: > > > > * Can ChatGPT be used as a reliable source and, if yes, how would the > > citation look like? > > > > * Are there any ongoing community discussions on introducing guidelines? > > > > My personal opinion is that ChatGPT should be avoided as a reliable > > source, and only the original source where the algorithm gets the > > information from should be used. > > > > Best regards, > > Kiril > > > > ___ > > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines > at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/WMGIBNPN5JNJGUOCLWFCCPD7EL5YN6KU/ > > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org > > -- > Ilario Valdelli > Wikimedia CH > Verein zur Förderung Freien Wissens > Association pour l’avancement des connaissances libre > Associazione per il sostegno alla conoscenza libera > Switzerland - 8008 Zürich > Wikipedia: Ilario > Skype: valdelli > Tel: +41764821371 > http://www.wikimedia.ch > > ___ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/4L4K2BUD3YYTAKN6JPHVSSVGOFHW5AKG/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org
[Wikimedia-l] Re: ChatGPT as a reliable source
Define "reliable source". A source is reliable if can be consulted by other people than the editor to check the content. Is this possible with ChatGPT? No, becaue if you address the same question to CHatGPT, you will have a different answer. In this case how the people verificaying the information can check that the editor did not invent the result? Kind regards On 17/05/2023 09:08, Kiril Simeonovski wrote: Dear Wikimedians, Two days ago, a participant in one of our edit-a-thons consulted ChatGPT when writing an article on the Macedonian Wikipedia that did not exist on any other language edition. ChatGPT provided some output, but the problem was how to cite it. The community on the Macedonian Wikipedia has not yet had a discussion on this matter and we do not have any guidelines. So, my main questions are the following: * Can ChatGPT be used as a reliable source and, if yes, how would the citation look like? * Are there any ongoing community discussions on introducing guidelines? My personal opinion is that ChatGPT should be avoided as a reliable source, and only the original source where the algorithm gets the information from should be used. Best regards, Kiril ___ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/WMGIBNPN5JNJGUOCLWFCCPD7EL5YN6KU/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org -- Ilario Valdelli Wikimedia CH Verein zur Förderung Freien Wissens Association pour l’avancement des connaissances libre Associazione per il sostegno alla conoscenza libera Switzerland - 8008 Zürich Wikipedia: Ilario Skype: valdelli Tel: +41764821371 http://www.wikimedia.ch ___ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/BHEH6R7OPS66VGRA2IIZNRCG3IWPGVN6/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org
[Wikimedia-l] Re: ChatGPT as a reliable source
Hi Kiril, Thanks for raising an interesting topic. On the first question – ChatGPT obviously shouldn't be used as a reliable source; for various reasons, but primarily because it's a text generator that tends to confidently present completely factually incorrect information. Even the notion of "consulting ChatGPT" when writing an article shouldn't be used. (Though I believe that it can be beneficial for supplementary tasks when used with caution, such as helping proofread text & spot spelling mistakes). On the second question – there's a lot of active discussion on this topic on English Wikipedia. I mostly haven't followed it, but can point you to this draft policy (and, of course, its talk page): https://en.wikipedia.org/wiki/Wikipedia:Large_language_models Best Regards Anton Protsiuk On Wed, May 17, 2023 at 10:22 AM David Gerard wrote: > I've mentioned AI text generators on English Wikipedia's Reliable > Sources Noticeboard a couple of times, and the consensus each time has > been that it's obvious that this rubbish absolutely doesn't belong in > en:wp in any manner. The discussions are how to deal with publishers > who indulge in this nonsense. So yes, I would suggest a text generator > could never be used as a source in this manner. The most unreliable of > sources. > > > > - d. > > On Wed, 17 May 2023 at 08:08, Kiril Simeonovski > wrote: > > > > Dear Wikimedians, > > > > Two days ago, a participant in one of our edit-a-thons consulted ChatGPT > when writing an article on the Macedonian Wikipedia that did not exist on > any other language edition. ChatGPT provided some output, but the problem > was how to cite it. > > > > The community on the Macedonian Wikipedia has not yet had a discussion > on this matter and we do not have any guidelines. So, my main questions are > the following: > > > > * Can ChatGPT be used as a reliable source and, if yes, how would the > citation look like? > > > > * Are there any ongoing community discussions on introducing guidelines? > > > > My personal opinion is that ChatGPT should be avoided as a reliable > source, and only the original source where the algorithm gets the > information from should be used. > > > > Best regards, > > Kiril > > ___ > > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines > at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/WMGIBNPN5JNJGUOCLWFCCPD7EL5YN6KU/ > > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org > ___ > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines > at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/ETULXH7VPWVOTOE73RPPAP7MBSTMNJ3I/ > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org > ___ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/PNDMO2QELPSHXAXGKHIWZ7LUNHMVPZAJ/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org
[Wikimedia-l] Re: ChatGPT as a reliable source
I've mentioned AI text generators on English Wikipedia's Reliable Sources Noticeboard a couple of times, and the consensus each time has been that it's obvious that this rubbish absolutely doesn't belong in en:wp in any manner. The discussions are how to deal with publishers who indulge in this nonsense. So yes, I would suggest a text generator could never be used as a source in this manner. The most unreliable of sources. - d. On Wed, 17 May 2023 at 08:08, Kiril Simeonovski wrote: > > Dear Wikimedians, > > Two days ago, a participant in one of our edit-a-thons consulted ChatGPT when > writing an article on the Macedonian Wikipedia that did not exist on any > other language edition. ChatGPT provided some output, but the problem was how > to cite it. > > The community on the Macedonian Wikipedia has not yet had a discussion on > this matter and we do not have any guidelines. So, my main questions are the > following: > > * Can ChatGPT be used as a reliable source and, if yes, how would the > citation look like? > > * Are there any ongoing community discussions on introducing guidelines? > > My personal opinion is that ChatGPT should be avoided as a reliable source, > and only the original source where the algorithm gets the information from > should be used. > > Best regards, > Kiril > ___ > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/WMGIBNPN5JNJGUOCLWFCCPD7EL5YN6KU/ > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org ___ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/ETULXH7VPWVOTOE73RPPAP7MBSTMNJ3I/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org
