Hey Guys, it's been awhile since I've asked the question about the usefulness of AI in our development world - for a seasoned senior developer for a complex digital transformation project. Since then I've met the microsoft board member on OpenAI and have completed an AI course at Stanford and seen the latest and greatest at the most advanced world leading AI lab. To be honest I think this AI thing is going to be 10 years away to be truly useful as it is hyped to be. But this is just a personal opinion.
I wanted to get some feedback from you guys last time michael said maybe about 1-2% in improvement in usefulness to a seasoned dev. I've had ongoing robust conversations. Because I feel the benefits are completely overstated in general (open to being corrected). The most senior tech lead in my team believe it's a negative benefit. As in it can't understand complex business analyst requirements, nor debug codes, nor find / correct defects. The effort to prompt engineer and also to correct what's generated is also understated. Plenty of tools already do what chatgpt/co-pilot can do. In this case we've experimented with github copilot which claims to provide 55% efficiency in coding. What's everyone's thoughts since Feb this year? - how useful is it in real world dev tasks, building complex transformation projects from BA requirements? Thanks in advance, Tom On Sat, Feb 24, 2024 at 9:21 PM Michael Ridland via ozdotnet < ozdotnet@ozdotnet.com> wrote: > > If you’re a seasoned practitioner in the language it’s probably sitting > around 1-2%, because it can help with some code conversions. Copilot code > completions are amateur. For .net resharper/rider etc is actually a much > better option than copilot, practical use of that is probably around 15% > productivity increase. > > Copilot does help if you don’t know the language or technology, which > maybe commbank staff are not skilled up enough? ;) > > > > On Sat, 24 Feb 2024 at 6:27 pm, DotNet Dude via ozdotnet < > ozdotnet@ozdotnet.com> wrote: > >> I’m not a subscriber to read the entire article but 30% is a load of BS >> at this stage >> >> On Sat, 24 Feb 2024 at 15:18, Tom Gao via ozdotnet <ozdotnet@ozdotnet.com> >> wrote: >> >>> Here's the article. >>> >>> >>> https://www.afr.com/technology/commbank-claims-ai-is-already-making-it-work-30pc-better-20240206-p5f2uu >>> >>> So what does everyone think about the claim that 30% efficiency? ie to >>> me as head of IT instead of hiring 10 developers to build applications now >>> only 7 is required. To me that just doesn't seem realistic. >>> >>> >>> >>> On Fri, Feb 23, 2024 at 3:04 PM Tom Rutter via ozdotnet < >>> ozdotnet@ozdotnet.com> wrote: >>> >>>> >>>> On Fri, 23 Feb 2024 at 12:32, Dr Greg Low via ozdotnet < >>>> ozdotnet@ozdotnet.com> wrote: >>>> >>>>> Worse, in the baseball umpiring exam I mentioned, someone said to me >>>>> “isn’t that what you’d get if you just asked a whole lot of fans about the >>>>> rules rather than asking umpires?” >>>>> >>>>> >>>>> >>>>> There’s probably something important about that. How does it know >>>>> which of the material it was trained on is valid? >>>>> >>>> >>>> Yep that’s the thing, it doesn’t actually know what is “valid”. From >>>> what I understand it just gets “pushed” in certain directions during >>>> training and not in all cases obviously. >>>> >>>> My experience with things like ChatGPT so far basically brings it down >>>> to a better Google really. Early days >>>> >>>> >>>>> >>>>> Regards, >>>>> >>>>> >>>>> >>>>> Greg >>>>> >>>>> >>>>> >>>>> Dr Greg Low >>>>> >>>>> >>>>> >>>>> 1300SQLSQL (1300 775 775) office | +61 419201410 mobile >>>>> >>>>> SQL Down Under | Web: https://sqldownunder.com >>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=o3oFliHztOF8D9Nbqaa7KQdqC-zkQNXWl4IqnEG58Wc&e=> >>>>> | >>>>> About Greg: https://about.me/greg.low >>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=NsAibgiqfCxsyc8m2DBKogKQcs3OqE3mkyCjmpoYxTk&e=> >>>>> >>>>> >>>>> >>>>> *From:* mike smith via ozdotnet <ozdotnet@ozdotnet.com> >>>>> >>>>> *Sent:* Friday, February 23, 2024 12:19 PM >>>>> *To:* ozDotNet <ozdotnet@ozdotnet.com> >>>>> *Cc:* mike smith <meski...@gmail.com> >>>>> *Subject:* Re: AI >>>>> >>>>> >>>>> >>>>> "old system views" >>>>> >>>>> >>>>> >>>>> That makes me wonder if it has any way of differentiating between >>>>> something it found from a decade ago to more recent data. >>>>> >>>>> >>>>> >>>>> Mike >>>>> >>>>> On Fri, 23 Feb 2024, 11:43 Dr Greg Low via ozdotnet, < >>>>> ozdotnet@ozdotnet.com> wrote: >>>>> >>>>> Hi Tom, >>>>> >>>>> >>>>> >>>>> For me, it depends what you want it to do. It certainly can appear to >>>>> help someone who’s new to an area. >>>>> >>>>> >>>>> >>>>> For most code writing, I’ve been pretty underwhelmed. As an example, >>>>> if I ask it to write SQL, I get a very poor outcome. It will use old >>>>> deprecated views instead of the current system views (that have been >>>>> around >>>>> for a decade), and often does things in a convoluted way. >>>>> >>>>> >>>>> >>>>> What I have been impressed with, is how it can help you understand >>>>> acronyms, etc. Quite amazing. I’ve also been pretty impressed with using >>>>> it >>>>> go generate some test data, including in multiple languages. And the test >>>>> data is fairly believable. If I ask it for family names, and I also ask >>>>> for >>>>> Chinese, it does pick common Chinese family names in the test output. >>>>> That’s pretty impressive. >>>>> >>>>> >>>>> >>>>> It can do a reasonable job of things like “here’s some DAX code, can >>>>> you simplify it?” It often can. Or “here’s a regular expression, can you >>>>> explain what it does?” and it does that just fine. I’ve seen people >>>>> happily >>>>> using it to explain code that they don’t understand, or to (sort of) >>>>> document some code. >>>>> >>>>> >>>>> >>>>> But it also is so confident on things, yet so wrong. I gave it a 25 >>>>> question baseball umpire test the other day. It was 100% confident >>>>> sounding, but 40% correct. The weird thing is that some of the questions >>>>> that it got right, are things that new human umpires often get wrong. Yet >>>>> for simpler questions, it would say that something legal is illegal. >>>>> >>>>> >>>>> >>>>> It’s certainly interesting, but it’s very much a work in progress. It >>>>> will be part of our futures. >>>>> >>>>> >>>>> >>>>> Regards, >>>>> >>>>> >>>>> >>>>> Greg >>>>> >>>>> >>>>> >>>>> Dr Greg Low >>>>> >>>>> >>>>> >>>>> 1300SQLSQL (1300 775 775) office | +61 419201410 mobile >>>>> >>>>> SQL Down Under | Web: https://sqldownunder.com >>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=o3oFliHztOF8D9Nbqaa7KQdqC-zkQNXWl4IqnEG58Wc&e=> >>>>> | >>>>> About Greg: https://about.me/greg.low >>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=NsAibgiqfCxsyc8m2DBKogKQcs3OqE3mkyCjmpoYxTk&e=> >>>>> >>>>> >>>>> >>>>> *From:* Tom Gao via ozdotnet <ozdotnet@ozdotnet.com> >>>>> *Sent:* Friday, February 23, 2024 11:58 AM >>>>> *To:* ozDotNet <ozdotnet@ozdotnet.com> >>>>> *Cc:* Tom Gao <t...@tomgao.com> >>>>> *Subject:* AI >>>>> >>>>> >>>>> >>>>> Hi guys, I haven't posted in a few years and haven't been on the tools >>>>> for a long time now as well. I'm on a panel on a digital conference coming >>>>> up in march. We had a pre meeting today and the topic of AI came up. Two >>>>> of >>>>> the panelist said cited CBA and Westpac using AI and were able to save 30% >>>>> on development effort. >>>>> >>>>> >>>>> >>>>> Personally I just finished an AI course my view is quite the opposite. >>>>> My personal opinion of the generative AI space and AI in general having >>>>> spent time with the academics is that the benefits are significantly over >>>>> inflated. >>>>> >>>>> >>>>> >>>>> I want to get some other opinions if you are seeing any significant >>>>> benefit and that I may be just out of touch or not aware. >>>>> >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Tom >>>>> >>>>> -- >>>>> ozdotnet mailing list >>>>> To manage your subscription, access archives: >>>>> https://codify.mailman3.com/ >>>>> >>>>> -- >>>>> ozdotnet mailing list >>>>> To manage your subscription, access archives: >>>>> https://codify.mailman3.com/ >>>> >>>> -- >>>> ozdotnet mailing list >>>> To manage your subscription, access archives: >>>> https://codify.mailman3.com/ >>> >>> -- >>> ozdotnet mailing list >>> To manage your subscription, access archives: >>> https://codify.mailman3.com/ >> >> -- >> ozdotnet mailing list >> To manage your subscription, access archives: >> https://codify.mailman3.com/ > > -- > ozdotnet mailing list > To manage your subscription, access archives: https://codify.mailman3.com/
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
