One thing that I did miss to mention, was that when I first joined Microsoft, I found out that Connect (or Ladybug at the time) wasn't designed to be two way conversation. It was designed to be an easy (which is debatable) way for customers to file bugs and then have them enter our (yes complex) system. There are approximately 30 odd product unit (with up to 3, 4 or 5 feature teams in each) contributing to VS/.NET - some teams handled the conversation part better than others, others did a terrible job.
While they have started put things in place to make sure the customer is aware of what's going on (I think it will no longer allows you to resolve a bug without adding a customer comment) - there still kinks that need to be ironed out. Just more on the triage part - each feature gets to triage bugs how they like, this means that to the customer what appears to be one contiguous system is actually one abstraction over a 100 different little systems. I'm not trying to defend this process at all - I think it used to suck really bad, a bunch of stuff fell though the cracks (hell I still have 9 bugs/suggestions still open[1] from before I joined Microsoft). However, it is getting better - it just takes a while to get 1500 people walking in the same direction... [1] https://connect.microsoft.com/VisualStudio/SearchResults.aspx?KeywordSearchIn=2&SearchQuery=%26quot%3bDavid+M.+Kean%26quot%3b&FeedbackType=0&Status=1&Scope=0&SortOrder=5&TabView=0 ________________________________ From: [email protected] [[email protected]] on behalf of Joseph Cooney [[email protected]] Sent: Wednesday, May 12, 2010 4:54 AM To: ozDotNet Subject: Re: benefits of using vs 2010 Maybe I peppered that last comment with a little bit too much "asshole" - now for a response that has been marinating in the milk of human kindness. After reading David's comments I get it - a big organization, a complex process with maybe everyone working in that process doing the best they can. But it seems in some cases (perhaps many cases) collectively the process is failing Microsoft, and failing the people who submit feedback, and thus I'm calling it broken. It's failing MS because they're disenfranchising people who could be (and in many cases ARE) some of their strongest supporters. Globally MS employ many thousands of evangelists to help people understand and use their products, and spend truck-loads of money on conferences, competitions, advertising and blogging with the end goal of building human connections with their dev customers around the world. If the process for submitting a bug leaves you feeling like you've been told to "go write a map reduce function in erlang"<http://browsertoolkit.com/fault-tolerance.png> then all that evangelism is wasted. It's also failing MS if they miss legitimate opportunities to fix bugs in their product. It's failing MS customers reporting bugs because they feel marginalized and ignored. If this happens often enough developers will start thinking "I can submit a bug report on an ASP.NET<http://ASP.NET> problem I'm having, wait 6 months only to be told it's by design/won't fix/go buy some premier support, or I can download django or RoR and fix my own problems". It sounds like from what others have said and what David is describing the process could be improved in a number of ways * Goal entry-level PSS people that triage bugs so they try harder to get a repro. If not having a repro greatly diminishes your chances of taking a bug seriously then there are going to be lots of legitimate bugs that slip through the cracks. * Open, honest communication - if you can't fix it for the current release explain why. Explain what the time-frame for fixing it (or considering it if it is a feature suggestion) might be. There's a real catch-22 with Microsoft dev product cycles - external developers don't want to build serious stuff on top of beta1 code, but by the time beta2 rolls around it's too late to change things for that release, and it may be impossible to change it EVER (given the need for backwards computability). * Accountability - if you're going to be asinine and close a bug that a 3rd party expert has reported to you (I'm thinking about Greg's story from earlier on in the thread here) and which multiple other people have commented on as being serious at least put your name beside it, and have some kind of channel where dialog about why the bug can occur. I'm calling the process as broken - you may disagree with me, but from the anecdotes above, and from many others I've read around the interwebs this seems like a black eye in MS's engagement with developers. Joseph On Wed, May 12, 2010 at 9:08 PM, Arjang Assadi <[email protected]<mailto:[email protected]>> wrote: Not sure if the process is broken, it is a complex product and a complex process. No matter how they turn and change the process, they will not be able to handle everything, That is not an excuse but just an intrinsic property of complex systems. Kind Regards Arjang Assadi On 12 May 2010 21:02, Joseph Cooney <[email protected]<mailto:[email protected]>> wrote: > With all due respect, explaining a broken process doesn't make it any less > broken. > > On Wed, May 12, 2010 at 6:44 PM, David Kean > <[email protected]<mailto:[email protected]>> > wrote: >> >> Alright, given that I've actually been on both sides of the fences on this >> one, let me try and explain what happens on the other side: >> >> 1) Customer files a bug on Connect, it appears in our TFS bug database >> internally. >> 2) First CSS/PSS or whatever they are called these days have a look at >> it to try and reproduce the problem. This is to reduce the amount of product >> support that the feature team themselves have to do (which would take us >> away from feature work). >> 3) As these are support personnal and did not actually work on the >> product, they usually don't have as much context as the feature team >> themselves - hence why some bugs that seem obvious to someone who's used the >> technology extensively might be resolved as not-reproducible. The best way >> to get through CSS is to produce a simple repro project that clearly shows >> the bug. >> 4) After CSS have reproduced the problem (or if it is a suggestion), they >> assign the bug to the feature triage team (usually a senior PM, dev and >> test) to look at it. CSS also will add a comment to the customer to tell >> them that they have done that. >> 5) We look at the bug to determine a couple of things: >> >> i) Is it by-design? The bug may be exhibiting behavior called out in the >> documentation, or called out in the spec (internal document describing the >> design of the feature). Or it may be relying on behavior that we don't >> guarantee (such as relying on the result of String.GetHashCode). >> ii) Is it really a suggestion? Is the request a new feature, API or >> behavior that we previously didn't have? These usually fall in priority >> against normal bugs - and we usually consider these in the next planning >> milestones (which in the 6 months to a year of the product cycle means next >> version). >> iii) Is it a bug that we would fix? A variety of factors come into play >> when we decide whether we should fix the bug; How risky is it? Would it >> break compatibility? How many customers would benefit from it? It is a >> corner case? Does it have a reasonable workaround? Is it in an area that >> we're no longer investing in? >> >> The ultimate resolution of a bug can take many months depending on the >> which part of the product cycle we are in, bouncing among various members on >> the team to gather information, and then usually sent back to the triage >> team to decide above. The triage team will then usually add a comment to the >> customer. >> >> Now the tricky part of above, is that you guys don't see the whole story >> behind the bug. While a bug may have one or two public comments from >> Microsoft, internally there are usually a whole bunch of comments from devs, >> PMs and QA explaining the underlying details, calling out the spec or doc >> that describes the behavior, or explains how it would break compatibility. >> Unfortunately, due to the design of the system - it's very easy to forget to >> add comment to the customer to explain what's occurring underneath or to >> even know that the bug itself is a customer filed bug. This is why sometimes >> bugs are closed without comments. >> >> On top of this, we actually change backend databases every product >> version, and when this happens sometimes the link between the external site >> and the internal database breaks, meaning that any updates to the bug are >> not shown externally. If you've got any bugs from 2005 that are still >> active, then chances are this is one of these cases. I've re-raised this >> issue a couple of weeks ago, so hopefully we can a resolution soon for >> these. >> >> Another thing that I should add about the 'Won't Fix' tag: This can >> actually have two meanings depending on the team: >> >> 1) It really means Won't Fix - something that the team won't look at >> fixing unless a lot of people hit the same bug or provide feedback. >> 2) It means Won't Fix for this release, but we'll add a special tag to it >> that means 'consider it in the planning for vNext' - this is similar, but >> confusingly not the same, to the Postponed resolution. >> >> Unfortunately, external customers can't see which one this is. You need to >> gleam this from the comments of the product team. >> >> Now this turned out to be longer than I'd planned, its late over here >> (1:41am) and I really should go to bed. >> >> ________________________________ >> From: [email protected]<mailto:[email protected]> >> [[email protected]<mailto:[email protected]>] on >> behalf of Eddie de Bear >> [[email protected]<mailto:[email protected]>] >> Sent: Tuesday, May 11, 2010 10:05 PM >> To: [email protected]<mailto:[email protected]>; ozDotNet >> Subject: Re: benefits of using vs 2010 >> >> I agree with Greg on this one. I've submitted bugs and enhancements which >> received positive responses (from Microsoft) only to be closed "Won't Fix" >> at the last minute. Even if they were migrated to VS-Next would have been a >> better option, but to have them closed with no explanation just discourages >> people from submitting anything. >> >> Ed. >> >> On Wed, May 12, 2010 at 2:04 PM, Greg Low (greglow.com<http://greglow.com>) >> <[email protected]<mailto:[email protected]>> >> wrote: >>> >>> Too true Mitch. >>> >>> >>> >>> Unfortunately, most of the folk I know that used to submit a lot of bugs >>> and suggestions have stopped doing so. There are way too many “by design” >>> responses. And most suggestions (rather than bugs) have no response until >>> the product is about to ship, then they come back with “closed won’t fix”, >>> without comment or even a name of who to talk to. >>> >>> >>> >>> It just isn’t a good feedback mechanism at present. >>> >>> >>> >>> I’ve even had entries submitted in detail, that a bunch of people have >>> voted for, many have commented that it’s important, and it’s been closed as >>> “closed not reproducible”. Again, with the decision attributed to >>> “Microsoft” and no other name present. You’d think if a number of people >>> think it’s important and you can’t reproduce it, you’d reach out to the >>> person posting it at the very least. >>> >>> >>> >>> I can’t make sense of many of the statuses either. I’ve had another one >>> that said “can’t reproduce” but also then said “fixed in SP1”. >>> >>> >>> >>> Regards, >>> >>> >>> >>> Greg >>> >>> >>> >>> From: [email protected]<mailto:[email protected]> >>> [mailto:[email protected]<mailto:[email protected]>] >>> On Behalf Of Mitch Wheat >>> Sent: Tuesday, 11 May 2010 10:22 AM >>> >>> To: 'ozDotNet' >>> Subject: RE: benefits of using vs 2010 >>> >>> >>> >>> While I'm sure the folks at Microsoft do their utmost to fix bugs, it >>> doesn't take long to 'burn' bug submitters with "This is by design" >>> responses >>> >>> >>> >>> Just my 2 cents. >>> >>> >>> >>> Mitch Wheat >>> >>> >>> >>> From: [email protected]<mailto:[email protected]> >>> [mailto:[email protected]<mailto:[email protected]>] >>> On Behalf Of David Kean >>> Sent: Tuesday, 11 May 2010 8:19 AM >>> To: ozDotNet >>> Subject: RE: benefits of using vs 2010 >>> >>> >>> >>> > Sadly most of the worst bugs from VS 2008 are still there. >>> >>> >>> >>> Can you tell the ones that you keep running into? Or can you head over to >>> Microsoft Connect and file these? Customer feedback is a huge factor in what >>> bugs in fix – if we find the bugs internally but no customer has reported >>> them, these fall in priority against other bugs that customers have filed. >>> >>> >>> >>> From: [email protected]<mailto:[email protected]> >>> [mailto:[email protected]<mailto:[email protected]>] >>> On Behalf Of Mark Jarzebowski >>> Sent: Monday, May 10, 2010 5:09 PM >>> To: ozDotNet >>> Subject: Re: benefits of using vs 2010 >>> >>> >>> >>> I've switched most of my current apps to VS2010. >>> >>> >>> >>> It's looks nicer and is more pleasant to work with. >>> >>> >>> >>> Sadly most of the worst bugs from VS 2008 are still there. >>> >>> >>> >>> Also not much there to improve productivity for coal face developers. >>> >>> Regards ..... Mark Jarzebowski >>> Director Software Engineering >>> Business Model Systems >>> Kew Victoria >>> www.bms.com.au<http://www.bms.com.au> >>> >>> On Mon, May 10, 2010 at 1:22 PM, Anthony >>> <[email protected]<mailto:[email protected]>> wrote: >>> >>> >>> >>> Anyone using vs2010? Is it worth upgrading some projects? >>> >>> >>> >>> >>> >>> regards >>> >>> Anthony (*12QWERNB*) >>> >>> >>> >>> >>> >>> >>> >>> >> >> >> -- >> Eddie de Bear >> Mob: 0417066315 >> Messenger: [email protected]<mailto:[email protected]> >> Skype: eddiedebear > > > > -- > Joseph Cooney > > http://jcooney.net > -- Joseph Cooney http://jcooney.net
