Charles has summarized what we learned better than I could, it was an interesting day… The problem is a classic system integration issue where you’ve put a bunch of what you think are working blocks together and the system misbehaves. In this case it was power distribution mostly. We didn’t “fix” it but have a path forward. The biggest problem that we ran into is the effect of capacitor ESR on high frequency switching circuits- this was clear in the main buck converter and for the brushless motor drivers. The vendor (Vishay) for the power supply IC didn’t specify in detail what the output filter cap should be. We found a reference design for an Evaluation Board- that used three 22 uF ceramic caps in parallel - these have an ESR of milliohms up to a megahertz. The parts in the circuit were SMT electrolytics with an ESR of fractions of an ohm (.4) at 100K, the power supply is running at 500Khz. Two of these in parallel with DC-DC sawtooth current waveforms with a amp of current ripple made for a lot of output ripple (+-1 volt on input and output) . We replaced these caps for the buck and improved things somewhat but the problems go a bit deeper and the buck converter is one I’m not familiar with- it’s also voltage mode device which is generally good for transient response (not so important) but tends to be noisy. (current mode control would have been better here). The very large caps to bypass the 4 BLDC motor drivers were also high ESR devices and did a nice job of contaminating the min input rails. If you have no specific direction in a design, you find a compatible part with the right value (1000 uF) not knowing that an equally important spec is ESR in this application. I think we have a path forward- improve power distribution and isolation, bypassing and taking a second look at a few things. The buck regulator that drives the Jetson Nano has some other issues and will be bypassed mostly in the short term. We think this will resolve the majority of problems and may make a demo possible. I’d like to help Charles go through the design in some detail to re-specify a few things that are over or under designed and help with a new layout. We might also do some cost reductions as parts of the circuits are designed somewhat conservatively and conflict with ultimate low cost goals. A lot of fun. A great opportunity to help someone that contributes considerable time to our TriEmbed community. Always onward. Regards, John M. Wettroth E: j...@mindspring.com M: (919) 349-9875 H: (984) 329-5420 From: TriEmbed <triembed-boun...@triembed.org> On Behalf Of Charles West via TriEmbed Sent: Saturday, March 23, 2024 9:36 PM To: Brian <triem...@undecidedgames.net> Cc: triembed@triembed.org Subject: Re: [TriEmbed] Integration trouble with motors + 5 volt buck + 3.3 volt linear Hello, I took up John's offer to help try to figure it out and had a great time working on it with him today. I've put the Kicad files up on Github if anyone else wants to take a look (https://github.com/charlesrwest/GoodBotControlBoard). Initial conclusions are below (John is welcome to double check my notes if he wants): 1. The power line feeding into the linear regulator that feeds the microcontroller on the board is fairly noisy. This noise gets significantly worse when the Nano is drawing larger amounts of current. 2. Adding a large capacitor on the microcontroller side of the linear regulator did not seem to help. It still started acting weird when the nano booted. 3. It appeared that the 5 volt buck regulator in the board was the primary source of the noise. 4. Replacing the large capacitors in the 5 volt regulators seemed to significantly reduce the noise (around half as much?) but didn't solve the primary problem. 5. John discovered that the buck regulator had a number of different modes it can operate in. Some are more likely to produce high frequency noise than others and switching to a different mode might help. However, switching a resistor appears to have somewhat changed the output voltage to 5.5 volts, which makes it unusable for the Jetson nano (but it wasn't working anyway and John came up with a work around, so no biggy). 6. The big capacitors for the motors are probably too high ESR and I should look into replacing them if possible (and doesn't kill price). 7. The linear regulator for the microcontroller should probably be replaced. It is probably near its limits with the current voltage drop and current draw. 8. It would probably be a good idea to isolate the voltage sources from each other using something like passive t filters in the next design. 9. If powering the nano with the buck on the board kills the MCU, maybe don't do that? A temporary solution for the demo may be to reuse an old 5 volt buck module to power the nano and leave the buck on the board idle. 10. The buck on the board might be overpowered for what it is currently being used for and that might be a partial cause for the noise. @Trampas: You are 100% correct and I thank you for the offer to use the O-Scope. It was the MVP working on this today with John. Your solution is also more or less what I'm going to do to see if I can get it demoable. Thank you. @Nick: I think it's more of a power for the boards than control signals issue. Also thank you for the battery offer. However, I think I would have to redesign the chassis to make that work. @brian: 100%. The power is not stable and needs to be figured out. I'm gonna try to patch it to see if I can get it demoable and then do a redesign (John's offered to review) to try to get it stable. I don't think it's the firmware (this time) because the serial communication between the nano and the microcontroller had been disconnected. The only possible way for them to effect each other right now is through the power rails. Thanks, Charlie On Sat, Mar 23, 2024 at 12:46 PM Brian via TriEmbed <triembed@triembed.org <mailto:triembed@triembed.org> > wrote: Hi Charles,
Hearty +1 for everything everyone has said so far. Trampas in particular for suggesting you use a scope on your power rails. Most DMMs do quite a bit of filtering (even if it's in the form of a very slow sample rate) and can't tell you about high-frequency trash on the supply rails (or other lines) that can wreak havoc with sensitive logic systems. If your DMM has a DC+AC mode, try turning that on and seeing what it says is the AC component of the power supply as an intermediate step. Definitely make sure your power is good first; if your power isn't stable, all other bets are off. You want to see a nice flat line with minimal ripple. Make sure you zoom right in on the time axis to see if there's any high-frequency garbage. Also measure as close to the consumers of power as possible (i.e., at IC pins, not at the regulator). Note that DC-DC converters tend to have relatively high amounts of ripple; the 3v3 regulator will filter a lot of that out, but if anything is being fed directly by the 5V supply, you might need more filtering on that rail. Following that, I'd check your ARM firmware for things like polling loops without adequate timeouts when talking to off-chip peripherals. The fact that everything is copacetic until the nano is in the picture makes me wonder if perhaps the ARM core is getting stuck waiting for communication with the nano to finish (assuming they are talking to each other). If you're doing time-critical tasks on the ARM and aren't already, I suggest picking an RTOS (I'm partial to FreeRTOS mainly because of its use on the ESP32 product family; several are available directly in the STM Cube IDE) and learning how to prioritize time-critical tasks so that unpredictable asynchronous communication issues won't get in the way. This is, of course, all very generic advice based on knowing absolutely nothing about your design. :-D Just stuff I've run into on my own travels. Cheers, -Brian On 3/23/24 07:07, Trampas Stern via TriEmbed wrote: > Charles, > > The problem could be several things, the best thing to do is put an > oscilloscope on the power rail and see if that is the problem. I have > an oscilloscope you can borrow if you need one. > > The problem could be many things from noise on ground, to noise on a pin > on the micro, or noise getting to crystal for the micro, again an > oscilloscope will help. However good diagnostic techniques can be > used. For example if you suspect it is the power supply, remove the > 3.3V linear and power the 3.3V from a bench supply and see if the > problem goes away. This does not tell you what is wrong with the 3.3V > rail but lets you isolate the problem for more investigation. > > Trampas > <snippysnip> _______________________________________________ Triangle, NC Embedded Interest Group mailing list To post message: TriEmbed@triembed.org <mailto:TriEmbed@triembed.org> List info: http://mail.triembed.org/mailman/listinfo/triembed_triembed.org TriEmbed web site: https://TriEmbed.org To unsubscribe, click link and send a blank message: mailto:unsubscribe-triem...@bitser.net <mailto:unsubscribe-triem...@bitser.net> ?subject=unsubscribe Searchable email archive available at https://www.mail-archive.com/triembed@triembed.org/
_______________________________________________ Triangle, NC Embedded Interest Group mailing list To post message: TriEmbed@triembed.org List info: http://mail.triembed.org/mailman/listinfo/triembed_triembed.org TriEmbed web site: https://TriEmbed.org To unsubscribe, click link and send a blank message: mailto:unsubscribe-triem...@bitser.net?subject=unsubscribe Searchable email archive available at https://www.mail-archive.com/triembed@triembed.org/
