Hi Ed, Thanks for your suggestions. I tried ftdi_readstream() but it blocks and can't be run simultaneously with ftdi_write_data(). Then I tried your suggestion of putting semaphores around ftdi_write_data() and ftdi_read_data() so they never get called at the same time. It works flawlessly.
The only problem is that my writes are now delayed by the read timeout so I have worse latency. I can live with that in exchange for error-free operation. Kind regards, -- Rick Walker From: WALKER,RICK (Agilent USA) <rick.wal...@agilent.com> Sent: Friday, February 5, 2021 4:26 PM To: Ed Jubenville <ejubenvi...@nsi-mi.com>; libftdi@developer.intra2net.com Cc: WALKER,RICK (Agilent USA) <rick.wal...@agilent.com> Subject: RE: Help debug dropped transfers in FT232H sync fifo application Hi Ed, I originally benchmarked 3 or 4 different architectures for latency and peak throughput before settling on this one. We are several years into the development now before we having complex enough payload to expose the bug. This is for a real-time instrument control application where round trip latency is a key parameter. All the variants of running ftdi_read_data and ftdi_write_data in a loop always increased my latency by at least one USB cycle timeout. I think I will change my ftdi_read_data() thread to using the ftdi_readstream() callback. This at least has the potential of being as fast as the present (almost working) solution. It makes sense now that I think about it. Sometimes an asynchronous write comes through and stomps on a buffer shared with the read thread. It might be worth going through the libftdi code and removing this conflict because it is such a high performance solution. Thanks for catching this problem. I'll report back on my proposed fix later. Kind regards, -- Rick Walker From: Ed Jubenville <ejubenvi...@nsi-mi.com<mailto:ejubenvi...@nsi-mi.com>> Sent: Friday, February 5, 2021 4:13 PM To: libftdi@developer.intra2net.com<mailto:libftdi@developer.intra2net.com> Cc: WALKER,RICK (Agilent USA) <rick.wal...@agilent.com<mailto:rick.wal...@agilent.com>> Subject: Re: Help debug dropped transfers in FT232H sync fifo application At one time, libftdi was not thread-safe. If that is still so, it could mean a lot of things, including indeterminate behaviors when accessed from separate threads. That's a general statement about non-thread-safe libraries. So yes, calling ftdi_read_data from one thread and ftdi_write_data from another thread would not be supported. But that is easily fixed. If it were me, I'd create wrapper functions for ftdi_read_data and ftdi_write_data, and have the wrappers lock a shared semaphore or critical section during their execution. That way, your two threads would block each other from making concurrent calls into the library. It is quick to try, and worth a shot, in my opinion. The intermittent symptoms you've described could easily be attributable to undesirable thread interactions in the library, not by anything in the chip itself. ________________________________ From: rick.wal...@agilent.com<mailto:rick.wal...@agilent.com> <rick.wal...@agilent.com<mailto:rick.wal...@agilent.com>> Sent: Friday, February 5, 2021 1:49 PM To: libftdi@developer.intra2net.com<mailto:libftdi@developer.intra2net.com> <libftdi@developer.intra2net.com<mailto:libftdi@developer.intra2net.com>> Cc: rick.wal...@agilent.com<mailto:rick.wal...@agilent.com> <rick.wal...@agilent.com<mailto:rick.wal...@agilent.com>> Subject: RE: Help debug dropped transfers in FT232H sync fifo application Hi Ed, Are you saying it is not supported to have ftdi_read_data() running at the same time in an independent thread from ftdi_write_data()? After the initial setup, these are the only two functions that are called. -- Rick From: Ed Jubenville <ejubenvi...@nsi-mi.com<mailto:ejubenvi...@nsi-mi.com>> Sent: Friday, February 5, 2021 1:42 PM To: libftdi@developer.intra2net.com<mailto:libftdi@developer.intra2net.com> Subject: Re: Help debug dropped transfers in FT232H sync fifo application External Sender - Use caution opening files, clicking links, or responding to requests. IIRC, libftdi is not designed to be thread-safe. You might want to consider having your multiple threads use a semaphore, critical section, or whatever to provide thread safety outside of the library. [cid:image001.png@01D6FBF4.5BA6D320] ________________________________ From: rick.wal...@agilent.com<mailto:rick.wal...@agilent.com> <rick.wal...@agilent.com<mailto:rick.wal...@agilent.com>> Sent: Friday, February 5, 2021 1:31 PM To: libftdi@developer.intra2net.com<mailto:libftdi@developer.intra2net.com> <libftdi@developer.intra2net.com<mailto:libftdi@developer.intra2net.com>> Cc: tom_kno...@agilent.com<mailto:tom_kno...@agilent.com> <tom_kno...@agilent.com<mailto:tom_kno...@agilent.com>>; rick.wal...@agilent.com<mailto:rick.wal...@agilent.com> <rick.wal...@agilent.com<mailto:rick.wal...@agilent.com>> Subject: Help debug dropped transfers in FT232H sync fifo application Dear libftdi users and developers, I am using an FT232H in synchronous FIFO mode for a scientific instrument control system and am having occasional skipped transfers.Here is a summary of how I use libftdi in hope that you may suggest a way to solve my problem. Here is an abstract of my initialization code: #define CHUNKSIZE 4096 unsigned char buf2ft[CHUNKSIZE]; struct ftdi_context ftdic; assert(ftdi_init(&ftdic) >= 0); retval=ftdi_usb_open_desc(&ftdic, 0x0403, 0x6014, NULL, NULL); assert(ftdi_set_latency_timer(&ftdic, 2) == 0); assert(ftdi_read_data_set_chunksize(&ftdic, CHUNKSIZE) >= 0); assert(ftdi_write_data_set_chunksize(&ftdic, CHUNKSIZE) >= 0); assert(ftdi_set_bitmode(&ftdic, 0xff, BITMODE_SYNCFF) >= 0); assert(ftdi_usb_purge_buffers(&ftdic) == 0); >From this point on I have two threads. The first is the RX thread which sits >in a tight loop calling ftdi_read_data(). I have seen no problems with this >code: void *rxthread(void *ptr) { int ret; int i; unsigned char buffromft[MAXSIZE]; while (!exitRequested) { ret = 0; // check for partial read while (ret == 0) { ret = ftdi_read_data(&ftdic, buffromft, MAXSIZE); } // launch tokens into the processing pipeline for (i = 0; i < ret; i++) { parse(buffromft[i]); } } return ((void *) NULL); } The other thread is the TX thread. My code accumulates user tokens in an array for transmission. When a token is written to the first element of the array, I start a 250us callback signal. If the array is filled, or if the 250us signal fires then I call ftdi_write_data() to transfer the array. This strategy gives me low latency for sporadic transfers while still allowing the system to burst to high data rates when needed. The core portion is the write code: if ((ret = ftdi_write_data(&ftdic, buf2ft, ftindex)) < 0) { fprintf(stderr, "bad write: %d\n", ret); } else if (ret != ftindex) { fprintf(stderr, "partial write: %d < %d\n", ret, ftindex); } // debugging to see what was transferred to libftdi printf("ftdi write: "); for (i = 0; i < ftindex; i++) { printf("%02x ", buf2ft[i]); } printf("\n"); Most of the time, my transfers consist of occasional small blocks of only 11 or 15 bytes. In all cases, I see the ftdi_write_data() accepting the data and returns the proper byte count transferred. To the best of my understanding, the data is properly accepted by the libftdi library for transmission. On our target FPGA, we take the output of the FT232 FIFO Verilog block and echo the data bytes to another FIFO and then to a 480Kb/s serial port for debugging. This gives us a way to see what actually is received by the target FPGA. What we observe is that a few percent of the transfers for the smaller packet sizes (~11 bytes) get lost. I see them go into libftdi, but they don't get delivered to the hardware interface of the FT232H chip. There could be some problem with our Verilog reading the FT232 fifo port, but I can't think of any problems that would result in an entire transfer being deleted. Alternatively, I might not be using the libftdi interface properly or there might be a bug in the library. Thank you if you've read this message so far, and I would be very happy for any suggestions that might help solve this bug. kind regards, -- Rick Walker ________________________________ libftdi - see http://www.intra2net.com/en/developer/libftdi<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.intra2net.com%2Fen%2Fdeveloper%2Flibftdi&data=04%7C01%7Crick.walker%40agilent.com%7C899ad653e4cd469d0ed008d8ca35bbf4%7Ca9c0bc098b46420693512ba12fb4a5c0%7C0%7C0%7C637481679430608884%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=VO6MxAYf6nPEDdxkwdfxbdzD0jDJX3sZfc2Om3EB3S0%3D&reserved=0> for details. To unsubscribe send a mail to libftdi+unsubscr...@developer.intra2net.com<mailto:libftdi+unsubscr...@developer.intra2net.com> ________________________________ libftdi - see http://www.intra2net.com/en/developer/libftdi<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.intra2net.com%2Fen%2Fdeveloper%2Flibftdi&data=04%7C01%7Crick.walker%40agilent.com%7C899ad653e4cd469d0ed008d8ca35bbf4%7Ca9c0bc098b46420693512ba12fb4a5c0%7C0%7C0%7C637481679430618841%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=I5EtbpIUdGX%2FfCxbCmUy9WcbfcZMg3GavqvAsCR%2Bg5k%3D&reserved=0> for details. To unsubscribe send a mail to libftdi+unsubscr...@developer.intra2net.com<mailto:libftdi+unsubscr...@developer.intra2net.com> ________________________________ libftdi - see http://www.intra2net.com/en/developer/libftdi<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.intra2net.com%2Fen%2Fdeveloper%2Flibftdi&data=04%7C01%7Crick.walker%40agilent.com%7C899ad653e4cd469d0ed008d8ca35bbf4%7Ca9c0bc098b46420693512ba12fb4a5c0%7C0%7C0%7C637481679430628795%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=TkOB51VkY0BcNPKvWUzBRv7DKfHbTAFGB0sMJZizBho%3D&reserved=0> for details. To unsubscribe send a mail to libftdi+unsubscr...@developer.intra2net.com<mailto:libftdi+unsubscr...@developer.intra2net.com> -- libftdi - see http://www.intra2net.com/en/developer/libftdi for details. To unsubscribe send a mail to libftdi+unsubscr...@developer.intra2net.com
