Kathleen, Ryan, Clint and the rest of the community: Again, we appreciate the opportunity to respond to these concerns.
Readers should take care to read our previous response (first) and all attachments in order to understand the full context of all subsequent replies and our subsequent responses. As before, given the publicity of this forum, I will do my best to respect individuals by not constantly using their names or calling out other root program member organizations as examples, even when it would be helpful to do so. Instead I will ask you to consider that aspect in my response. Also, I will use "our company" when speaking of TrustCor (the CA operator) and MsgSafe (the email service). I will use "the researchers" when speaking about Serge Edelman, Joel Reardon, their commercial enterprise AppCensus, and the universities for which they work (University of California, Berkeley and the University of Calgary, respectively). Again as a blanket statement, it is important readers understand we have never been accused of, and there is no evidence to suggest that TrustCor violated conduct, policy, or procedure, or wrongfully issued trusted certificates, or worked with others to do so. We have not done any of those things. It’s also important to understand TrustCor operates a certificate authority (TrustCor CA) which provides CA services protected and insulated by an exclusion agreement, and TrustCor operates a privacy-enhancing communications service (MsgSafe.io) as two discrete business units. In reading related reporting and blogging off-list, I need to address an elephant in the room. Apparently it may also come as a surprise to some readers that other root program members are in fact international governments, and some are also defense companies, or companies who are wholly-owned by defense companies and/or state-owned enterprises, meaning "businesses" that are completely owned or controlled by governments. Further, some of those governments are not free/democratic and in fact some have histories of tragic human rights violations. We are none of those things and our company does not identify with those values. I will respond below to the additional questions raised by the researchers. ————————————— In Response to Joel’s (University of Calgary/AppCensus) supplemental concerns, In Response to "Please correct me if I am wrong, but from my understanding there appears to be nothing in what I presented to this forum that is being disputed as incorrect or imagined." and "… generally agrees with me that concerns that I raised were indeed legitimate and worth the attention and scrutiny of this community.": Professionally speaking we are obliged to respond that there are aspects of what Joel has stated both in his report and in his supplemental statements with which we disagree, would dispute or are factually inaccurate. We cannot ignore a blanket statement designed to confirm our assent, of which there were several of these. However, now that I’ve made this point in response I won’t expound unnecessarily in this forum in favour of responding constructively to Joel’s supplemental points assuming they are of interest to the greater community. In Response to "I have listed a few of the public audits I have found here [1], and Mozilla also has them listed here [2]. What I've found is that in the standard and BR audits for 2018, 2019, 2020, and 2021, as well as the code signing audits for 2020 and 2021, their auditor consistently describe the CA's "Certification Authority (CA) operations at Toronto, Ontario, Canada". According to what I’ve learned from this thread (please correct me if I am wrong) TrustCor was not a Canadian company during this time and did not have an office in Canada. This is ten different audits over four years.": Prior context: TrustCor’s Ontario Headquarters address was previously located at 7270 Woodbine Avenue, Suite 308 Markham ON L3R 4B9 Canada. When Ian’s health began to decline, and given that Ian, Ryan and I were the only Canadian employees, and our technical staff were centrally located to our data centre locations, we decided on a remote-work structure in Canada. While the Canadian filings ended in 2016, we still maintain personnel, a secure storage facility, and we have always maintained a consistent mailing address in Ontario where the public can contact the TrustCor Policy Authority in writing for policy related enquiries and this is the same address printed on TrustCor’s letterhead and within our CPS documents. Supplemental response: Quoted from the WebTrust Illustrative Examination Reports Under SSAE 18 and SSAE 21, Version 2.0 Published 1 February 2022. "All reports issued should list the state/province, and country of all physical locations of CA facilities that were included in the scope of the engagement. CA facilities may include data centre locations (primary and alternate sites), registration authority locations (for registration authority operations performed by the CA), and all other locations where general IT and business process controls that are relevant to CA operations in scope (including cloud and remote locations)." Based on the requirements coming directly from WebTrust, we believe we are reporting correctly inside the managements assertions. It still remains true that a good portion of our business process controls, relevant to the CA operations, are conducted by key personnel from Canada. As previously stated, we have always disclosed both our Canadian and United States locations in our CPS document and management assertions. In Response to "Regarding the SDK, I agree that speculating why it was included is neither useful nor helpful. It may still be possible to get a better understanding of how it was included. For example, source code version control history and commit messages may give some context. According to reporting by the Wall Street Journal who interviewed app makers who included this SDK that "Several developers said Measurement Systems required them to sign nondisclosure agreements." [3] As well, the code is not available for download but was send to developers who agreed to include it. This was indeed some years ago, and I agree that the means of delivery, etc., may have changed. Nevertheless if such emails or other communications are available to you it may help elucidate the context around how this SDK was obtained.": Prior context: Our company never published a production or supported version of the MsgSafe mobile app containing the Measurement Systems SDK. Relative to the tiny population of Beta product-testers (which were largely our own employees) who chose to test a Beta version of the app containing that SDK, I will add that during the development stages of MsgSafe’s BETA mobile app, our developers sought out the help from 3rd party software services to obtain better app analytics. We are aware that they evaluated other SDKs and tools like Firebase, Bugsnag, etc. but they claimed to not help much in troubleshooting and improving the app functionality across all device manufacturers and OS versions. There was a time during this process where they incorporated additional software developers to help with the issues we were facing. Whether or not the SDK was added for a developer’s own financial gain or otherwise is beyond us and we don’t care to speculate. Again, the MsgSafe BETA mobile app was never released in a production supported version and has been abandoned for years, and we can confirm the mobile-first web UI, which is the only supported mobile interface in-use today and for the last few years, which does not contain any SDK from anyone. As far as how the MsgSafe mobile app obtained an “unobfuscated version” of the SDK? It is not our place to speculate, but it only makes sense that any company would provide updates to their software over time. The third-party app archive website containing MsgSafe’s APK, as referenced in the researcher’s post, is over 3 years old. It should come as no surprise that the software found there doesn’t match up exactly to the software found in apps they reported about in April 2022. Our developers probably didn’t even notice subtle changes like this because it’s not our practice to reverse engineer other company’s software and violate license agreements. Supplemental response: Prior to my original reply, we had already completed an investigation related to this activity. Our software revision control system revealed immediately when the software was introduced and which developer introduced it. As I had previously stated, "… they incorporated additional software developers to help with the issues we were facing." The additional developers were contract developers. Also as I previously stated, "Whether or not the SDK was added for a developer’s own financial gain or otherwise is beyond us and we don’t care to speculate." Our investigation found the developer in question properly signed our standard "Confidentiality Obligation and Invention Agreement” that requires any developer to obtain a corporate license to any 3rd party software or intellectual property the developer chooses to include. We confirmed through corporate records and email searches that no such agreement was ever obtained by the company or company counsel. Also, none was included inside the software/check-in to revision control. We also confirmed no approval for including this third-party software was ever obtained from Wylie (technically the manager of the developers at that time). Technically that individual developer violated our Confidentiality Obligation and Invention Agreement. When we discussed this with legal counsel, their opinion that a labor dispute over an agreement violation that took place over 3 years ago would be limited/difficult to pursue, especially since the developer has not worked for the company in 3 years since native app development was abandoned in favor of our decision to focus on our mobile-first web application, which involves completely-different skillsets / personnel. Further, their opinion was that damages might not be readily provable because (as previously stated) the mobile app was never "released" or rather it was only provided for testing in a BETA form (that specifically admonished it was a beta and not supported) and used primarily by our own employees testing it. I realize the existence or proposition of this basically-unused beta software may have been a boon to AppCensus reporting and is of intellectual interest to the researchers, but our company sees no benefit to any legal or other pursuit with regard to benefitting our customers, or any relying parties. (more to come on this topic below in response to Serge) ————————————— In Response to Serge’s (Berkely/AppCensus) supplemental concerns, In Response to "… I decided to spend a little bit more time exploring it … [many lines of technical details] … This certificate was issued by/to MsgSafe (it appears to be self-signed)": Prior context: Our company never published a production or supported version of the MsgSafe mobile app containing the Measurement Systems SDK. Relative to the tiny population of Beta product-testers (which were largely our own employees) who chose to test a Beta version of the app containing that SDK, I will add that during the development stages of MsgSafe’s BETA mobile app, our developers sought out the help from 3rd party software services to obtain better app analytics. We are aware that they evaluated other SDKs and tools like Firebase, Bugsnag, etc. but they claimed to not help much in troubleshooting and improving the app functionality across all device manufacturers and OS versions. There was a time during this process where they incorporated additional software developers to help with the issues we were facing. Whether or not the SDK was added for a developer’s own financial gain or otherwise is beyond us and we don’t care to speculate. Again, the MsgSafe BETA mobile app was never released in a production supported version and has been abandoned for years, and we can confirm the mobile-first web UI, which is the only supported mobile interface in-use today and for the last few years, which does not contain any SDK from anyone. As far as how the MsgSafe mobile app obtained an “unobfuscated version” of the SDK? It is not our place to speculate, but it only makes sense that any company would provide updates to their software over time. The third-party app archive website containing MsgSafe’s APK, as referenced in the researcher’s post, is over 3 years old. It should come as no surprise that the software found there doesn’t match up exactly to the software found in apps they reported about in April 2022. Our developers probably didn’t even notice subtle changes like this because it’s not our practice to reverse engineer other company’s software and violate license agreements. Supplemental response: Yes, we agree this certificate you found embedded inside that software / SDK appears to be for "an1.msgsafe.io" and that it was NOT issued by TrustCor CA — it appears to be self-signed by either the developer who added it, or more likely the author of the actual SDK. This is a strong indication TrustCor CA was not involved in the software / SDK. Given a legitimately-signed certificate could have been integrated by MsgSafe.io’s own development team who could easily request it from TrustCor’s CA team, it certainly is evidence that the TrustCor Certificate Authority was not involved in this piece of software or its temporary addition to MsgSafe.io. Further, it indicates this is likely an act of an individual developer, and not even the MsgSafe.io team being duped. We looked at all the URLs you provided in your supplement and can confirm TrustCor CA has not issued any of those certificates at any time, and those URLs are not otherwise known to us. In Response to "… it appears to be programmed to instead send data to MsgSafe's own servers! Why are MsgSafe's servers hardcoded to receive data from Measurement Systems' SDK?": We had previously performed a forensic investigation of the an1.msgsafe.io hostname you found, along with its IP address history and the host itself, in addition to the certificate search (which again came up blank since it was not issued by TrustCor CA or any legitimate CA so far as we could tell). The hostname you mentioned pointed to a Linux VM that had been provisioned by the same developer we discussed. We were able to recover from backup a copy of this VM and forensically analyze it. The only user was a generic administration account (presumably known by the developer) and the only customization beyond any standard VM configuration was a firewall configuration and the open port 443 that accepted traffic from anywhere. The only listener attached to port tcp/443 was a proxy program which appears to be this: https://github.com/kklis/proxy The only version of the VM we were able to retrieve had no customizations to the configuration file, but it was set up using "systemd" so our conclusion was that the developer was likely redirecting the actual TCP stream somewhere else/offsite, and we confirmed the firewall configuration both on the host itself and the upstream router as far as the egress ACL/filter would have allowed this so long as he was sending it to any of tcp/443, tcp/8080, tcp/8181 or a few others. There is no indication the developer was "hosting" anything at MsgSafe.io beyond a simple redirector designed to send it somewhere else. There was no web server on this system and no certificates, so it was being redirected at the TCP level. This above disagrees with your statement "… MsgSafe's servers hardcoded to receive data ..." where you intimate that our company is receiving the data. A proper characterization would be that a developer had set up a TCP proxy to reflect data somewhere else, but not necessarily to receive it. What you wrote is at best a mischaracterization, or potentially another false claim as it may be forensically proven to be false. Thank you all, Rachel McPherson VP of Operations https://www.trustcor.com [email protected] +1 (289) 408-9998 -- You received this message because you are subscribed to the Google Groups "[email protected]" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/476F8066-5DFF-4348-8364-873C9B0418D8%40trustcor.ca.
signature.asc
Description: Message signed with OpenPGP
