Re: Problem after Ubuntu Upgrade from 16.04 to 18.04
Yup, that did it.I just replaced the TrustedTimestamps.php with the newest
from the master branch and it worked again.
Thanks for the help!
Frank
Am Dienstag, 6. April 2021, 18:36:56 MESZ hat Folgendes
geschrieben:
Hello Frank,
in openssl 1.1.x the output of the openssl command has changed. Assuming
your mentioned piler installation is older than 2020-10-17, then I
suggest
to try updating system/helper/TrustedTimestamps.php from the master
branch,
and let's see how it goes.
Janos
On 2021-04-06 16:50, Frank Schmitz wrote:
> Hello all,
>
> I had to perform an Ubuntu Upgrade to 18.04 on a server running an
> older version of piler.
>
> The update ran without any problems, but the piler Web-GUI seems to be
> "broken".
>
> I get the following error in the apache log (pseudo anonymized):
>
> [Tue Apr 06 16:35:53.008548 2021] [php7:error] [pid 14368] [client
> xxx:64562] PHP Fatal error: Uncaught Exception: Systemcommand failed:
> Using configuration from /usr/lib/ssl/openssl.cnf, Verification: OK in
> /var/www/xxx/htdocs-ssl/system/helper/TrustedTimestamps.php:186\nStack
> trace:\n#0 /var/www/xxx/htdocs-ssl/model/search/message.php(512):
> TrustedTimestamps::validate('1c79bef0265fxxx...',
> 'MIIVSzADAgEAxxx...', '1617716xxx', '/var/piler/free...')\n#1
> /var/www/xxx/htdocs-ssl/model/search/message.php(51):
> ModelSearchMessage->check_rfc3161_timestamp_for_id('2519')\n#2
> /var/www/xxx.com/htdocs-ssl/model/search/message.php(228):
> ModelSearchMessage->verify_message('4000606xxx8...', 'Return-Path:
> ModelSearchMessage->extract_message('4000606cxxx...', '')\n#4
> /var/www/xxx/htdocs-ssl/system/front.php(36):
> ControllerMessageView->index()\n#5
> /var/www/xxx/htdocs-ssl/system/front.php(14): Fro in
> /var/www/xxx/htdocs-ssl/system/helper/TrustedTimestamps.php on line
> 186
>
> I see that TrustedTimestamps.php throws an exception, but I don't
> understand why...
> It says "Verification: OK" which should indicate that everything has
> gone well, right?
>
> openssl is installed (1.1.1), the cron job signing messages using
> TrustedTimestamps.php is also running fine.
>
> But using the gui throws an exception...
>
> Help anyone?
>
> Kind regards
> Frank Schmitz
Problem after Ubuntu Upgrade from 16.04 to 18.04
Hello all,
I had to perform an Ubuntu Upgrade to 18.04 on a server running an older
version of piler.
The update ran without any problems, but the piler Web-GUI seems to be "broken".
I get the following error in the apache log (pseudo anonymized):
[Tue Apr 06 16:35:53.008548 2021] [php7:error] [pid 14368] [client xxx:64562]
PHP Fatal error: Uncaught Exception: Systemcommand failed: Using configuration
from /usr/lib/ssl/openssl.cnf, Verification: OK in
/var/www/xxx/htdocs-ssl/system/helper/TrustedTimestamps.php:186\nStack
trace:\n#0 /var/www/xxx/htdocs-ssl/model/search/message.php(512):
TrustedTimestamps::validate('1c79bef0265fxxx...', 'MIIVSzADAgEAxxx...',
'1617716xxx', '/var/piler/free...')\n#1
/var/www/xxx/htdocs-ssl/model/search/message.php(51):
ModelSearchMessage->check_rfc3161_timestamp_for_id('2519')\n#2
/var/www/xxx.com/htdocs-ssl/model/search/message.php(228):
ModelSearchMessage->verify_message('4000606xxx8...', 'Return-Path:
extract_message('4000606cxxx...', '')\n#4
/var/www/xxx/htdocs-ssl/system/front.php(36):
ControllerMessageView->index()\n#5
/var/www/xxx/htdocs-ssl/system/front.php(14): Fro in
/var/www/xxx/htdocs-ssl/system/helper/TrustedTimestamps.php on line 186
I see that TrustedTimestamps.php throws an exception, but I don't understand
why...It says "Verification: OK" which should indicate that everything has gone
well, right?
openssl is installed (1.1.1), the cron job signing messages using
TrustedTimestamps.php is also running fine.
But using the gui throws an exception...
Help anyone?
Kind regardsFrank Schmitz
Re: Piler 1.3.5 on Ubuntu 18.04 WebUI doesn't render timestamped Mails
Regarding TSA https://freetsa.org/ works well for me and is completely free... Am Freitag, 16. Oktober 2020, 21:20:02 MESZ hat Folgendes geschrieben: Hello Patrick, for starters the DECRYPT_ATTACHMENT_BINARY should be pileraget (not pilerget). Also it might be worth to go over the steps and check if everything is setup properly: https://bitbucket.org/jsuto/piler/issues/480/support-rfc3161-trusted-timestamps https://www.mail-archive.com/piler-user@list.acts.hu/msg00785.html In the meantime I try to find a TSA provider suitable for testing purposes. If anyone on the list was familiar with any, or better could provide a test account, it would be great. Janos On 2020-10-16 20:02, Patrick Wagner wrote: > Hello everyone, > > we're testing the TSA signing feature of Piler. When I login as > auditor all mails are listed correctly in the Web GUI upper pane / > list. When I click on a mail the header and content are displayed > below if that mail was not signed (yet). Clicking on a signed mail > however does not refresh the lower pane and continues to display > either the last non-stamped mail or remains blank (if no mail had been > selected before). With pilerget on the CLI the mails are displayed > correctly. > > What's wrong? Do I have to change any configuration? > > I added these lines in config-site.php: > $config['TSA_PUBLIC_KEY_FILE'] = '/etc/piler/tsa.publickey.pem'; > $config['TSA_START_ID'] = 1; > $config['TSA_STAMP_REQUEST_UNIT_SIZE'] = 500; > $config['TSA_URL'] = 'http://zeitstempel.dfn.de'; > > $config['DECRYPT_BINARY'] = '/usr/bin/pilerget'; > $config['DECRYPT_ATTACHMENT_BINARY'] = '/usr/bin/pilerget'; > > > Thanks, > Patrick > > # piler -V > piler 1.3.5, build 997, Janos SUTO > > Build Date: Sun Apr 21 16:50:30 UTC 2019 > ldd version: ldd (Ubuntu GLIBC 2.27-3ubuntu1) 2.27 > gcc version: gcc version 7.3.0 (Ubuntu 7.3.0-27ubuntu1~18.04) > OS: Linux 188ae4f9894f 4.15.0-47-generic #50-Ubuntu SMP Wed Mar 13 > 10:44:52 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux > Configure command: ./configure --prefix=/usr --sysconfdir=/etc > --localstatedir=/var --with-database=mariadb > MySQL client library version: 10.2.9 > Extractors: /usr/bin/pdftotext /usr/bin/catdoc /usr/bin/catppt > /usr/bin/xls2csv /usr/bin/unrtf /usr/bin/tnef > > Ubuntu 18.04.5 LTS > > PHP 7.2.24-0ubuntu0.18.04.6 (as supplied with the distro) > > MariaDB 10.1.44 (as supplied with the distro)
Re: Timestamp Authority "disbanded", what to do now?
Hi Janos, yeah I thought so too.The TSA isn't completely gone though, it currently just doesn't issue any new timestamps but checking hashes still seems to work.Even though I probably don't want to risk it disappearing completely. So any idea how I can restamp all mails in the archive? The stamps are all in the database right? Delete all hashes from the database and then simply run the stamping process? The archive is really small, but can I really stamp all mails at one time or do I need several timestamps? Thanks for the help! Am Sonntag, 14. April 2019, 21:41:55 MESZ hat Folgendes geschrieben: Hello Frank, well, I think your only option is to re-stamp existing emails, since you can't really use them to validate without the (now disappeared) TSA authority who issued them. Janos On 2019-04-13 23:33, Frank Schmitz wrote: > seems like the timestamping authority I used "disbanded" and kind of > tells me to go elsewhere for timestamping. > > Sure, I can configure another TSA to get NEW timestamps, but what > about all the old timestamps in my archive? > Do I have to restamp every mail in the archive? > What's the best way to do that? > Can I simply add another certificate to be able to verify older mails > with the old timestamps? > Anyone ever had this problem before? > > Thankful for any advice...
Timestamp Authority "disbanded", what to do now?
Hello,seems like the timestamping authority I used "disbanded" and kind of tells me to go elsewhere for timestamping. Sure, I can configure another TSA to get NEW timestamps, but what about all the old timestamps in my archive?Do I have to restamp every mail in the archive?What's the best way to do that? Can I simply add another certificate to be able to verify older mails with the old timestamps?Anyone ever had this problem before? Thankful for any advice...
Re: timestamp feature
Hi, actually you can choose which way piler uses timestamps. If you want, you can use a different timestamp for each incoming mail. This strategy might prove quite costly though, since most TSAs bill you for each timestamp they issue to you (And you won't know how much they charge you each day beforehand). The other way is to use one timestamp for all incoming mails within a specified timeframe (every 1/2/3 hours, every hour from 09:00 to 18:00 and so on). This way, you know beforehand how many timestamps you will need per day and how much that will cost you. When I set this up, I chose a TSA (http://tsa.safecreative.org/) that gives away 5 free timestamps per day and I configured piler to create timestamps every 2 hours from 9:00 to 17:00 (business hours) -> No costs and still tamper proof according to german law (revisionssicher/GDPR). Regarding your question about deleting a mail: Timestamps are stored in a different DB table than emails. They have their own ID and they also store the range of email IDs this timestamp is valid for.In another table you'll find all mails with their unique IDs. Even though I never tried it (different setup), you should be able to delete individual mails without a problem, since the timestamp is not stored within the mail itself. Instead the timestamp "knows" which mails it's valid for. Hope that helps,Frank Christian Röser - PELMA schrieb am 10:56 Mittwoch, 7.Februar 2018: Hello, I have some questions about the timestamping feature. As far as I understand from this posthttps://www.mail-archive.com/piler-user@list.acts.hu/msg00785.html piler collects a bunch of mails, generates a hash for all of them and then this hash is signed by the tsa server. To verify a single mail, piler looks up what hash belongs to this e-mail, what other e-mails where involved, computes the hash for all of them and then compares this hash with the signed hash. Am I right? Now I want (have to) use the delete feature of piler. For example in Germany you have to delete job applications after some time. Although we have a separate e-mail address for such stuff, which will not be archived, it happens from time to time that someone sends a mail to the a general address like info@. Now german law tells me that I have to delete such mails. What does that mean for the timestamp feature? If I delete a mail, that message is no longer available for the hash computing. Does that mean that the verification for a some mails or – if I use the unit way – some hundred or thousand emails fails? Do I have get a stamp for every single mail? Is this even possible? Or exists there some magic in piler to prevent this?😉 Best regards, Christian
Re: GOBD certification
Hi Janos, since I didn't write a procedural manual for my girlfriends business (yet, of course ;-)) and reading about you and Zyrixx planning to write such a "monster" (the linked sample has 42 pages and still needs to be completed!) http://www.awv-net.de/upload/pdf/Belegablage_V1_20151026.pdf I finally took a short dive into the sample manual. I think it's REALLY good and does have quite a lot of explanations on how to apply it to your company. My opinion: If this is not enough, you're the wrong guy for this job, give it to someone who knows your company better than you do ;-) The only chapter that would really profit from your input (again, in my opinion of course) is 3.1 in which the used hard- and software needs to be explained (in regard to GoBD compliance). If you provide a good explanation of how it's possible to configure piler to be GoBD compliant for this chapter, everything else can be taken and modified from this sample document, no need to reinvent the wheel here... Interestingly, the authors recommend to use "certified software" and even provide a chapter to list the certifications... But I still don't think it relevant, especially for a small (digital) business... You have to describe how you configured the "certified" software and how you use it exactly like you would describe it for uncertified software anyway. Come to think of it, it actually might be counterproductive if you had to tweak the certified software a lot to make it work for your company. According to the authors, the more you have to "customize" the software, the more "suspicious" the auditors will get ;-) Personally I think GoBD had two main goals: 1.) A crack down on tax evasion by using cash transactions (If you use cash, you really HAVE to get a GoBD compliant cash register, no way around it) 2.) To make an auditors life easier As I already said, GoBD compliant email storage (contrary to HW handling cash transactions!) is not a law, therefore no auditor can punish you for not abiding to it (Of course it might piss the auditor off and make him look really closely at things, which can be a costly mistake too...). If you don't take cash and all the money comes in by bank transfer, Paypal et. all, there is absolutely no way you could cheat anyway, since all bank transfers can be traced... Which is the reason I didn't put too much effort into writing a manual for my girlfriend incidentally... If an auditor really is interested in such a small business, you can go like "Here's the archive. Don't trust it or don't understand the IT behind it? Fine, go and check the bank transfers and have fun..." Every time I look into the admin panel I have to laugh: Less than 400 mails since January 2017 and the archive/server will probably be full in "34755 years, 10 months, 15 days" :-) Funny side notes: - If you get invoices only per snail mail, you actually don't have to archive them electronically - If you only use a "Template" on a computer to print your invoices and do NOT save the invoice itself on the computer (Using it like a typewriter), you don't have to store the invoices electronically either Since we're talking about funny stuff, I also found a comment from a guy on a german message board, who actually called his local tax authority and asked about GoBD. He spoke to several persons including an auditor, who told him saving invoices in PDF format on his harddrive is perfectly fine, since PDFs can't be changed afterwards... ROFL! Here's the link if anyone is interested, it does have a really good explanation of GoBD requirements, even though there are some mistakes (like time-critical saving of digital invoices, which only need to be saved as a file within 10 days but NOT also archived within 10 days) in it, but it is in german: http://www.selbstaendig-im-netz.de/2017/04/12/selbstaendig/gobd-darauf-musst-du-bei-digitalen-rechnungen-und-belegen-achten/ Kind regards, Frank "s...@acts.hu" schrieb am 21:42 Dienstag, 25.Juli 2017: Hello Frank, On 2017-07-25 14:47, Frank Schmitz wrote: > > a GoBD certification would surely result in a higher "visibility" for > piler, since quite a lot of companies are basing their business > decisions on those. > > But please do NOT believe that Piler needs a GoBD certification to be > used in germany! > The ministry of finance in germany does not care whether the software > is certified, it cares about > > 1. whether the software fulfills the legal requirements (i.e. to use > piler for GoBD in germany you need to use timestamping) do you mean that it's mandatory to use an external timestamp provider with piler? Or do you refer to the timestamps piler provides and stores in the metadata table? > 2. how
Re: GOBD certification
Hi Janos, see below... "s...@acts.hu" schrieb am 21:42 Dienstag, 25.Juli 2017: Hello Frank, On 2017-07-25 14:47, Frank Schmitz wrote: > > a GoBD certification would surely result in a higher "visibility" for > piler, since quite a lot of companies are basing their business > decisions on those. > > But please do NOT believe that Piler needs a GoBD certification to be > used in germany! > The ministry of finance in germany does not care whether the software > is certified, it cares about > > 1. whether the software fulfills the legal requirements (i.e. to use > piler for GoBD in germany you need to use timestamping) do you mean that it's mandatory to use an external timestamp provider with piler? Or do you refer to the timestamps piler provides and stores in the metadata table? Well, it depends. The crux of the matter is the so called "revisionssicherheit". Meaning the company must not be able to manipulate anything in the archive. If you host piler yourself, you can obviously manipulate anything in the archive by replacing encrypted mails and "correcting" the hashes within the database, since you have full access to the server piler is hosted on. By adding a third party (i.e. the Time Stamping Authority), this is no longer possible without a verification fail. As far as I understand it, you can't even delete an email from the archive without it being quite obvious in the database (Missing Timestamp etc.). If you actually pay a third party that is hosting piler, this might not be necessary since the company has no direct access to the server... On the other hand, the company might be able to bribe someone from the hoster to manipulate the archive ;-) The hoster might want to use timestamps too to make that temptation a no-brainer... So no, it's probably not mandatory in all use cases, but if you host piler yourself you need it to meet the legal requirements... > 2. how the company USES the software (You need a procedural > documentation) I'll make it soon. That might actually be impossible, since every company is handling mails differently, therefore you cannot write a one-size-fits-all procedure... What would probably help are examples of differently sized companies (Even though that might be more effort than it's worth...) Apart from that, a good explanation why piler cannot be manipulated (even if self-hosted) without it being obvious might help in any discussion with an auditor. I set piler up for my girlfriend and her really small business and she certainly could not explain why her mailarchive cannot be manipulated to an auditor... It probably would also help, if you could check the timestamp validity within the GUI, i.e. call the TSA validity check with the timestamp used for the mail in a new browser windows :-) > In case of an audit, both will be checked and the auditor won't care > whether piler is certified or not... > > To prove the point: > https://www.bundesfinanzministerium.de/Content/DE/Downloads/BMF_Schreiben/Weitere_Steuerthemen/Abgabenordnung/Datenzugriff_GDPdU/2014-11-14-GoBD.pdf?__blob=publicationFile > > This is an official statement from the ministry of finance in germany, > under heading 12 / 181 (last page) it reads: > > _„Zertifikate“ oder „Testate“ Dritter können bei der Auswahl > eines Softwareproduktes _ > _dem Unternehmen als Entscheidungskriterium dienen, entfalten jedoch_ > > _ aus den in Rz. 179 genannten Gründen gegenüber der Finanzbehörde > keine Bindungswirkung._ > > Roughly translated: > > Certificates or testimonies of third parties may be used by companies > to choose a software, but they DO NOT have a binding effect for the > ministry of finance because of the reasons named in Rz. 179. I see. My point is that the usefulness of the certificate is that the auditing company has examined the given software and by providing the gobd compatible stamp they verify that the software complies with all demands by the law. The funny thing is, there simply is no certificate for GoBD compliance (As of yet, at least). I hesitate to call it a scam, even while less respectable companies use exactly that description to market their products, since the products (probably :-)) fulfill the legal requirements. In germany a "certificate" goes a long way to impress customers... You could probably put a "Bio(logical)" certificate (self-designed label and different from existing ones) on piler without any problems, since that term/certificate is not protected at all in germany :-))) (We have other less obvious certifications for that) The one from audicon is not actually a GoBD certification, but a "IDW PS 880" compliance check. Which kind of means more or less the same, since that is a compliance "check" of
Re: GOBD certification
Hi Janos, a GoBD certification would surely result in a higher "visibility" for piler, since quite a lot of companies are basing their business decisions on those. But please do NOT believe that Piler needs a GoBD certification to be used in germany!The ministry of finance in germany does not care whether the software is certified, it cares about 1. whether the software fulfills the legal requirements (i.e. to use piler for GoBD in germany you need to use timestamping)2. how the company USES the software (You need a procedural documentation) In case of an audit, both will be checked and the auditor won't care whether piler is certified or not... To prove the point: https://www.bundesfinanzministerium.de/Content/DE/Downloads/BMF_Schreiben/Weitere_Steuerthemen/Abgabenordnung/Datenzugriff_GDPdU/2014-11-14-GoBD.pdf?__blob=publicationFile This is an official statement from the ministry of finance in germany, under heading 12 / 181 (last page) it reads: „Zertifikate“ oder „Testate“ Dritter können bei der Auswahl eines Softwareproduktes dem Unternehmen als Entscheidungskriterium dienen, entfalten jedoch aus den in Rz. 179 genannten Gründen gegenüber der Finanzbehörde keine Bindungswirkung. Roughly translated: Certificates or testimonies of third parties may be used by companies to choose a software, but they DO NOT have a binding effect for the ministry of finance because of the reasons named in Rz. 179. In short, those certificates for GoBD compliance aren't worth the paper they're written on if an auditor is knocking on your door... You must be able to show that piler is able to fulfill all GoBD Requirements. For a (german) "checklist" you can look those up here 15 Kriterien für GoBD-konforme Software | Scopevisio Ratgeber or here Neue GoBD: Ein umfassender Überblick. I'm no expert by any means, but as far as I understand it, piler is quite capable of doing all that IF you use timestamping so you can prove the emails haven't been changed since they were timestamped/received... Apart from the technical requirements, you will also need an "extensive" documentation about what exactly you are doing with your receipts/invoices/etc. You can even find a sample documentation to use here: GoBD - Verfahrensdokumentation, praxisrelevante Hilfestellungen / PSP München if you have no idea what to do... Make no mistake, THIS is what really matters to an auditor! (Well, maybe not if you use really crappy software ;-)) So unless you really want to spend several thousand euros on increasing the visibility/user base of piler, I would recommend you forget about purchasing a GoBD certificate... Speaking for myself, I would certainly consider throwing a bit of money into crowdfunding "useful" additions to piler, but for this I won't pay anything at all, sorry... Kind regards,Frank | | | | || | | | | | GoBD - Verfahrensdokumentation, praxisrelevante Hilfestellungen / PSP München Von PSP / Developer: Michael Cammannn Erfahren Sie in den Fachbeiträgen zur Verfahrensdokumentation, wie ein Unternehmen die Vollständigkeit, Ordnung ... | | | | | | | Neue GoBD: Ein umfassender Überblick Von Stefan Groß VeR-Vorsitzender Stefan Groß erklärt in diesem Fachbeitrag, was Unternehmen über die neuen Buchführungsregeln wi... | | | | | | 15 Kriterien für GoBD-konforme Software | Scopevisio Ratgeber Die GoBD definieren Anforderungen an elektronische Buchführung. Unsere Checkliste sagt Ihnen, ob Ihre Software a... | | | "s...@acts.hu" schrieb am 20:56 Montag, 24.Juli 2017: Dear piler users (especially the German ones at this time), probably all of you have heard that German law mandates that all German businesses must archive emails starting from this year. I've been convinced that it's worth to obtain GOBD certification for piler to offer the choice of freedom to German users to pick an open source email archiving product. Unfortunately it costs huge money* even for open source applications as well, so I thought some crowdfunding might help getting the required amount of money. For a successful fundraising campaign I need some helpers from Germany to spread the word (email, social media, etc) before the actual campaign launches, pick a crowdfunding platform, donating the initial seed (I have a small donation in mind, like 5 EUR or so), etc. If you agreed and willing to help, then please drop me a line. Perhaps I'll create a mailing list for the topic. Janos *: I got a quote from Audicon of 6500 EUR (not sure if any additional costs may occur during the process)
