Re: [Ledger-smb-devel] Plans for financial rewrite in 1.5
Hi Chris, On Thu, Jan 23, 2014 at 3:48 AM, Chris Travers chris.trav...@gmail.comwrote: Hi all; I would like to submit my plans for financial rewrite and suggest we discuss any expectations. Existing areas of testing and what we can do to ensure they are still met will be added below. Thanks for writing this up. My current thinking it's a good start to list the requirements. I'd like to add any requirements that we know the current system already meets, so we can be sure that a new design will not drop any of the important ones. The financial rewrite will be done in such a way as to allow 1.5 to be released even if it is not complete. However, we need to try hard to avoid making it a moving target, so hopefully this will provide some guidance there as well. I have basically scoped out the work required, and come up with the following requirements: 1. We need to be able to handle arbitrary two-currency or even three currency transactions. For example, if one has one's books in USD and transferring money from a GBP to an EUR account, this must be supported. Right. What's more, I'd like the design (although possibly not the immediate implementation) to allow for assets to be held in non-base currencies. E.g. to hold a USD current account in a company which is EUR based. Part of this requirement is the need to be able to revalue the asset -- report it as having a different current value than the value that it had when the posting was originally created. 2. We need to be able to handle per-transaction exchangerates. Right. I'd say that we need a per-date default exchange-rate as well -- one that's picked up if no other exchange rate is specified. To me, this requirement solves the following use-cases: - different banks (or bank and credit card) with different exchange rates for the same currencies on the same date - FX transaction reversal of a transaction in a closed period with a currency different from the current default 3. For reporting and referential integrity purposes, we need a more unified database structure. Some reports have test cases, notably the PNL reports. I must say that it's not clear to me what you mean with this one. Could you try to explain what it is that you consider to be required here? 4. For COGS tracking, we need to be able to audit how this is done, so as bugs are reported, we can track full changes of state across time. I have COGS tests btw. What does audit how it's done mean here? Do you plan to record all movements and their calculated COGS prices? Do you plan to write routines which produce intermediate results for specific transactions so transactions can be audited? Or do you mean anything else? 5. We need to get rid of the silly restriction that ensures you can't have accounts in different places of the ar/ap transaction screen. I'm not following here. Do you have a bug report or feature request to point to? Are you talking customer accounts (eca's) or gl accounts? Other requirements I'd like to keep in place: * Period closing * Period-end/closed period balance snapshots (for performance) * Recurring transactions of some sort * automatic fx result calculation based on the fx rates of the invoice and the payment What I think we want to get into place: * Period-end balance snapshots for AR/AP on a customer/eca level This would solve the issue that we've talked about where the POS customer would cumulate thousands of AR items and the POS would get slower due to that (trying to calculate the outstanding limit) [ Another solution would be to allow a POS module *not* to use a customer ] * Accrual/deferral transactions What I mean here is that we discussed how e.g. an interest-paid-in-arrears accrues interest until the customer is really invoiced. In my current business, I have yearly payments which I distribute over the year using monthly postings. We discussed that it's possible to have accrual/deferral be part of reporting, if we would create a specific transaction type. It's definitely something I'd like to help investigate viability. * A functional interface (before a webservice interface), so we get per posting/transaction integrity checks in place * Separation of duties based on the employee: I'd like to be able to support separation of duties where two employees would be able to post to the books, but only one can post the transactions of the other. (Really: the one who last edited the transaction can't post him/her self) * A different workflow for handling pre-/overpayments Entry of overpayments should be separated from handling normal payments Use of overpayments should be part of the normal receipt-handling workflow What I think we want to drop: * The current recurring transaction/invoices way of working To think about when creating our new implementation * A design (no implementation yet, I presume) for handling reconciliation (differently) in combination with automatic transaction
Re: [Ledger-smb-devel] Plans for financial rewrite in 1.5
Hi, On the whole, this all seems like a good strategy, and for what I understand of the system, I'm entirely on board. Couple comments: On 01/23/2014 02:37 PM, Erik Huelsmann wrote: Hi Chris, On Thu, Jan 23, 2014 at 3:48 AM, Chris Travers chris.trav...@gmail.com mailto:chris.trav...@gmail.com wrote: Hi all; 1. We need to be able to handle arbitrary two-currency or even three currency transactions. For example, if one has one's books in USD and transferring money from a GBP to an EUR account, this must be supported. Right. What's more, I'd like the design (although possibly not the immediate implementation) to allow for assets to be held in non-base currencies. E.g. to hold a USD current account in a company which is EUR based. Part of this requirement is the need to be able to revalue the asset -- report it as having a different current value than the value that it had when the posting was originally created. 2. We need to be able to handle per-transaction exchangerates. Right. I'd say that we need a per-date default exchange-rate as well -- one that's picked up if no other exchange rate is specified. To me, this requirement solves the following use-cases: - different banks (or bank and credit card) with different exchange rates for the same currencies on the same date - FX transaction reversal of a transaction in a closed period with a currency different from the current default One new consideration: Bitcoin. I just attended a talk yesterday that really made me see the value of digital currencies in terms of fraud prevention -- compared to credit cards, it's a much, much better deal and far less risky for merchants. But its value, obviously, fluctuates dramatically at any time. At least being able to store and report on accounts stored in BTC would be an easy way to become an early accounting system with support for it... then we could add a digital wallet support later as an add-on or something... See some of my thoughts here: http://www.freelock.com/blog/john-locke/2014-01/bitcoin-future-e-commerce * A functional interface (before a webservice interface), so we get per posting/transaction integrity checks in place Not sure I agree with this one. If we make the web service interface essentially extend an ORM to http, we can use http requests for testing. It's pretty easy to put together a generic test harness that would allow us to test requests and responses, aside from any UI issues. By functional do you mean functions in the database, or functional forms for user input? If the first, ok. If the last, I would like to see the REST interface use exactly the same code as the form handlers -- the form handlers just providing an HTML client layer that shares the same business logic. Cheers, John Locke http://www.freelock.com -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk___ Ledger-smb-devel mailing list Ledger-smb-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ledger-smb-devel
Re: [Ledger-smb-devel] Plans for financial rewrite in 1.5
On Thu, Jan 23, 2014 at 2:37 PM, Erik Huelsmann ehu...@gmail.com wrote: Hi Chris, On Thu, Jan 23, 2014 at 3:48 AM, Chris Travers chris.trav...@gmail.comwrote: Hi all; I would like to submit my plans for financial rewrite and suggest we discuss any expectations. Existing areas of testing and what we can do to ensure they are still met will be added below. Thanks for writing this up. My current thinking it's a good start to list the requirements. I'd like to add any requirements that we know the current system already meets, so we can be sure that a new design will not drop any of the important ones. The financial rewrite will be done in such a way as to allow 1.5 to be released even if it is not complete. However, we need to try hard to avoid making it a moving target, so hopefully this will provide some guidance there as well. I have basically scoped out the work required, and come up with the following requirements: 1. We need to be able to handle arbitrary two-currency or even three currency transactions. For example, if one has one's books in USD and transferring money from a GBP to an EUR account, this must be supported. Right. What's more, I'd like the design (although possibly not the immediate implementation) to allow for assets to be held in non-base currencies. E.g. to hold a USD current account in a company which is EUR based. Part of this requirement is the need to be able to revalue the asset -- report it as having a different current value than the value that it had when the posting was originally created. Sure. I am thinking of keeping the base + foreign system. Revaluing can then be done by querying foreign values and posting an adjustment against base. That seems the simplest and most transparent way to do it. 2. We need to be able to handle per-transaction exchangerates. Right. I'd say that we need a per-date default exchange-rate as well -- one that's picked up if no other exchange rate is specified. Yep. I didn't mention this but per currency you'd have an allowed variance from the default rate that could be set. This would default to 15% (meaning the per transaction report must be within 15% of the set rate). This is high enough I think to cover most variances but low enough to catch most cases of entering the rate in reverse or order of magnitude issues, To me, this requirement solves the following use-cases: - different banks (or bank and credit card) with different exchange rates for the same currencies on the same date - FX transaction reversal of a transaction in a closed period with a currency different from the current default 3. For reporting and referential integrity purposes, we need a more unified database structure. Some reports have test cases, notably the PNL reports. I must say that it's not clear to me what you mean with this one. Could you try to explain what it is that you consider to be required here? Instead of ar/ap/gl/transactions/acc_trans, I would like to have something which avoids a need to union ar, ap, and gl tables in reporting. We don't need a separate table per journal. 4. For COGS tracking, we need to be able to audit how this is done, so as bugs are reported, we can track full changes of state across time. I have COGS tests btw. What does audit how it's done mean here? Do you plan to record all movements and their calculated COGS prices? At least enough information to track it. The reason is that if someone says their COGS calculation seems off right now we have no way to say it is or is not, much less what went wrong. I don't expect this to be an issue in 1.4 but it has been an issue in 1.2. Do you plan to write routines which produce intermediate results for specific transactions so transactions can be audited? Or do you mean anything else? 5. We need to get rid of the silly restriction that ensures you can't have accounts in different places of the ar/ap transaction screen. I'm not following here. Do you have a bug report or feature request to point to? Are you talking customer accounts (eca's) or gl accounts? The issue I am thinking about is something like reserves for credit card processors. Some of the money gets put into a payable reserve account and becomes payable at a later date. On the AP transaction screen this basically means you have to have a liability account on the expense side and a different liability account on the AP summary side, and move money between them as the reserves expire. It would be much cleaner to allow an account to appear multiple places on the screen. We can't do that because there is no way of tagging the line item as to which part it goes on, and so if you have an AP_amount and AP_paid account, fun things happen when you enter stuff against it. Other requirements I'd like to keep in place: * Period closing * Period-end/closed period balance snapshots (for performance) * Recurring transactions of
Re: [Ledger-smb-devel] Plans for financial rewrite in 1.5
On Thu, Jan 23, 2014 at 3:54 PM, John Locke m...@freelock.com wrote: Hi, On the whole, this all seems like a good strategy, and for what I understand of the system, I'm entirely on board. Couple comments: On 01/23/2014 02:37 PM, Erik Huelsmann wrote: Hi Chris, On Thu, Jan 23, 2014 at 3:48 AM, Chris Travers chris.trav...@gmail.comwrote: Hi all; 1. We need to be able to handle arbitrary two-currency or even three currency transactions. For example, if one has one's books in USD and transferring money from a GBP to an EUR account, this must be supported. Right. What's more, I'd like the design (although possibly not the immediate implementation) to allow for assets to be held in non-base currencies. E.g. to hold a USD current account in a company which is EUR based. Part of this requirement is the need to be able to revalue the asset -- report it as having a different current value than the value that it had when the posting was originally created. 2. We need to be able to handle per-transaction exchangerates. Right. I'd say that we need a per-date default exchange-rate as well -- one that's picked up if no other exchange rate is specified. To me, this requirement solves the following use-cases: - different banks (or bank and credit card) with different exchange rates for the same currencies on the same date - FX transaction reversal of a transaction in a closed period with a currency different from the current default One new consideration: Bitcoin. I just attended a talk yesterday that really made me see the value of digital currencies in terms of fraud prevention -- compared to credit cards, it's a much, much better deal and far less risky for merchants. But its value, obviously, fluctuates dramatically at any time. At least being able to store and report on accounts stored in BTC would be an easy way to become an early accounting system with support for it... then we could add a digital wallet support later as an add-on or something... See some of my thoughts here: http://www.freelock.com/blog/john-locke/2014-01/bitcoin-future-e-commerce * A functional interface (before a webservice interface), so we get per posting/transaction integrity checks in place Not sure I agree with this one. If we make the web service interface essentially extend an ORM to http, we can use http requests for testing. It's pretty easy to put together a generic test harness that would allow us to test requests and responses, aside from any UI issues. By functional do you mean functions in the database, or functional forms for user input? If the first, ok. If the last, I would like to see the REST interface use exactly the same code as the form handlers -- the form handlers just providing an HTML client layer that shares the same business logic. It's a lot easier to test the db safely in a production environment than stateless web services since you can always guarantee no writes may be made permanent. From my perspective the db tests will always be the primary tests for accounting logic because we can guarantee they can be run safely on production systems, that we can read and write whatever we need to for the test and have it all go away at the end of the test run. Best Wishes, Chris Travers Cheers, John Locke http://www.freelock.com -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk ___ Ledger-smb-devel mailing list Ledger-smb-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ledger-smb-devel -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more.shtml -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk___ Ledger-smb-devel mailing list Ledger-smb-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ledger-smb-devel
[Ledger-smb-devel] Plans for financial rewrite in 1.5
Hi all; I would like to submit my plans for financial rewrite and suggest we discuss any expectations. Existing areas of testing and what we can do to ensure they are still met will be added below. The financial rewrite will be done in such a way as to allow 1.5 to be released even if it is not complete. However, we need to try hard to avoid making it a moving target, so hopefully this will provide some guidance there as well. I have basically scoped out the work required, and come up with the following requirements: 1. We need to be able to handle arbitrary two-currency or even three currency transactions. For example, if one has one's books in USD and transferring money from a GBP to an EUR account, this must be supported. 2. We need to be able to handle per-transaction exchangerates. 3. For reporting and referential integrity purposes, we need a more unified database structure. Some reports have test cases, notably the PNL reports. 4. For COGS tracking, we need to be able to audit how this is done, so as bugs are reported, we can track full changes of state across time. I have COGS tests btw. 5. We need to get rid of the silly restriction that ensures you can't have accounts in different places of the ar/ap transaction screen. So my plan is basically as follows: 1. Rewrite currency handling, and provide an opportunity to set per-currency allowed exchangerate variances. This option would be initially hidden by css. Initially I will just propagate the changes to the defaults table to avoid messing with the old code. This will be done in trunk and affect the full release both new and old code. 2. Write a journal entry module. Basic test cases would be written including on the db-side posting and reversing a balanced transaction, and on the Perl side, attempting to post an unbalanced transaction (and having it fail). I am not quite sure how to handle this in terms of save points, but this can be figured out. Extending our current test framework to add this will not be problematic. After this point, I would really like new reports to include logic against this db model instead of the old acc_trans tables. 3. Port the invoice module I wrote using PGObject::Simple::Role from wxpos to Moose and move this area over. We'd remove payment handling since that would be separate. I don't expect this to be too hard. 4. We'd write a new payment object model following this. Once this is done, wxpos can take over some part of the role of a data insert tool for reports development etc. I expect that the end result will be some of our current sql will be portable, and a lot of stuff will be rewritten. This may be one of the harder aspects. 5. The reconciliation module would need to be ported. This is not likely to be too hard. Existing reconciliation tests would be expected to pass. 6. The financial reports would be gone through and ported over. 7. At that point we'll branch, switch-over, make sure tests work, and pull the version with the removed code back into 1.5. With the advancements that have taken place in 1.4, I expect 1-3 to happen fairly quickly. The real unknowns are 4 and 6. Any feedback? -- Best Wishes, Chris Travers Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in. http://www.efficito.com/learn_more.shtml -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk___ Ledger-smb-devel mailing list Ledger-smb-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ledger-smb-devel
