[Ledger-smb-devel] Proposed framework changes for 1.4
Hi all; INTRODUCTION: Looking towards 1.4 and trying to improve things significantly from where they are on 1.3. In general, I'd hope that with each new version, these changes become more incremental (and smaller) until we are ready to start on 2.0. The goals of these proposals are as follows: 1) To future-proof the LedgerSMB architecture, trying to standardize on something solid, building on what works well with the 1.3 codebase and addressing shortcomings that come up in relation to the directions we have been going. 2) To address specific issues we will run into in the areas we are re-engineering in 1.4 and later. 3) To better support a variety of architectures, such as mod_perl, going forward. All changes in this document are backwards compatible, so unless otherwise noted we don't have to rewrite code specifically because of this document though we will rewrite for other reasons through the 1.x process. WHAT 1.3 DOES WELL == 1.3 is a substantial accomplishment. We've added real security to an inherited codebase, and we've added impressive capabilities at the same time, and without serious bugs post-1.3.0 release in the core functions (those which most users use). Major reasons for the successes include: 1) A focus on declarative logic in the database, both for permissions and for data constraints 2) A clear separation of presentation and data logic. The database procedures handle data logic and the presentation is handled by Perl and TT. SHORTCOMINGS OF 1.3 The code during the 1.3 development process evolved quite a bit as maintainability issues were uncovered and addressed and coding styles evolved. This makes 1.3 a bit of a mishmash of coding paradigms and we can expect some maintenance headaches as a result. Longer-run we need more standardization in particular, and coding practices that are more conducive to writing great documentation. This should make it easier for people to follow new code and contribute. In particular some of the early 1.3 code is relatively unstructured, and classes don't have specific properties associated. This makes the code impossible to document and consequently harder to manage and get contributions for going forwards. A second major issue has been the issue of stored procedure discovery and the interface between these and the application. The issue is that the stored procedures aren't discoverable enough--- one can determine that a specific function name exists, but then invoking it, we can't tell what are intrinsic vs extrinsic arguments. In other words, we can't differentiate a property from a (presumably) pre-existing objects and those which are expected to be new properties supplied directly by the end user. Finally, I am not entirely happy with the dependency on PostgreSQL contrib modules especially since this is changing to a new extension system. This adds needless complexity in areas where we don't necessarily need it. FUTUREPROOFING CONSIDERATIONS == One of the goals of the LedgerSMB project has been to make the LedgerSMB logic reusable not only in other web apps but other applications in other environments and even other languages. The emphasis should be on coding conventions at the lowest levels that make it possible in the future to implement add-ons in Python, PHP, etc... The second side of this though is continuing the separation of presentation and logic. Support for environments like mod_perl is not yet handled, and can't be handled easily with our current codebase. Further separation would mean less of a need to change things down the road, since the web handling logic could be carefully contained in a single module which wouldn't be necessary to include for console applications, etc. SPECIFIC CHALLENGES OF 1.4 AND LATER == With the move into the financial logic in 1.4, a couple of specific challenges appear. The big ones relate to storage and reporting of journal entries, along with GL reports. On the first, we want to make sure that journal entries are balanced and reject entries that are not. Doing this with the 1.3 framework has a number of obstacles, because we can't easily pass the structures into the database in a discoverable format. The second has to do with reporting. In the current framework we cannot add running totals to the db queries gracefully without affecting our ability to re-order results. Adding running totals here would be quite useful and I think we should be looking at doing this. PROPOSALS Proposal 1: Use Moose 2.x -- Using Moose 2.x would give us an ability to declare properties and constraints on those properties in a declarative, clear, and consistent way. These can then be easily documented and so forth. It also gives us access to BEFORE/AROUND/AFTER syntax for overriding existing methods gracefully. These
[Ledger-smb-devel] upgrade db 1.2 - 1.3 experiences
Hello, -- I had to do following extra-steps before being able to upgrade db without errors: dangling acc_trans , invoice delete from lsmb12.acc_trans where trans_id in (SELECT trans_id FROM lsmb12.acc_trans where trans_id not in (select id from lsmb12.ap union select id from lsmb12.ar union select id from lsmb12.gl) group by trans_id); delete from lsmb12.invoice where trans_id in (SELECT trans_id FROM lsmb12.invoice where trans_id not in (select id from lsmb12.ap union select id from lsmb12.ar union select id from lsmb12.gl) group by trans_id); Is this something we should check in sql/upgrade/1.2-pre-upgrade-checks.sql ? -- i had to do following extra-step before being able to work with upgraded db: update parts p set expense_accno_id=(select c.id from chart c,lsmb12.chart lc where c.accno=lc.accno and lc.id=p.expense_accno_id); Should we not adapt import of parts in e.g. sql/upgrade/1.2-1.3-manual.sql --INSERT INTO parts SELECT * FROM lsmb12.parts; insert into parts select id,partnumber,description,unit,listprice,sellprice,lastcost,priceupdate,weight,onhand,notes,makemodel,assembly,alternate,rop,inventory_accno_id,income_accno_id, --expense_accno_id, (select c.id from chart c,lsmb12.chart lc where c.accno=lc.accno and lc.id=l12parts.expense_accno_id), bin,obsolete,bom,image,drawing,microfiche,partsgroup_id,project_id,avgcost person from lsmb12.parts l12parts idem for inventory_accno_id or income_accno_id ?? otherwise, i get error: INSERT INTO acc_trans (trans_id, chart_id, amount, transdate, project_id, invoice_id) VALUES (?, ?, ?, ?, ?, ?) ERROR: insert or update on table acc_trans violates foreign key constraint acc_trans_chart_id_fkey DETAIL: Key (chart_id)=(10841) is not present in table account. Salutations, Herman Vierendeels -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ Ledger-smb-devel mailing list Ledger-smb-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ledger-smb-devel
Re: [Ledger-smb-devel] upgrade db 1.2 - 1.3 experiences
On Tue, Oct 18, 2011 at 7:05 AM, herman vierendeels herman.vierende...@gmail.com wrote: Hello, -- I had to do following extra-steps before being able to upgrade db without errors: dangling acc_trans , invoice delete from lsmb12.acc_trans where trans_id in (SELECT trans_id FROM lsmb12.acc_trans where trans_id not in (select id from lsmb12.ap union select id from lsmb12.ar union select id from lsmb12.gl) group by trans_id); delete from lsmb12.invoice where trans_id in (SELECT trans_id FROM lsmb12.invoice where trans_id not in (select id from lsmb12.ap union select id from lsmb12.ar union select id from lsmb12.gl) group by trans_id); Is this something we should check in sql/upgrade/1.2-pre-upgrade-checks.sql ? Probably a good idea. Go for it. -- i had to do following extra-step before being able to work with upgraded db: update parts p set expense_accno_id=(select c.id from chart c,lsmb12.chart lc where c.accno=lc.accno and lc.id=p.expense_accno_id); Should we not adapt import of parts in e.g. sql/upgrade/1.2-1.3-manual.sql --INSERT INTO parts SELECT * FROM lsmb12.parts; insert into parts select id,partnumber,description,unit,listprice,sellprice,lastcost,priceupdate,weight,onhand,notes,makemodel,assembly,alternate,rop,inventory_accno_id,income_accno_id, --expense_accno_id, (select c.id from chart c,lsmb12.chart lc where c.accno=lc.accno and lc.id=l12parts.expense_accno_id), bin,obsolete,bom,image,drawing,microfiche,partsgroup_id,project_id,avgcost person from lsmb12.parts l12parts idem for inventory_accno_id or income_accno_id ?? yep. Good catch. Yeah, go ahead and fix this. Best Wishes, Chris Travers -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ Ledger-smb-devel mailing list Ledger-smb-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ledger-smb-devel
Re: [Ledger-smb-devel] Proposed framework changes for 1.4
Hi, Sounds like a good direction to me. Couple thoughts -- not being that active in Perl, don't know if these are realistic or not, but just what comes to mind, feel free to disregard. - Are there really no other date or number libraries available that do what we need, to the point we need to create our own? (or are these primarily wrappers around something else?) - Ran across this project recently: http://ledger-cli.org/ - a Perl cli app that does double-entry accounting in text files. I wonder if they've worked out some good accounting logic we can use, at a minimum -- or perhaps incorporate into LSMB? I'm thinking with a little collaboration, ledger-cli might be extended to use LSMB as back end storage, while LSMB might potentially leverage the work already done in ledger? Have not looked at the code, have no idea whether this is practical. But an easier CLI interface sounds very useful, if it's not too much of a stretch. And might be a good gateway to new users... - Regarding extensions, I just had to do some upgrades to some other web apps where customizations and extension code was scattered across a bunch of libraries. Basically an MVC organization -- but all extension code had to be split up, with the M in the M, the V in the V, and the C in the C directories I just want to express my extreme distaste for that file organization, from an ongoing maintenance point of view. Using a framework for development is great, as long as we can set it up in a way so that if I want to add some sort of custom module to the site, I can put all the code for it in one directory. Extending that last point a bit -- might it be possible to re-organize the core modules this way, to provide good examples? Instead of having a stub in the root, a file in scripts, the class in LedgerSMB, etc -- might it be possible to have a modules directory with each module in it's own subdirectory? modules/AR, modules/AP, modules/HR, etc. And then inside those, if necessary, mirror the rest of the file structure for MVC -- modules/AR/scripts, modules/AR/LedgerSMB, modules/AR/UI, etc? And to take it one step further, what Drupal has done brilliantly is provided an override mechanism -- if you want to override a core module, you can simply copy it to a place that gets checked first, then hack away, leaving the original code intact. Then an upgrade won't clobber your customizations, and you can much more easily do a simple diff to see what's changed. LSMB doesn't need to have the multiple paths that Drupal uses (I don't think) but maybe something as simple as a path that's checked first for a module - local/ comes to mind. You wanna hack AR? Copy modules/AR to local/AR and go to it. Cheers, John On 10/18/2011 04:49 AM, Chris Travers wrote: Hi all; INTRODUCTION: Looking towards 1.4 and trying to improve things significantly from where they are on 1.3. In general, I'd hope that with each new version, these changes become more incremental (and smaller) until we are ready to start on 2.0. The goals of these proposals are as follows: 1) To future-proof the LedgerSMB architecture, trying to standardize on something solid, building on what works well with the 1.3 codebase and addressing shortcomings that come up in relation to the directions we have been going. 2) To address specific issues we will run into in the areas we are re-engineering in 1.4 and later. 3) To better support a variety of architectures, such as mod_perl, going forward. All changes in this document are backwards compatible, so unless otherwise noted we don't have to rewrite code specifically because of this document though we will rewrite for other reasons through the 1.x process. WHAT 1.3 DOES WELL == 1.3 is a substantial accomplishment. We've added real security to an inherited codebase, and we've added impressive capabilities at the same time, and without serious bugs post-1.3.0 release in the core functions (those which most users use). Major reasons for the successes include: 1) A focus on declarative logic in the database, both for permissions and for data constraints 2) A clear separation of presentation and data logic. The database procedures handle data logic and the presentation is handled by Perl and TT. SHORTCOMINGS OF 1.3 The code during the 1.3 development process evolved quite a bit as maintainability issues were uncovered and addressed and coding styles evolved. This makes 1.3 a bit of a mishmash of coding paradigms and we can expect some maintenance headaches as a result. Longer-run we need more standardization in particular, and coding practices that are more conducive to writing great documentation. This should make it easier for people to follow new code and contribute. In particular some of the early 1.3 code is relatively unstructured, and classes don't have specific properties associated. This makes the code impossible to
Re: [Ledger-smb-devel] Proposed framework changes for 1.4
On Tue, Oct 18, 2011 at 9:26 AM, John Locke m...@freelock.com wrote: Hi, Sounds like a good direction to me. Couple thoughts -- not being that active in Perl, don't know if these are realistic or not, but just what comes to mind, feel free to disregard. - Are there really no other date or number libraries available that do what we need, to the point we need to create our own? (or are these primarily wrappers around something else?) PGDate is primarily a wrapper around DateTime. I have some temporary parsing logic in there now, but would really like to outsource this somewhere. Just haven't decided where. PGNumeric is likely to be a light-weight wrapper too, but not sure of what yet. - Ran across this project recently: http://ledger-cli.org/ - a Perl cli app that does double-entry accounting in text files. I wonder if they've worked out some good accounting logic we can use, at a minimum -- or perhaps incorporate into LSMB? I'm thinking with a little collaboration, ledger-cli might be extended to use LSMB as back end storage, while LSMB might potentially leverage the work already done in ledger? Have not looked at the code, have no idea whether this is practical. But an easier CLI interface sounds very useful, if it's not too much of a stretch. And might be a good gateway to new users... I will look into it. - Regarding extensions, I just had to do some upgrades to some other web apps where customizations and extension code was scattered across a bunch of libraries. Basically an MVC organization -- but all extension code had to be split up, with the M in the M, the V in the V, and the C in the C directories I just want to express my extreme distaste for that file organization, from an ongoing maintenance point of view. Using a framework for development is great, as long as we can set it up in a way so that if I want to add some sort of custom module to the site, I can put all the code for it in one directory. that's good feedback. Extending that last point a bit -- might it be possible to re-organize the core modules this way, to provide good examples? Instead of having a stub in the root, a file in scripts, the class in LedgerSMB, etc -- might it be possible to have a modules directory with each module in it's own subdirectory? modules/AR, modules/AP, modules/HR, etc. And then inside those, if necessary, mirror the rest of the file structure for MVC -- modules/AR/scripts, modules/AR/LedgerSMB, modules/AR/UI, etc? Oh, one more proposal: I have already done this and made it work but could undo it if necessary. Moving all the scripts/*.pl files to LedgerSMB/Scripts/*.pm. Two justifications here: 1) It makes inheritance across work for workflow scripts, which means easier and more robust customization 2) It means more intuitive reading of the lsmb-handler.pl (and whatever we do for new code) 3) It occurs to me right now, it makes it easier to address your concern above. And to take it one step further, what Drupal has done brilliantly is provided an override mechanism -- if you want to override a core module, you can simply copy it to a place that gets checked first, then hack away, leaving the original code intact. Then an upgrade won't clobber your customizations, and you can much more easily do a simple diff to see what's changed. LSMB doesn't need to have the multiple paths that Drupal uses (I don't think) but maybe something as simple as a path that's checked first for a module - local/ comes to mind. You wanna hack AR? Copy modules/AR to local/AR and go to it. First, a full scale file reorganization of this sort is probably a long project, one we should aim for around 2.0. I'd like to suggest a couple points about this at present: 1) With the move of workflow scripts to LedgerSMB/Scripts, there is a lot we can do with paths. We'd still need some sort of hook in the directory somewhere (probably in LedgerSMB/Scripts), but the rest could reside in an add-ons directory (the sort of file breakout you describe is exactly how we store our add-ons in SVN) in a configured location, and the add-on could grab this configuration option and use it for both PERLLIB environment variable setting, template location, etc. 2) The menu structure actually poses an issue here. Somehow we have to be able to identify nodes for add-on/role purposes, so that roles can be re-applied if necessary. Maybe we should add some sort of a unique specified identifier to the menu_node table? Best Wishes, Chris Travers -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ Ledger-smb-devel
[Ledger-smb-devel] Issue found with i18n for 1.3
Hi all;
One thing new in LedgerSMB 1.3 is the use of parameterized
localization strings. These are designed to accommodate different
word orders in different languages.
This works generally well when the issue is not abused. However,
following up with on a discussion with fellow-committer Herman
Vanderdeels, I went through the LedgerSMB.pot and found that in fact
the feature was abused, and we have some issues here.
The following is acceptable use of this feature:
$locale-text('The item [_1] could not be saved, $form-{item_name})
[_1] refers to some sort of string or number which would be
untranslated but could be interpolated in different places in
different translated strings.
The following is an abuse of this feature:
my $op;
if ($form-{id}){
$op = 'Edit';
} else {
$ob = 'Add';
}
$locale-text('[_1] SIC', $op);
We shouldn't be interpolating constnt strings in other constant
strings. Names and numbers can and should be interpolated, but not
this. Given the volume of commits on this subject, the code in
question is probably mine.
I wanted to write this email to let translators know that I will be
addressing this issue over the next several minor releases, and that
any help from translators is appreciated.
Best Wishes,
Chris Travers
--
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct
___
Ledger-smb-devel mailing list
Ledger-smb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ledger-smb-devel
Re: [Ledger-smb-devel] Proposed framework changes for 1.4
On Tue, Oct 18, 2011 at 9:26 AM, John Locke m...@freelock.com wrote: Hi, Sounds like a good direction to me. Couple thoughts -- not being that active in Perl, don't know if these are realistic or not, but just what comes to mind, feel free to disregard. - Are there really no other date or number libraries available that do what we need, to the point we need to create our own? (or are these primarily wrappers around something else?) Just following up on this a little more. My current plan for PGNumeric is to try to write a wrapper that uses Math::BigFloat for storage and locale for formatting. The idea is that each class basically includes 4 methods: $object = Class-from_input($string); # human entered $string = $object-to_output(); # Localized $object = Class-from_dbstring($string); #Standard db format $string = $object-to_dbstring(); # Standard db format The underlying storage object would be available via an accessor for calculation purposes. Best Wishes, Chris Travers -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ Ledger-smb-devel mailing list Ledger-smb-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ledger-smb-devel
