Re: [GNC-dev] Building Mac version using SDK 12

[Mike Alexander]

On 15 Sep 2022, at 13:54, john wrote:

I don't understand the error but it seems to be an SDK error rather 
than anything we're doing. Might the compiler have gotten out of sync 
with the SDK?


Yes, it was a compiler version skew.  It seems that Apple's frameworks 
won't compile with some versions of Clang++.  I was using a version that 
is slightly newer than Apple's version and switching to Apple's version 
fixed the problem.  I figured it couldn't be a GnuCash problem, but 
didn't see the obvious answer last night.  Thanks for pointing me in the 
right direction.


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

[GNC-dev] Building Mac version using SDK 12

[Mike Alexander]

When I try to build either current maint or master using MacOSX12.1.sdk 
or MacOSX12.3.sdk for SDKROOT I get compile errors in 
gnc-filepath-utils.cpp.  The first error is


In file included from 
/System/Volumes/Data/tools/gnucash-maint/gnucash/libgnucash/core-utils/gnc-filepath-utils.cpp:64:
In file included from 
/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/Foundation.framework/Headers/Foundation.h:12:
/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSBundle.h:91:143: 
error: function does not return NSString
- (NSAttributedString *)localizedAttributedStringForKey:(NSString *)key 
value:(nullable NSString *)value table:(nullable NSString *)tableName 
NS_FORMAT_ARGUMENT(1) NS_REFINED_FOR_SWIFT API_AVAILABLE(macos(12.0), 
ios(15.0), watchos(8.0), tvos(15.0));
/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:103:48: 
note: expanded from macro 'NS_FORMAT_ARGUMENT'

#define NS_FORMAT_ARGUMENT(A) __attribute__ ((format_arg(A)))

This is probably because there is something wrong with my build 
environment, but I was wondering if anyone else had seen this?


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Removing "unused" code

[Mike Alexander]

On 12 Jun 2021, at 7:57, Christopher Lam wrote:

This particular commit aimed to simplify and reduce old code. It can 
be reverted.


 Would you consider submitting them into the repository?


Whether or not to delete code that may be unused is always a judgment 
call.  I think we are sometimes too aggressive about this.  Particularly 
for things that might be used by custom reports.  Unless the code is 
potentially causing problems and there is a clear alternative to it, I 
think it should stay.  The hour or two I spent sorting this out negates 
much of whatever time was saved by removing this and I would have had to 
do that sooner or later.


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Removing "unused" code

[Mike Alexander]

On 12 Jun 2021, at 8:59, D. wrote:


I thought the eguile report was included in the distribution already.


There is at least one EGuile report in the distribution (a balance sheet 
report, I think) but I don't think this one is included..


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

[GNC-dev] Removing "unused" code

[Mike Alexander]

I would like to request that we avoid removing code that is thought to 
be unused, but which may in fact be used, just for the sake of cleaning 
things up.  I use a couple of reports that are not part of the standard 
GnuCash distribution and every now and then they stop working because 
something they depend on is gone.  Since some of these are reports I 
didn't write it's often a nuisance to fix them.


In particular I like the EGuile budget report written by Benoit 
Blancard, et al.  It stopped working after commit 4e38b68 which removed 
a number of parameters from gnc:html-acct-table including account-name 
which this report uses.  Assuming something is unused everywhere just 
because no built-in report uses it is not a safe assumption.


Rather than try to figure out what to use instead and how to fix it I 
just reverted that change, but this may cause me trouble later if 
something else is done that depends on this change.


I understand that cleaning up code and removing dead code are good 
things to do, but I think this is going too far.


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Auto reconcile from register window

[Mike Alexander]

Yes, but, Autoclear doesn't need an account any more than reconcile 
needs one.  Both gnc_plugin_page_register_cmd_reconcile and 
gnc_plugin_page_register_cmd_autoclear call 
gnc_plugin_page_register_get_account to get the account to work on.  
This may return a null pointer if the register doesn't have a clearly 
defined account (e.g. search results), but it works as well for one 
caller as for the other.  Reconcile calls recnWindow to do the real work 
and it checks for a null account and returns immediately if given one.  
The corresponding autoclear method is autoClearWindow and it doesn't 
check for a null account pointer.


It seems to me that if we fix autoClearWindow to check for a null 
account and hook up the menu item in the register window things should 
work fine.  I can do that unless someone knows why it's a bad idea.


Mike

On 26 Oct 2020, at 2:49, Christopher Lam wrote:

Having said that, when we type into the blank entry it knows which 
account to tie the split to... So, the account could be found 
somewhere...


On Mon, 26 Oct 2020, 2:38 pm Christopher Lam, 
 wrote:
From my understanding, this code is not hooked up to the register 
because: a register is always a search list, and not necessarily tied 
to an account. Proof: the blank transaction register entry has no 
account.


I gather the original coder noticed the same, therefore didn't hook 
it.

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

[GNC-dev] Auto reconcile from register window

[Mike Alexander]

I noticed the recent checkins related to the auto reconcile feature.  
This intrigued me since I didn't know such a feature existed.  After 
looking around for a while it appears that this is because it is only 
available for the account tree and I always start a reconcile from the 
register window.  Code exists in gnc-plugin-page-register.c to implement 
it, but there isn't anything in gnc-plugin-page-register-ui.xml to 
create the menu entry to invoke it.  I presume this is an oversight, or 
is there some reason this shouldn't be hooked up?  Or am I missing 
something obvious?  The same thing applies to 
gnc-plugin-page-register2-ui.xml.


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Crash during OFX import

[Mike Alexander]

On 30 Jun 2020, at 11:39, jean laroche wrote:


I believe you're right. The bug was mine.


But you had a difficult merge combining your changes with mine.  Back 
when I was gainfully employed handling merges was one of my jobs and I 
know that it can be difficult.


 Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Crash during OFX import

[Mike Alexander]

On 29 Jun 2020, at 12:58, John Ralls wrote:

It's the right way. We're just in an unusual phase right now where 
master is the current stable.


It's only on master because those struct _ofx_info members were 
introduced by 
https://github.com/Gnucash/gnucash/commit/0a4347bd5ef155193a1d5b18f9ed8596283e488d 
that was applied only to master.


But I think the bug is from the multi file import change (55d7385) which 
failed to reinitialize those fields for each file.  At any rate the fix 
looks ok.


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] gnucash master: Fix binreloc compile error with gcc.

[Mike Alexander]

On 27 May 2020, at 12:18, John Ralls wrote:

It had nothing to do with mach-o/dyld.h: The build failures were on 
the TravisCI Linux dockers.


Why did I not get an EMail from the Travis failure?  On other projects I 
sometimes work on a Travis failure (or even a Travis success) produces 
an EMail to the author of the commit.  Do I need to do something to sign 
up for this?


 Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] gnucash master: Fix binreloc compile error with gcc.

[Mike Alexander]

On 25 May 2020, at 11:01, John Ralls wrote:

Updated	 via  https://github.com/Gnucash/gnucash/commit/fde6be6e 
(commit)

from  https://github.com/Gnucash/gnucash/commit/159ceb64 (commit)



commit fde6be6e018151c7884fd8754470c13924bbb683
Author: John Ralls 
Date:   Mon May 25 08:01:47 2020 -0700

Fix binreloc compile error with gcc.


Sorry about that, I don't use gcc much anymore.  I gather that including 
mach-o/dyld.h requires including stdint first in gcc, but not clang.  
Thanks for fixing it.


 Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Resolving symlinks in _br_find_exe in binreloc.c

[Mike Alexander]

On 22 May 2020, at 13:32, John Ralls wrote:


Even if you say
  GNC_UNINSTALLED=1 GNC_BUILDDIR=`pwd` bin/gnucash
?

Not that I'm suggesting you shouldn't fix binreloc, I just wonder if 
the uninstalled bypass is broken.


GNC_BUILDDIR works, but it has some other side effects which may not be 
desirable (like bypassing all preferences and using defaults for 
everything).


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Resolving symlinks in _br_find_exe in binreloc.c

[Mike Alexander]

On 21 May 2020, at 14:41, John Ralls wrote:

The code that we have is gnomified from the original autopackage.org 
version. I agree it's pretty awful; the comparison of sizeof(ptr) to 
SSIZE_MAX particularly so; trying to allocate SSIZE_MAX-1 in the 
impossible case that sizeof(ptr) > SSIZE_MAX borders on comical. I 
suggest that rather than allocating the path buffers you just create 
two char[PATH_MAX +1] buffers on the stack.


Yes, I noticed those glaring errors.

There's another problem though, /proc/self/exe is Linux-only so it 
still won't work for the BSDs. They don't all have proc (OpenBSD 
doesn't, it's optional on FreeBSD; NetBSD has procfs and uses 
/proc/curproc/exe; DragonFly BSD uses /proc/curproc/file [1]). The 
most general solution is to store argv[0] in inner_main and then look 
for that file in $PATH. I dunno if it's that prominent a use case. 
There have been only 2 bugs about binreloc ever and neither of them is 
that it doesn't work.


If /proc/self/exe doesn't lead anywhere useful it falls back to reading 
from /proc/self/maps.  Is that any better?  Or is it even worth doing?  
It sounds like it isn't likely to succeed.  Too bad there isn't a 
general way to deal with this.


I started down this rabbit hole because running the Mac build of GnuCash 
from the build directory fails without this fix.  I'll try to make it 
work at least in MacOS.


While looking for a general way to do this I found this StackOverflow 
thread:


https://stackoverflow.com/questions/1528298/get-path-of-executable/1528493

This is the most complete discussion of this problem that I found.  It 
contains a link to 
https://svn.sullivanandkey.com/SnKOpen/cpp/executable_path/trunk/ which 
seems to be a remarkably complete solution to this problem.  It is 
probably overkill for GnuCash, but it is interesting.  I'll probably 
just make the existing code work right and leave it at that.  Is that ok 
with you, or should I try to use this more general solution?  Since 
GnuCash already uses Boost and CMake it might not be too hard to do.


 Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

[GNC-dev] Resolving symlinks in _br_find_exe in binreloc.c

[Mike Alexander]

I've had some fixes in binreloc.c for some time that I'm trying to clean 
up and commit.  Part of the problem is that _br_find_exe doesn't work at 
all for non-Aqua Mac builds (i.e. using X11).  I've fixed this by 
calling _NSGetExecutablePath if GNC_PLATFORM_OSX is set and 
MAC_INTEGRATION is not set.  This has been working for me for several 
years.  It shouldn't affect any other platforms.


The other problem is that it doesn't resolve symlinks properly.  There 
are bugs in that code that mean it never works, even in the simple case. 
 I can make simple cases (where the last path component is a symlink) 
work easily enough, but the general case is a bit more complicated if 
you limit yourself to using only readlink.  Is there any reason that we 
don't use the realpath function for this?  I think it exists in all 
non-Windows systems GnuCash cares about (Windows is a completely 
separate implementation of _br_find_exe).  It is in MacOS since 10.4, in 
Posix since POSIX.1-2001, in BSD4.4, and in Linux.  It's essentially 
trivial to call and will solve the problem.


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] XCode build not building gnc-vcs-info

[Mike Alexander]

On 6 May 2020, at 13:27, John Ralls wrote:

There should be a target for gnc-vcs-info. Maybe it's not getting into 
the build-all target, or isn't in the right place?


There is.  I discovered that everything works ok if you build using 
xcodebuild from the command line.  In that case it makes gnc-vcs-info.h 
from gnc-vcs-info.h.in as it should.  However if you try to build using 
XCode itself it somehow skips that step and later complains that 
gnc-vcs-info.h doesn't exist.  This feels like a bug in either cmake or 
XCode.  I don't think it is worth pursuing since building using 
scodebuild works.  I was using XCode since it seems faster.  Since the 
notes on using SCode say to use xcodebuild I don't think there is 
anything more that needs to be done.  It might be possible to work 
around the problem, but probably not worth the trouble.


   Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] XCode build not building gnc-vcs-info

[Mike Alexander]

Thanks, but the dependencies were already installed, and worked fine 
when I hand-built gnc-vcs-info.h so that's not the problem.  I'm not 
using, and have never used jhbuild so it's not involved.  I'm building 
using cmake and ninja.  The result isn't packaged into an application 
bundle, but it is a perfectly good Quartz application that runs fine.  
I'm not asking for help since I got GnuCash built and debugged it using 
XCode.  I was just reporting the problem.  I probably should have used 
BugZilla, but I didn't think that was necessary since this has no effect 
on anything users see.  It's just a problem for developers.


  Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

[GNC-dev] XCode build not building gnc-vcs-info

[Mike Alexander]

I tried the XCode project on the master branch this afternoon and the 
build failed because there was no rule to build 
libgnucash/core-utils/gnc-vcs-info.h.  When I copied it from another 
recent build directory everything worked fine so this seems to be the 
only problem with using XCode with master.


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] PDF Reports

[Mike Alexander]

On 25 Mar 2020, at 11:47, John Ralls wrote:


OK. That's in MacPorts, right?


Yes.  It's the standard version of the webkit2-gtk port, version 2.26.2, 
I have no local mods.  I didn't use their prebuilt binaries since I want 
the Aqua variant.  It was built with the toolchain from XCode Version 
11.3.1 (11C504) on Mojave.


 Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] PDF Reports

[Mike Alexander]

FWIW I use WebKit2 (currently version 2.26.2) with the Quartz version of 
GnuCash and reports with charts seem to work fine.  Even the tooltips 
work.  You might want to give it a try again in case they've fixed 
something.


Mike

On 18 Mar 2020, at 12:14, John Ralls wrote:

To complete that comment, pagination is fixed on webkit2, but that 
works only on Linux. The WebKitGtk maintainers removed support for 
Windows as part of the WebKit 1->2 transition and on MacOS WebKit2 
Javascript fails to return anything so charts don't render. The 
simplest workaround is to export the HTML, load that in a native 
browser, and print or generate a PDF from there.

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] A little hint for xcode usage (scheme)

[Mike Alexander]

I used to use XEmacs much like John describes, although I did use the 
debugging mode in XEmacs.  That was long enough ago that lldb wasn't 
around, at least not for what I was working on.  Now I use XCode for 
debugging and BBEdit for editing.  They work together pretty well and 
XCode's debugging GUI is quite nice.  I remember debugging using address 
stops and console switches, so this is a big step up.


   Mike

On 8 Mar 2020, at 23:27, John Ralls wrote:


From the command line, lldb bin/gnucash

br se -n foo
or you can use the gdb compatibility version, b foo.

emacs gud mode apparently doesn't support lldb and I've never bothered 
to try anyway. I do sometimes wish for compiling in emacs when I'm 
working through 400 lines of template errors, but so far not enough to 
figure out how to import all of the necessary environment to make it 
work. Immediate productivity always seems more important than taking a 
week to learn a new tool.


I'm occasionally tempted to consider switching to bbedit or vscode, 
but after 35 years of emacs it would be a lot of learning time that 
could be spent on more useful things.

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] A little hint for xcode usage (scheme)

[Mike Alexander]

On 8 Mar 2020, at 19:41, John Ralls wrote:

For my part I seldom use Xcode at all. I use emacs for editing and do 
the builds in a terminal window (I use iTerm2 rather that Terminal).


That works too.  Back when I developed code for Windows and Unix as well 
as Mac I used XEmacs for everything, including my shell environment.  
Now that I'm basically Mac only, and XEmacs is dead, I use BBEdit and 
XCode.  The BBEdit worksheet is unusual (much like the old MPW worksheet 
if you're old enough to remember that) but I like it.  BBEdit is a bit 
like [X]Emacs: it's an environment, not just an editor.


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] A little hint for xcode usage (scheme)

[Mike Alexander]

On 8 Mar 2020, at 17:59, jeanl wrote:

The xcode project has tons of scheme. My question is: which scheme do 
I use
to be able to build gnucash, place a break point in any given file and 
start

debugging?


The short answer is that you want the ALL_BUILD scheme.

For more info look for the section "Using Xcode on OS X" in 
gnucash/cmake/README_CMAKE.txt in the source tree.  I've used XCode with 
GnuCash quite a lot and it works fine.  I don't, however, use jhbuild to 
build it.  Instead I install all the dependencies using MacPorts and 
then run cmake and either XCode or scodebuilt to build it.  This may be 
relevant since the result is a traditional out-of-source build tree not 
a MacOS package.  This may or may not matter.


It's much faster, by the way, to let XCode do the build than using 
xcodebuild.  The latter seems to single-thread the build while XCode 
does parallel builds.  At least that's what it seems like, I haven't 
really verified that.


 Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] gnc:debug changes

[Mike Alexander]

On 7 Mar 2020, at 5:22, Christopher Lam wrote:

Ok I think your changes are fine. I don't understand qof_log_check I'd 
hope it's not slow.


I just pushed the change.

qof_log_check is called a *lot* and I don't think it's a problem.  It 
does a couple of hash table lookups and a couple of string comparisons, 
but that's it.  The hash table will generally only have half a dozen 
entries so the lookup should be fast.


 Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] gnc:debug changes

[Mike Alexander]

On 7 Mar 2020, at 0:39, Christopher Lam wrote:

I'd thought that the log level would be an invariant per session. If 
log levels could be amended mid-session I'd like to see how.


Call qof-log-set-level.  It works fine without the most recent change to 
gnc:debug.  I put a few calls to it into 
gnc:price-quotes-install-sources to test this.  I made sure to compile 
the Scheme code in case compilation changed things.


The gnc:debug was optimized because there was (gnc:debug pricelist) 
whereby a long pricelist (which is realistic) would obligatorily lead 
to string-join and map in strify, and was definitely a  _major_ 
 slowdown. Conclusion: we don't want to call strify with a long list 
unnecessarily.


I understand.  Avoiding the call to strify is a good idea and I don't 
intend to change that.  I was simply pointing out that evaluating the 
args to gnc:debug may also take some time.  In the case you mention 
presumably pricelist already exists, but sometimes the debug output has 
to be created by calling other functions which can take time.  I don't 
think it's possible to avoid that by changing gnc:debug, at least not 
easily.


If log level is not an invariant, assuming qof_log_check is fast, 
would be:


(define (gnc:debug items)
  (when (qof-log-check "gnc" QOF-LOG-DEBUG)
    (gnc-scm-log-debug (strify items


Yes, that's what I have in mind except change the first arg to 
qof-log-check to "gnc.scm" which is the original bug that I was looking 
for.


   Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

[GNC-dev] gnc:debug changes

[Mike Alexander]

I spent some time yesterday figuring out why gnc:debug never produced 
any output regardless of the gnc.scm log level.  I tracked it down to 
commits 42b6fb9 and b3a4cd6 from last July.  The actual bug is trivial 
(they test for the log level for "gnc" instead of "gnc.scm") but I 
wonder if b3a4cd6 is a good idea.  It has the effect of binding the 
gnc.scm log level on the first call to gnc:debug and ignoring subsequent 
changes to it.  This means that later calls to (qof-log-set-level 
"gnc.scm" ...) will have no effect.  It is sometimes useful to turn on 
and off Scheme debugging around sections of code you care about.


The optimization from b3a4cd6 seems to be relatively minor.  The 
arguments to gnc:debug are evaluated when debugging is off even with 
this change (something I verified) and the call to strify is avoided 
even without it.  All you're really avoiding is the actual call to 
gnc:debug and the call to qof-log-check.  I propose reverting back to 
42b6fb9 with the bugs in it fixed.  Is this ok? If so I'll push a 
change.


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Robust error handling and respecting the Apha Vantage API limits in gnc-fq-helper.

[Mike Alexander]

On 11 Feb 2020, at 8:14, Edward d'Auvergne wrote:


Although most runs pass, I have noticed that some do not [4].  Have
you seen that behaviour or know what might be happening?


Sometimes AlphaVantage refuses to return a quote.  Right now

 
https://www.alphavantage.co/query?function=CURRENCY_EXCHANGE_RATE_currency=USD_currency=XAU=xxx

returns

{}

The ILS quote works now, but must not have been working when you tried 
it.  I don't think this is an F::Q problem.  GnuCash deals with this 
failure and uses the inverse quote.  For example


mta@bayswater /<4>bin>./gnc-fq-dump currency XAU USD
1 XAU = 1565.2 USD
mta@bayswater /<4>bin>./gnc-fq-dump currency USD XAU
1 XAU = 1565.2 USD

Note that they both return the same thing since the other quote doesn't 
work.  GnuCash is smart enough to use an inverset currency rate if the 
one it wants isn't stored.


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Robust error handling and respecting the Apha Vantage API limits in gnc-fq-helper.

[Mike Alexander]

On 11 Feb 2020, at 3:59, Edward d'Auvergne wrote:


F::Q uses this URL for all queries:

https://www.alphavantage.co/query?function=GLOBAL_QUOTE=json=IDRUSD=$MY_KEY

The result I see is:

{
"Global Quote": {
"01. symbol": "IDRUSD",
"02. open": "0.0001",
"03. high": "0.0001",
"04. low": "0.0001",
"05. price": "0.0001",
"06. volume": "0",
"07. latest trading day": "2020-02-10",
"08. previous close": "0.0001",
"09. change": "0.",
"10. change percent": "0.%"
}
}


It doesn't use that URL for currency exchange rates.  It uses

https://www.alphavantage.co/query?function=CURRENCY_EXCHANGE_RATE_currency=IDR_currency=USD=xxx

which returns

{
"Realtime Currency Exchange Rate": {
"1. From_Currency Code": "IDR",
"2. From_Currency Name": "Indonesian Rupiah",
"3. To_Currency Code": "USD",
"4. To_Currency Name": "United States Dollar",
"5. Exchange Rate": "0.7297",
"6. Last Refreshed": "2020-02-12 06:30:18",
"7. Time Zone": "UTC",
"8. Bid Price": "-",
"9. Ask Price": "-"
}
}


It looks like this code then kicks in, from the "currency" subroutine
in lib/Finance/Quote.pm:

"""
if ( $exchange_rate < 0.001 ) {
# exchange_rate is too little. we'll get more accuracy by using
# the inverse rate and inverse it
my $inverse_rate = $this->currency( $to, $from );
{
local $^W = 0;
return undef unless ( $exchange_rate + 0 );
}
$exchange_rate = int( 1 / $inverse_rate + .5 ) / 
1;

}
"""



That code should be removed.  It actually gets the same accuracy and it 
gets a different answer.  The reverse quote is 13683.5.  This inverts to 
a rate of 0.7308 which is not the same as 0.7297.  That's a 
trivial difference, but F::Q should really return the quote it was asked 
for if possible.


I noticed that there was some action on my pull request after you 
rattled their cage.  Thanks for doing that.  That may make this 
discussion largely irrelevant.


   Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Robust error handling and respecting the Apha Vantage API limits in gnc-fq-helper.

[Mike Alexander]

On 10 Feb 2020, at 6:27, Edward d'Auvergne wrote:


I realise that F::Q is broken in some places.  As I said, I reported
an issue.  However, as you have seen yourself, the F::Q developers are
simply non-responsive.  I believe that asking all GnuCash users to
manually patch their own Finance::Quote perl modules to get around
this 2 year old issue is too much to ask of users.  My reasoning is
that a little more logic on the GnuCash side to handle F::Q
misbehaving is easier for all GnuCash users.  And, from the behaviour
of the scripts below, I now wonder if GnuCash could benefit by
switching from piping '(currency "USD" "EUR")' into gnc-fc-helper to
instead piping in '(alphavantage "USDEUR")'?  The only disadvantage is
that gold and silver quotes do not work via the fetch() interface with
Alpha Vantage.


I think I understand now why we've been talking past each other.  There 
isn't any API in F::Q to fetch multiple currency quotes in one call.  
Instead it's a feature of AlphaVantage to treat exchange rates as if 
they were stock quotes and use the stock quote API to fetch them.  I 
wasn't aware that this existed.  I played with it for a few minutes and 
see a couple of problems with it.  Some currencies are not available 
using the stock quote API but are available using the currency API.  For 
example three in my list of currencies which don't work with the quote 
API are MNT, SBD, and VUV.  These all work fine with the currency API.


The other problem is that for very small exchange rates the stock quote 
interface loses precision.  For example, compare


./gnc-fq-dump alphavantage IDRUSD
Finance::Quote fields Gnucash uses:
symbol: IDRUSD   <=== required
  date: 02/10/2020   <=== recommended
  currency: USD  <=== required
  last: 0.0001   <=\
   nav:  <=== one of these
 price:  <=/
  timezone:  <=== optional

and

./gnc-fq-dump currency IDR USD
1 IDR = 7.295e-05 USD

These problems would make the quote API for currencies less useful.

By the way, I didn't mean to suggest that we should encourage all users 
of GnuCash to patch F::Q.  I still remain hopeful that the maintainers 
will merge my pull request that fixes this bug.  In the meantime 
switching to the quote API might be a good idea even if it does have 
issues.  The code would have to be smart enough to work around the 
problems.  For example if the quote API fails for a given currency it 
could try the currency API.  If the returned value appears to have lost 
all precision then it could try the reverse quote and compute the 
reciprocal itself.


Another possibility would be to abandon AlphaVantage entirely for 
currency exchange rates.  For example https://currencylayer.com/ looks 
promising.  They allow up to 250 queries per month for free and it seems 
very fast.  Of course this would require changes to F::Q which gets us 
right back into those problems.


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Robust error handling and respecting the Apha Vantage API limits in gnc-fq-helper.

[Mike Alexander]

On 9 Feb 2020, at 3:49, Edward d'Auvergne wrote:


That sounds like it would work, however I see quite a different
behaviour with the debugging (in my first post) and from the code
(gnucash/price-quotes.scm and libgnucash/quotes/gnc-fq-helper.in).
What I see in both is that for each iteration of the while loop, F::Q
is called once per currency.  If the fetch fails, due to the
Alphavantage API limits, the first result is #.  This seems to
cause the guile 'fdes' file descriptor to close.  Hence all subsequent
currencies are looped through without calling F::Q.

There is no sleeping in this.  F::Q 1.49 does not recognise serial
calls.  F::Q's delay is only invoked when a list of currencies is feed
into it in a single call.  But GnuCash clearly does not do this.

In any case whether or not the pipe is closed, the loop runs at full
Perl execution speed, limited solely by the IO.  F::Q currently does
not insert a sleep period into this loop.  The result is that only the
first few currencies that are within the Alphavantage API limit will
be fetched, the rest will be skipped.  From the debugging output, the
currency skipping is very fast.  This is exactly the behaviour I have
always seen in GnuCash since the Yahoo API shutdown.


You don't seem to be following what I'm saying.

First, the delay when currency quotes exceed the AlphaVantage rate limit 
is in Finance::Quote itself, not in any code distributed as part of 
GnuCash.  You're right that GnuCash asks for currency exchange rates as 
fast as possible, but this is not relevant to what I'm saying.  As of 
F::Q 1.49 the code in F::Q to rate limit currency exchange rate calls is 
broken.  Earlier I gave a patch to fix this.  If you haven't applied 
this patch to F::Q, GnuCash won't be able to retrieve more than a few 
currency exchange rates.  Have you applied this patch?


Second, you keep saying that GnuCash should ask for multiple currency 
quotes per call to F::Q.  I don't know of any way to do this.  I just 
read the F::Q documentation and code again and can't see any method that 
returns more than one currency exchange rate per call.  How do you do 
this in F::Q?  Is there some undocumented method that I am missing?


   Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Robust error handling and respecting the Apha Vantage API limits in gnc-fq-helper.

[Mike Alexander]

On 4 Feb 2020, at 6:21, Edward d'Auvergne wrote:


That's a good point.  In the future, F::Q might also be able to
respect the API fetching limits across multiple calls, and then this
would add an additional delay.  That could be fixed by properly timing
this while loop.  As for sleeping for different times in the loop for
those with an AlphaVantage subscription, how could that be done?  Do
you know how F::Q knows about this?  I can see that you can currently
buy 30, 120, 300, 600, or 1200 API requests per minute.  Should there
be a GnuCash user setting for this, that is then used by the while
loop in this perl script?


I don't think this should be a GNuCash configuration option.  Right now 
AlphaVantage tells the caller if too many requests have been made in too 
short a time, but it doesn't return information about how many are too 
many.  F::Q recognizes this error return and delays the next call.  It 
is pretty stupid about how long to delay.  If AlphaVantage returned more 
information about how fast is too fast it could be smarter about it.  
Unless this is changed in AlphaVantage I don't think thee is much that 
could usefully be done about this.


Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Robust error handling and respecting the Apha Vantage API limits in gnc-fq-helper.

[Mike Alexander]

I'm glad you're looking at this since it does need some work.  However I 
think there are a couple of things you're not aware of.  I, too, have 
quite a ffew currencies in my file (around 30) and by coincidence I was 
running a price fetch in the background when I first saw your message.  
It completed successfully after fetching all currency quotes as usual.


You're right that GnuCash fetches one currency per call, but this is the 
only API for currencies that F::Q provides.  It fetches all the other 
quotes for a given source in one call to F::Q.  The F::Q currency quote 
code has a test for the rate limiting message that AlphaVantage returns 
when currency quote fetches come too fast.  However at some time in the 
past AlphaVantage changed this message.  I updated F::Q to look for the 
new message and sent a pull request upstream.  This pul request has 
never been applied which means that the current F::Q does not recognize 
when AlphaVantage is trying to tell it to slow down.  My copy of F::Q 
has this change which is probably why it works for me and not for you.


The patch is simple, just change it to look for a "Note" field in the 
returned data instead of an "Information" field.


Try this and see if it helps.

Your change to make GnuCash handle F::Q failures better seems like a 
good idea.  Putting the delay on GnuCash's side of the API might also be 
a good idea, but really shouldn't be necessary.  It also would be a 
problem for someone who has actually paid for an AlphaVantage 
subscription and hence can get currency quotes faster.  Some day I hope 
that F::Q will find a currency quote source that works better and faster 
at which time you wouldn't want the delay in GnuCash.


Mike

diff --git a/lib/Finance/Quote.pm b/lib/Finance/Quote.pm
index b66bca9..6703935 100644
--- lib/Finance/Quote.pm
+++ lib/Finance/Quote.pm
@@ -274,9 +274,9 @@ sub currency {
 if ( !$json_data || $json_data->{'Error Message'} ) {
   return undef;
 }
-# print "Failed: " . $json_data->{'Information'} . "\n" if 
(($try_cnt < 5) && ($json_data->{'Information'}));

-sleep (20) if (($try_cnt < 5) && ($json_data->{'Information'}));
-  } while (($try_cnt < 5) && ($json_data->{'Information'}));
+# print "Failed: " . $json_data->{'Note'} . "\n" if (($try_cnt < 5) 
&& ($json_data->{'Note'}));

+sleep (20) if (($try_cnt < 5) && ($json_data->{'Note'}));
+  } while (($try_cnt < 5) && ($json_data->{'Note'}));

   my $exchange_rate = $json_data->{'Realtime Currency Exchange 
Rate'}->{'5. Exchange Rate'};


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Robust error handling and respecting the Apha Vantage API limits in gnc-fq-helper.

[Mike Alexander]

I'm glad you're looking at this since it does need some work.  However I 
think there are a couple of things you're not aware of.  I, too, have 
quite a ffew currencies in my file (around 30) and by coincidence I was 
running a price fetch in the background when I first saw your message.  
It completed successfully after fetching all currency quotes as usual.


You're right that GnuCash fetches one currency per call, but this is the 
only API for currencies that F::Q provides.  It fetches all the other 
quotes for a given source in one call to F::Q.  The F::Q currency quote 
code has a test for the rate limiting message that AlphaVantage returns 
when currency quote fetches come too fast.  However at some time in the 
past AlphaVantage changed this message.  I updated F::Q to look for the 
new message and sent a pull request upstream.  This pul request has 
never been applied which means that the current F::Q does not recognize 
when AlphaVantage is trying to tell it to slow down.  My copy of F::Q 
has this change which is probably why it works for me and not for you.


The patch is simple, just change it to look for a "Note" field in the 
returned data instead of an "Information" field.


Try this and see if it helps.

Your change to make GnuCash handle F::Q failures better seems like a 
good idea.  Putting the delay on GnuCash's side of the API might also be 
a good idea, but really shouldn't be necessary.  It also would be a 
problem for someone who has actually paid for an AlphaVantage 
subscription and hence can get currency quotes faster.  Some day I hope 
that F::Q will find a currency quote source that works better and faster 
at which time you wouldn't want the delay in GnuCash.


Mike

diff --git a/lib/Finance/Quote.pm b/lib/Finance/Quote.pm
index b66bca9..6703935 100644
--- lib/Finance/Quote.pm
+++ lib/Finance/Quote.pm
@@ -274,9 +274,9 @@ sub currency {
 if ( !$json_data || $json_data->{'Error Message'} ) {
   return undef;
 }
-# print "Failed: " . $json_data->{'Information'} . "\n" if 
(($try_cnt < 5) && ($json_data->{'Information'}));

-sleep (20) if (($try_cnt < 5) && ($json_data->{'Information'}));
-  } while (($try_cnt < 5) && ($json_data->{'Information'}));
+# print "Failed: " . $json_data->{'Note'} . "\n" if (($try_cnt < 5) 
&& ($json_data->{'Note'}));

+sleep (20) if (($try_cnt < 5) && ($json_data->{'Note'}));
+  } while (($try_cnt < 5) && ($json_data->{'Note'}));

   my $exchange_rate = $json_data->{'Realtime Currency Exchange 
Rate'}->{'5. Exchange Rate'};


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] gnucash.xcodeproj obsolete ?

[Mike Alexander]

On 16 Jan 2020, at 16:40, Geert Janssens wrote:

With this capability baked into cmake do we still want to keep the old 
xcode
project in our repo ? Can it do something we wouldn't be able to do 
with the
cmake generated xcode project (possibly after some tweaking of our 
current

cmake scripts) ?


It's gone.  I've been meaning to get rid of it for ages but hadn't 
gotten around to it.  Thanks for nudging me.  THe CMake generated XCode 
project is far superior to the one I created which was very limited.  
I've used the CMake one several times and it has always worked well.


 Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Online ID matching for OFX import

[Mike Alexander]

On 5 Jan 2020, at 23:27, John Ralls wrote:


I want you to code review and test the PR when I finish. ;-)

The online_id comes from from the import source, AQBanking in this 
particular case--and we must be more careful because the code in 
question applies to all of the importers except QIF--and the import 
source doesn't generally contain any commodity information, so basing 
the match on that is a non-starter.


Returning the first match is what we do now. I get that that was the 
wrong approach.


As for GnuCash creating a new account when you've got a new TIAA 
security, does that actually work? For that matter, are you able to 
connect to your TIAA master account and import transactions for all of 
the securities you have, all in one go? I thought that GnuCash didn't 
support that, never mind creating a new security account if it fails 
to find a match. Mind, this isn't the same as the matcher letting you 
create a new account when you double-click on a yellow line, this is 
during the matcher setup when it decides what asset/liability account 
the transactions belong to.


Yes, the online ID comes from the import source, but the OFX importer is 
dealing with two accounts for investment transactions: the cash account 
and the security account.  It uses the online ID from the OFX record for 
the cash account.  Then around line 635 in gnc-ofx-import.c it 
constructs an online ID for the security account by concatenating that 
with the security's ID.


It also has a commodity for the security account.  This is converted to 
a GnuCash commodity around line 624 and passed to 
gnc_import_select_account.  This function is documented to use this as 
part of the match (although it actually didn't before I changed it) so I 
think using it as part of the match would be ok.


I am certainly able to import a bunch of security transactions for 
several accounts from an OFX file in one go.  I've been doing this every 
month for several years without much trouble.  I just imported 50 or so 
tonight.  It isn't perfect since the cash side of the buy and sell 
transactions is always the parent TIAA account and I have to manually 
enter transactions to properly account for this cash, but it sure helps 
a lot.  Note that I am not connecting to TIAA using AQBanking.  That 
might work, but I download an OFX file from their web site and import 
that.


I don't know for sure that I can create an account during the import 
since it's been a year or two since I created one and I can't recall if 
I did it from within the import or not.  I think I have done this in the 
past. I do know that selecting an existing account from the import 
dialog works fine and this, too, will store the online ID in the 
account.  I tried to munge one of the OFX files I have to cause it to 
create a new account, but didn't manage to do it properly.


Your solution is no doubt more complete and elegant than mine although 
mine seems to work ok.  I'll be glad to give it a try when you're ready.


  Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Online ID matching for OFX import

[Mike Alexander]

On 5 Jan 2020, at 19:46, John Ralls wrote:

Complexity is not a valid reason to reject a solution to a complex 
problem unless there's a simpler solution that solves it *just as 
well*. Hackish solutions--and I'm just as guilty on that front as I 
wrote the strncmp hack--are far more likely to trip us up again and 
again every time someone else pops up with another use case. Piling up 
hacks creates technical debt: we're already going through the account 
list twice because of an another hack for a libofx peculiarity. We 
need to work out a correct general solution that doesn't pile up more 
technical debt. We have quite enough of that already.


Whether the user can do something about it or not it saves us work on 
a support email or bug report later if we at least log that we 
couldn't match because the online account number has multiple matches.


You're right about the list, the allocations would be expensive and 
pointless since we can't do anything with it. All we need is an 
Account* for the first partial match and either an int to count the 
partials (for the error message) or a gboolean to record that we had 
more than one partial.


We can promote Daniel Ditman's create_online_id to import-utilities 
and add stripping the space there then use it everywhere we're 
creating an online id and avoid that pass as well. I've started on 
this already, I'll have a PR tomorrow sometime.


So what do you want me to do?  It seems to me that a simple 90% solution 
would be for gnc_import_select_account to refuse to return an account 
whose commodity didn't match the one given to it (if any).  The case 
that led me to this idea was where there is an account with a partial 
match in the wrong commodity but no account with a full match.  If you 
do the full match if possible and partial match if not solution it 
becomes hard to get it to create an account for the full match if a 
partial match account already exists.  This will happen in my books if I 
add a new security to my TIAA/CREF accounts.  The parent account will be 
a partial match for the wrong commodity, but there won't be a full match 
and I want it to let me create or select one.


I can either change gnc_import_select_account to never return an account 
that doesn't match the commodity it's given or add a parameter to it 
telling it whether to make this test.  I looked at the calls to it and I 
don't think making this test unconditionally will cause problems.  Many 
of the calls don't pass a commodity so it won't matter anyway.  For that 
matter, if we compare the commodity it might be ok to just do the 
partial match and not look for a full match instead.  The description of 
gnc_import_select_account says that it warns if the found account's 
commodity doesn't match, but this is not correct.  It only uses that 
commodity when creating a new account.


Just punting and returning the first match, partial or full, isn't 
really an option since it will make it difficult to import my OFX files. 
 Worse, when I do select an account it will use the commodity's CUSIP 
as the online ID and this is not unique since the same commodity can 
(and does) exist in multiple parent accounts.  This will lead to random 
matches to the wrong account on future imports.


The bottom line is that I think we should have it return the first 
account with the correct commodity and a partial match to the online ID>


   Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Online ID matching for OFX import

[Mike Alexander]

I agree that running the search twice is probably ok.  However, John 
raises another point which seems relevant.  What if there is more than 
one partial match?  Right now it returns the first one.  It would seem 
better to punt if there is no full match and more than one partial 
match.


  Mike

On 5 Jan 2020, at 15:47, Christian Stimming wrote:

I think two passes is fine. We have a O(n) complexity anyway, and 
O(2n) is not
worse than that. So I think running one pass looking for exact match, 
then (if

none was found) looking for the partial match, is fine overall.

Caching partial matches in an extra list with element creation and 
such IMHO
introduces a whole lot of extra complexity that isn't really needed. 
Just

running the search twice should be fine.

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Online ID matching for OFX import

[Mike Alexander]

Ok, how about changing it to try an exact match first and only try the
partial match if that fails. If that makes sense I'll fix it tonight. One
question I have is whether this will do the right thing if I add a new
security so the parent account exists but the security account doesn't. I
think it will since the accounts commodity will be wrong.

Mike

On Sun, Jan 5, 2020, 10:01 AM John Ralls  wrote:

>
>
> > On Jan 5, 2020, at 12:40 AM, Mike Alexander  wrote:
> >
> > When I tried to do my monthly import of an OFX file containing my
> TIAA/CREF transactions it failed miserably.  I tracked it down to commit
> 7853f5a2 which changed the matching of online IDs for accounts to only
> match an initial substring of the required ID.  My accounts are structured
> with a parent account in USD corresponding to a TIAA/CREF account and a
> sub-account under it for each asset in that account.  The online ID for the
> parent account is something like "TIAA-CREF.ORG C8304GY8 RAM001 101011".
> The online ID for a sub-account storing an asset is created by appending
> the asset ID to the parent accounts online ID giving something like "
> TIAA-CREF.ORG C8304GY8 RAM001 101011878094101".
> gnc_import_select_account does a linear search of all accounts in no
> particular order looking for one where the online ID matches in initial
> substring of the ID it wants.  If it finds the parent account before the
> sub account it returns that and the process fails since it's the wrong
> currency.  I don't understand enough about the problem that commit is
> fixing to know how to fix it without breaking things.
>
> Mike,
>
> Rats.
>
> The problem the commit was trying to fix is that AQB 5.99 for FinTS uses a
> raw SWIFT account code that for many German banks includes extra
> characters--often a currency code--on the end. Some banks also pad the code
> with leading zeros and a subsequent commit attempts to resolve that. This
> is https://bugs.gnucash.org/show_bug.cgi?id=797432.
>
> Regards,
> John Ralls
>
>
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

[GNC-dev] Online ID matching for OFX import

[Mike Alexander]

When I tried to do my monthly import of an OFX file containing my 
TIAA/CREF transactions it failed miserably.  I tracked it down to commit 
7853f5a2 which changed the matching of online IDs for accounts to only 
match an initial substring of the required ID.  My accounts are 
structured with a parent account in USD corresponding to a TIAA/CREF 
account and a sub-account under it for each asset in that account.  The 
online ID for the parent account is something like "TIAA-CREF.ORG 
C8304GY8 RAM001 101011".  The online ID for a sub-account storing an 
asset is created by appending the asset ID to the parent accounts online 
ID giving something like "TIAA-CREF.ORG C8304GY8 RAM001 
101011878094101".  gnc_import_select_account does a linear search of all 
accounts in no particular order looking for one where the online ID 
matches in initial substring of the ID it wants.  If it finds the parent 
account before the sub account it returns that and the process fails 
since it's the wrong currency.  I don't understand enough about the 
problem that commit is fixing to know how to fix it without breaking 
things.


  Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] gnucash maint: remove unnecesary 'home' level in path for US Income Tax book tax information

[Mike Alexander]

> On Jan 26, 2019, at 1:57 PM, Alex Aycinena  wrote:
> 
> Mike - Is your build working now?
> 

Yes, it’s fine now.

 Mike
 
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] gnucash maint: remove unnecesary 'home' level in path for US Income Tax book tax information

[Mike Alexander]

This commit is causing build failures for me.  I get a number of errors of the 
form

/tools/gnucash-git/gnucash/libgnucash/app-utils/gnc-ui-util.c:264:73: 
error: expression which evaluates to zero treated as a null pointer constant of 
type 'const char *' [-Werror,-Wnon-literal-null-conversion]
qof_book_set_string_option(book, OLD_OPTION_TAXUS_NAME, '\0');

I gather others aren’t seeing this.  I’m building with the clang from XCode 
10.1 which identifies itself as 

Apple LLVM version 10.0.0 (clang-1000.11.45.5)
Target: x86_64-apple-darwin17.7.0
Thread model: posix

If no one else is seeing this I’ll try to figure out what is different for me.

 Mike

> On Jan 24, 2019, at 9:54 PM, J.Alex Aycinena  
> wrote:
> 
> Updatedvia  https://github.com/Gnucash/gnucash/commit/fc153643 
> (commit)
>   from  https://github.com/Gnucash/gnucash/commit/3a486727 (commit)
> 
> 
> 
> commit fc1536432619888dc28cd5c0aa58658094a93c45
> Author: Alex Aycinena 
> Date:   Thu Jan 24 18:36:13 2019 -0800
> 
>remove unnecesary 'home' level in path for US Income Tax book tax 
> information
> 
> 
> 
> Summary of changes:



___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Currencies in Price Database UI gone weird in 3.4

[Mike Alexander]

> On Jan 13, 2019, at 7:44 AM, Wm via gnucash-devel  
> wrote:
> 
> I tried to get new exchange rates today for the first time since updating to 
> 3.4.   Tools / Price Database / Currencies was largely unpopulated.  I looked 
> in the file and the prices are there.  I reverted to 3.3 and all looks well 
> again.
> 
> Would someone else that uses one or two currencies other than their home 
> currency have a look if things are looking odd for them too?
> 
> Effect seems the same with both XML and sqlite backends.
> 
> Stock prices seem OK, it is just the currencies that have gone wrong.
> 
> I thought I'd ask here first as I can't see it mentioned on the bug list and 
> it seems to obvious a thing for everyone to have missed so possibly something 
> at my end though I can't think what.

I see the same thing.  I hadn’t noticed since I rarely open the price editor 
dialog and prices are still there and transactions find them when relevant.  If 
there’s not a bug report for this, you should file one.

I also noticed that it seems to be impossible to edit a transaction when the 
price editor is open.  It is permanently in front of the register window and 
seems to steal all keyboard events.

   Mike
 

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: [GNC-dev] Compiling Gnucash on MacOS - not for the faint of heart...

[Mike Alexander]

> On Jul 1, 2018, at 11:23 PM, John Ralls  wrote:
> 
> Last question first, bash and emacs. ISTR Mike Alexander uses vim and emacs.

Sorry for the delay, I’m a bit behind on EMail

My GnuCash environment is probably unique, or at least was until recently.  
I’ve been too busy to update things for the current release so for the last few 
weeks I’ve been using the prebuilt binaries.  I hope to get back to my previous 
setup soon, however.  This means running the X-Window version on MacOS instead 
of the native Quartz version.  GTK3 is better than GTK2, but even with it I 
find the support for Quartz deficient in many ways.  For example there is 
almost no support for any accessibility features such as Voice Over.  Since I’m 
partly blind I depend on this and find apps that don’t suport it difficult to 
use.

I use a combination of MacPorts, BBEdit, and XCode to work on GnuCash.  I use 
MacPorts to install the dependencies.  This works fine if you’re using the 
X-Window version since that’s what MacPorts does for a living.  I’m not sure 
MacPorts has a quartz build for everything GnuCash needs for a dependency but 
it probably would be possible to add any that are missing.  It’s hard to have 
MacPorts build both X-Window and Quartz versions of things on the same machine 
so you really need to pick one or the other.

I use BBEdit to look at and edit the source and to do the builds.  I used to 
use XEmacs (never vi)  a lot back when I worked on Unix and Windows systems, 
but since XEmacs is essentially dead now and BBEdit is a very good replacement 
I use it.  I do the builds in a BBEdit worksheet.  If any of you have been 
around Macs long enough to remember MPW, a BBEdit worksheet is much like an MPW 
worksheet.  It’s an editable text window with a shell attached so you can 
execute shell commands and have the output appear in the window.   This is 
somewhat of an acquired taste, but I like it for some things.

I also have an XCode project (which is in git) but it is not used for building 
(the build script is /usr/bin/true).  It is useful for debugging since XCode 
provides a quite reasonable GUI for lldb.  I’ve got most of the source files in 
the project (although it’s probably not up to date right now) so XCode can find 
them.  I point the binary at the copy I build in BBEdit.  Then I can use XCode 
to debug GnuCash.  This works surprisingly well.

When I just  want to run GnuCash I invoked it from the terminal using a bash 
script I’ve put in ~/bin/gnucash.

This is admittedly an odd setup, but it works for me.

 Mike
 
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Moving .gnucash and friends from $HOME to XDG_DATA_HOME (and impact on other applications like KMyMoney)

[Mike Alexander]

> On Aug 23, 2017, at 1:09 PM, Geert Janssens  
> wrote:
> 
> Linking has its drawbacks as well though. If I link the directory, the files 
> and subdirectories will be essentially the same. However if the user 
> misinterprets the message and removes file by file from $HOME/.gnucash 
> instead 
> of the directory link, the files are still lost.

Hard linking of directories is not allowed by most systems so if you link the 
directories it would have to be a soft link.  This is good anyway since it 
makes it more obvious that the linking has occured (if the user knows how to 
look).  I’m still not sure it’s a good idea for the reasons you mention.

  Mike
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Python bindings and Gtk3

[Mike Alexander]

> On Aug 10, 2017, at 3:22 AM, John Ralls  wrote:
> 
> 3.22.what? I pushed the change to gtk that caused the pango breakage between 
> 3.22.11 and 3.22.12 so 3.22.11 wouldn't have had the problem... and of course 
> if pango is using the fontconfig backend it shouldn't manifest anyway. Of 
> course it's also possible that MacPorts picked up my patch from the bug.

Sorry, it’s 3.22.17.  I didn’t copy and paste enough.

I discovered that switching to the HighContrast theme improved the sizing of 
the tabs so they are similar to the size in Gtk2.  This doesn’t seem like 
something I would expect.

  Mike
 
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Python bindings and Gtk3

[Mike Alexander]

> On Aug 9, 2017, at 1:49 PM, John Ralls  wrote:
> 
> Hmm, looks like both whitespace and font are larger. What version of Gtk+-3?
> 

It’s version 3.22.

> Hmm, looks like both whitespace and font are larger. What version of Gtk+-3?
> 
> You can try fiddling with the tab-label styling in ~/.config/gtk-3.0/gtk.css. 
> There's some guidance in 
> https://developer.gnome.org/gtk3/unstable/theming.html 
> .

I’ll take a look at that.

Mike
 
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Python bindings and Gtk3

[Mike Alexander]

> On Aug 9, 2017, at 12:21 AM, Mike Alexander <m...@umich.edu> wrote:
> 
>> On Aug 8, 2017, at 10:23 AM, John Ralls <jra...@ceridwen.fremont.ca.us> 
>> wrote:
>> 
>> I don't know if the MacPorts X-build uses the CoreText or FontConfig backend 
>> for Pango, but if it uses CoreText it might be your problem.
> 
> I don’t know either.  I see that Pango’s configure looks for (and finds) 
> fontconfig, but then says "checking which cairo font backends could be 
> used... quartz freetype”.  I suspect it is using fontconfig.  I am using a 
> Retina display, but the text in the tabs seems like it’s normal size.  There 
> is a lot of blank space around the text that makes the tab bigger.  In fact 
> the text itself is smaller in the Gtk3 version.  Here are a couple of 
> screenshots.


I guess attachments aren’t allowed, I should have known.  The screenshots are 
at 

https://www.dropbox.com/s/9ge32ydfc95fum8/Gtk2%20tabs.png?dl=0 
<https://www.dropbox.com/s/9ge32ydfc95fum8/Gtk2%20tabs.png?dl=0> (Gtk2)
and
https://www.dropbox.com/s/c650pqyjbyl1zvn/Gtk3%20tabs.png?dl=0 
<https://www.dropbox.com/s/c650pqyjbyl1zvn/Gtk3%20tabs.png?dl=0> (Gtk3)

  Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Python bindings and Gtk3

[Mike Alexander]

> On Aug 8, 2017, at 10:23 AM, John Ralls  wrote:
> 
> I don't know if the MacPorts X-build uses the CoreText or FontConfig backend 
> for Pango, but if it uses CoreText it might be your problem.

I don’t know either.  I see that Pango’s configure looks for (and finds) 
fontconfig, but then says "checking which cairo font backends could be used... 
quartz freetype”.  I suspect it is using fontconfig.  I am using a Retina 
display, but the text in the tabs seems like it’s normal size.  There is a lot 
of blank space around the text that makes the tab bigger.  In fact the text 
itself is smaller in the Gtk3 version.  Here are a couple of screenshots.

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Python bindings and Gtk3

[Mike Alexander]

> On Aug 7, 2017, at 8:51 PM, Sumit Bhardwaj  wrote:
> 
> For the tab issue, can you share a screenshot or something to that effect? I 
> am on master build and have tabs on right-hand side as well, but it's 
> perfectly usable.
> 

It’s certainly usable, I didn’t mean to imply otherwise.  It’s just that I have 
to scroll to see all my tabs where I don’t in the old version.  It’s a minor 
annoyance that should be easily dealt with.  I should probably learn enough 
about Gtk3 to fix it myself, but if I start playing around with that I won’t 
get any of the things I should be doing done.  Also, note that I’m using the X 
Window version, not the Quartz version.  The Quartz version may be quite 
different (I’m not sure if it’s even being built yet).

Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Python bindings and Gtk3

[Mike Alexander]

You may already know this, but the Python bindings done’t work in the current 
master branch (they still use Gtk2).  I made a brief attempt to fix this, but I 
don’t know either Python or Gtk3 well enough.  

I also noticed a cosmetic issue that you may not be aware of because I use a 
non-default window configuration.  I keep the tabs arranged along the right 
hand side of the  window.  In the Gtk3 version these tabs are about twice as 
high as previously.  This means that my tabs don’t fit along the side of the 
window without scrolling.  Before they took up about 2/3 of the window height.  
This is obviously not a fatal problem, but it is annoying.

In general, I’m impressed by how well this works given the magnitude of the 
change.  It’s going to be a big improvement.

  Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: xaccTransSetCurrency and Bug 763146

[Mike Alexander]

> On Mar 16, 2016, at 10:54 PM, John Ralls  wrote:
> 
> In the course of working on and testing bug 763146 [1] the reporter 
> discovered an interesting auto-completion bug: If the transaction that's 
> being copied is multi-currency and was created in the other register (so the 
> transaction currency is for a different currency from the present register's) 
> then attempts to edit the debit/credit amount will produce incorrect results. 
> The problem was made more obvious because the reporter's currency, BYR, 
> trades at around 21000 to the USD, so the incorrectly calculated values and 
> prices were 0 after rounding, causing GnuCash to blank the cells.
> 
> On investigating I found that the most obvious problem was that the 
> calculations were being run in the wrong direction because of the transaction 
> currency to account currency mismatch. Adding a line to 
> gnc_split_register_autocompletion() to change xaccTransSetCurrency() didn't 
> fix the problem, though, because the split values weren't changed by that 
> call. Adding code to recalculate the split values does correct the problem.
> 
> While it seems pretty obvious to me that recalculating the split values after 
> re-setting a transaction's currency is The Right Thing To Do, I'm worried 
> anyway. 
> 
> Can anyone think of use cases where it's not the right thing to do?

Do you have any specific reason you are concerned, or just the general feeling 
that something so obvious can't have been missed for so long?  It seems clear 
that the values must be changed somehow if the currency changes, since they are 
supposed to be expressed in the currency that just changed.  I guess my only 
question is whether that's the right place to do it.

  Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: xaccTransSetCurrency and Bug 763146

[Mike Alexander]

> On Mar 17, 2016, at 6:55 PM, John Ralls  wrote:
> 
> Neither. I'm used to finding bugs like that because the early authors weren't 
> thinking about multi-currency transactions and by the time you started to the 
> codebase had become far too bloated and convoluted to track down all of the 
> problems.
> 
> In fact I even wrote a test that specifically checked that this worked wrong 
> because I didn't recognize it as a bug and was documenting with the tests the 
> existing behavior so that when we got to rewriting the test would show that 
> the behavior had been preserved through the rewrite. That was 4 1/2 years ago.
> 
> What I'm worried about is that there's code in GnuCash that depends on that 
> bug. I'm less worried today because I went through every call of 
> xaccTransSetCurrency and almost all of them operate only on new transactions. 
> The few that don't (cap-gains.c, Scrub.c) should be OK with the change. With 
> that done I've pushed the commit.

That makes sense.  I should have noticed this when I was working in that area, 
but didn't.  I think I've seen things wrong as a result of this and didn't 
spend enough time to track it down.  I saw you had pushed the change so figured 
you had convinced yourself it was ok.  Thanks for fixing it.

  Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Tag for 2.6.9

[Mike Alexander]

--On November 18, 2015 at 6:37:06 AM -0800 John Ralls 
 wrote:



The “missing” tags along with all of the branches that you
don’t use are in .git/packed-refs.

Might be a config issue. You can force retrieval of the tags with
`git fetch -t`.


"git fetch -t" more or less fixed things.  That copy now has 161 tags 
including all the recent ones.  I don't know why it still is missing 48 
others, but I'm not going to worry about it.


   Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Problems building with guile2

[Mike Alexander]

--On November 18, 2015 at 6:38:51 AM -0800 John Ralls 
 wrote:



Have you upgraded to ElCap? If so, see
https://bugzilla.gnome.org/show_bug.cgi?id=757616.



No, I'm still on Yosemite.  This isn't the problem.  Anyway, by 
adjusting the value of DYLD_LIBRARY_PATH I was able to get it to load 
the XML backend.  Since I can't tell why it's trying to load it, and 
since that only fixed one of the three errors, I didn't think that 
change was useful.


 Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Tag for 2.6.9

[Mike Alexander]

--On November 17, 2015 at 9:41:49 AM +0100 Geert Janssens 
<geert.gnuc...@kobaltwit.be> wrote:



On Tuesday 17 November 2015 02:33:08 Mike Alexander wrote:

Is there a reason that there's no 2.6.9 tag in the Git repository?
It seems like it should go on 4241505.

   Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel


It is there on my local repository and on github. I'm surprised you
don't see it ?

Geert


This is odd.  I have two copies of the Git repository on my machine and 
I see it in one and not the other.  In one of them I've used fetch for 
maint and master, but not pull (i.e. maint and master are behind 
origin/maint and origin/master).  This one doesn't see the tag.  On the 
other one I've used pull to keep the local branches up to date.  There 
must be something about Git I don't understand.  Actually, there's a 
lot about it I don't understand well.


Sorry for the confusion, the tag is ok.

   Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Tag for 2.6.9

[Mike Alexander]

--On November 17, 2015 at 8:09:55 PM -0800 John Ralls 
<jra...@ceridwen.us> wrote:





On Nov 17, 2015, at 4:38 PM, Mike Alexander <m...@umich.edu> wrote:

--On November 17, 2015 at 9:41:49 AM +0100 Geert Janssens
<geert.gnuc...@kobaltwit.be> wrote:


On Tuesday 17 November 2015 02:33:08 Mike Alexander wrote:

Is there a reason that there's no 2.6.9 tag in the Git repository?
It seems like it should go on 4241505.

  Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel


It is there on my local repository and on github. I'm surprised you
don't see it ?

Geert


This is odd.  I have two copies of the Git repository on my machine
and I see it in one and not the other.  In one of them I've used
fetch for maint and master, but not pull (i.e. maint and master are
behind origin/maint and origin/master).  This one doesn't see the
tag.  On the other one I've used pull to keep the local branches up
to date.  There must be something about Git I don't understand.
Actually, there's a lot about it I don't understand well.

Sorry for the confusion, the tag is ok.


Huh. That’s strange, retrieving tags is fetch’s job and pull is
just fetch + merge. Do the tags appear if you merge origin/maint into
maint on the “fetched” repo? Use —ff-only to make sure you
don’t get a gratuitous merge commit.


Yes, it is strange.  I tried updating my local maint and master 
branches to be the same as the remote branches and this didn't help. 
That tag still doesn't exist in that copy of the repo.  That, however, 
is just the tip of the iceberg.  This copy has only 9 tags defined. 
The other one, which has 2.6.9, has 209 tags defined.  Somehow one copy 
of the repo lost 200 tags.  Furthermore .git/refs/tags is empty in that 
directory, so I don't know where the 9 that are defined come from.  I 
have no idea what is going on, but it is clearly a local problem.


 Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Problems building with guile2

[Mike Alexander]

I tried to merge maint into master tonight.  Although the merge seemed 
to be fairly easy, the result wouldn't build.  I tried to figure out 
why, but eventually gave up.  The problem exists in maint too, it won't 
build with guile2.   There were a lot of changes to various Scheme 
files recently on maint and something broke building with guile2.


The first error I get is when it tries to compile 
app-utils/business-prefs.scm.  It fails trying to load 
gncmod-backend-dbi and gncmod-backend-xml and then gives up because 
gnc:module-begin-syntax is unbound.  I have no idea why this file 
should be loading those two backends.  I managed to make it load 
gncmod-backend-xml ok, but nothing I did would make it load 
gncmod-backend-dbi.  As far a I can tell no other Scheme file loads 
this.  I also don't understand why gnc:module-begin-syntax is unbound. 
Even adding an explicit load-module call for gnc-module didn't get it 
bound and that module should already be loaded anyway.


The complete build log is below.  Does anyone have any idea what is 
wrong?


   Mike

make: Entering directory `/tools/gnucash-git/build/darwin/src/app-utils'
Making all in .
make[1]: Entering directory 
`/tools/gnucash-git/build/darwin/src/app-utils'

GNC_UNINSTALLED=yes \
GNC_BUILDDIR=../.. \
	GNC_MODULE_PATH="../../src/engine/.libs:${GNC_MODULE_PATH}" 
GUILE_LOAD_PATH="../../src/app-utils:../../src/core-utils:../../src/engine:../../src/gnc-module:../../src/scm:${GUILE_LOAD_PATH}" 
GUILE_LOAD_COMPILED_PATH="../../src/app-utils:../../src/core-utils:../../src/engine:../../src/gnc-module:../../src/scm:${GUILE_LOAD_COMPILED_PATH}" 
LD_LIBRARY_PATH="../../src/libqof/qof/.libs:../../src/core-utils/.libs:../../src/gnc-module/.libs:../../src/engine/.libs:../../src/app-utils/.libs:${LD_LIBRARY_PATH}" 
DYLD_LIBRARY_PATH="../../src/libqof/qof/.libs:../../src/core-utils/.libs:../../src/gnc-module/.libs:../../src/engine/.libs:../../src/app-utils/.libs:${DYLD_LIBRARY_PATH}" 
\

/opt/local/bin/guild compile -o business-prefs.go business-prefs.scm
make[1]: Leaving directory 
`/tools/gnucash-git/build/darwin/src/app-utils'

make: Leaving directory `/tools/gnucash-git/build/darwin/src/app-utils'
* 02:07:58  WARN  failed to load gncmod-backend-dbi from 
../../src/backend/dbi/.libs
* 02:07:58  CRIT  required library gncmod-backend-dbi not 
found.
* 02:07:58  WARN  failed to load gncmod-backend-xml from 
../../src/backend/xml/.libs
* 02:07:58  CRIT  required library gncmod-backend-xml not 
found.

;;; note: source file ../../src/app-utils/gnucash/app-utils.scm
;;;   newer than compiled 
/Users/mta/.cache/guile/ccache/2.0-LE-8-2.0/tools/gnucash-git/gnucash/src/app-utils/app-utils.scm.go

Backtrace:
In ice-9/eval-string.scm:
 44: 19 [read-and-eval # #:lang ...]
 37: 18 [lp (use-modules (gnucash engine))]
In ice-9/eval.scm:
505: 17 [# 
(use-modules #)]

In ice-9/psyntax.scm:
1106: 16 [expand-top-sequence ((use-modules (gnucash engine))) () ...]
989: 15 [scan ((use-modules (gnucash engine))) () ...]
279: 14 [scan ((# #) #(syntax-object *unspecified* # #)) () (()) ...]
In ice-9/boot-9.scm:
3597: 13 [process-use-modules (((gnucash engine)))]
702: 12 [map #(mif-args)> ((#))]
3598: 11 [# 
(#)]

2864: 10 [resolve-interface (gnucash engine) #:select ...]
2789: 9 [##:optional autoload version #:key ensure)> # ...]

3065: 8 [try-module-autoload (gnucash engine) #f]
2401: 7 [save-module-excursion #ice-9/boot-9.scm:3066:17 ()>]

3085: 6 [#]
In unknown file:
  ?: 5 [primitive-load-path "gnucash/engine" ...]
In ice-9/boot-9.scm:
1724: 4 [%start-stack load-stack ...]
1729: 3 [#]
In unknown file:
  ?: 2 [primitive-load-path "engine-utilities"]
In ../../../../gnucash/src/engine/engine-utilities.scm:
 25: 1 [#]
In ice-9/boot-9.scm:
106: 0 [#args)> unbound-variable ...]


ice-9/boot-9.scm:106:20: In procedure #ice-9/boot-9.scm:97:6 (thrown-k . args)>:
ice-9/boot-9.scm:106:20: In procedure module-lookup: Unbound variable: 
gnc:module-begin-syntax

make[1]: *** [business-prefs.go] Error 1
make: *** [all-recursive] Error 1


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: understanding currency exchange...

[Mike Alexander]

--On November 14, 2015 at 12:37:04 PM -0800 John Ralls 
 wrote:



Let’s at least make the non-trading account view look like the
trading account view with each split amount. That will help some with
the confusion.


You mean always show the amount instead of the value?  I thought about 
doing that when I implemented trading accounts but didn't since one of 
my design criteria was to change the behavior of GnuCash when trading 
accounts were turned off as little as possible.  I changed the display 
with trading accounts on since I though it was very confusing 
otherwise.  Changing it to always show the amount instead of value 
would be a good idea, I think.



I don’t think it makes sense to show the value without also showing
the price. I think ideally we’d switch split-view to the layout in
STOCK accounts any time there’s more than one commodity involved in
a transaction in ledger view any time there’s a visible transaction
with multiple commodities).


Having a way to use the stock account layout for any register sounds 
like a great idea.  However doing it based on what's visible may not be 
the best way.  If I understand what you're suggesting then scrolling 
around in a register would make the extra columns appear and disappear 
depending on what is currently visible.  This would be disconcerting.


You could base the decision on whether there is a transaction anywhere 
in the register that uses two or more commodities, but then one 
multi-currency transaction way in the past would change the register 
layout forever, which isn't too good either.


How about just making it an option for the register?  We could an a 
command to the View menu, View->Show Value and Price, (assuming amount 
is always shown).   The default would be based on the current behavior. 
This would seem to give us the most flexible solution.  It would also 
make the exchange rate dialog less important, which wouldn't be a bad 
idea.


 Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Tag for 2.6.9

[Mike Alexander]

Is there a reason that there's no 2.6.9 tag in the Git repository?  It 
seems like it should go on 4241505.


  Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: understanding currency exchange...

[Mike Alexander]

> On Nov 14, 2015, at 10:22 AM, John Ralls <jra...@ceridwen.us> wrote:
> 
> 
>> On Nov 13, 2015, at 8:49 PM, Mike Alexander <m...@umich.edu> wrote:
>> 
>> --On November 11, 2015 at 5:12:38 PM -0800 John Ralls <jra...@ceridwen.us> 
>> wrote:
>> 
>>> Multi-currency split view can be confusing. In split view in a
>>> currency account, the amounts shown in the register are in the open
>>> account’s currency — in the case of your
>>> Assets:Exchanges:Benjamin Brewer account, USD, so you should enter
>>> 500 in both, then set the rate to 9/10 in the transfer dialog, click
>>> OK, and complete the transaction. Switch to the Euro account and the
>>> transaction will show as being for €450.  Analogous views for a
>>> bogus ATM withdrawal: Checking is USD, Cash—Euro is what it says.
>> 
>> This is correct if Trading Accounts are turned off, but not if they are 
>> being used.
> 
> We need to fix that so that it’s consistent. How?

If trading accounts are on then expanded splits show the amount of the split, 
not the value.  They also show the currency symbol for all splits not in the 
transaction's currency (i.e., all splits where it matters which is shown).  
It's been this way since trading accounts were first implemented, and was 
discussed then.  Changing it would be easy, but I think would add confusion.  I 
could easily argue that changing it to always show the amount, with or without 
trading accounts, would be better.  It really should show both.  

The current situation is not ideal.  When I implemented this, I assumed that 
the Register2 code would replace the Register code soon and this would all be 
moot.  That didn't happen and isn't going to happen anytime soon.  Should we 
add a column to the register to show the amount in expanded splits in 
non-priced registers?  That might help people understand what's going on.

Mike
 
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Rounding in the price db.

[Mike Alexander]

 On Aug 17, 2015, at 5:36 AM, John Ralls jra...@ceridwen.us wrote:
 
 
 On Aug 12, 2015, at 2:13 PM, John Ralls jra...@ceridwen.us wrote:
 It might help to consider what the price db is used for: Aside from 
 providing a default rate in the transfer dialog, it’s used for pricing 
 assets in the Accounts page and the summary bar — only the latest entry is 
 used there. It’s used for calculating values in reports, where the reports 
 can use a weighted average, nearest in time, most recent, or average cost. I 
 need to look to see if the average cost actually looks at the buy 
 transactions or if it does something similar to nearest in time and at 
 weighted average to see what it’s averaging and how it’s weighting the 
 averaged values. Depending on what those two are doing might drive whether 
 multiple F::Q entries per day make sense, assuming that the algorithms make 
 sense.
 
 OK, I’ve looked. They don’t use the price db, which is what one would expect. 
 Weighted-average is weighted by the transaction size, which will be distorted 
 by stock splits but is otherwise OK.

The Advanced Portfolio report uses the Price DB to get the current value of a 
holding.  There is an option to use a transaction price instead, but the 
default is to use the Price DB.  The only Price DB options it supports are 
latest and nearest in time.  If you ask it to use a transaction price (or 
if it can't find a price in the Price DB) it uses the most recent transaction 
which defines a price that can be converted to the report's currency.

I can't speak for other reports.  I'm out of town and don't have easy access to 
the source code.

   Mike
 
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Rounding in the price db.

[Mike Alexander]

--On August 13, 2015 at 10:06:09 AM +0100 John Ralls 
jra...@ceridwen.us wrote:



Another related thought: Answering a dialog box every time one enters
a two-currency transaction is going to annoy anyone who has to enter
a bunch of such transactions. I can think of several ways of dealing
with that, in order of my preference: A radio button on the transfer
dialog whose last value is retained, an “always do this” checkbox
on the what to do with the rate dialog, or a preference setting.
Which do you prefer (ignore the problem is another option)?


Actually, I think ignore the problem is a reasonable approach in this 
case.  If we get the prices in the Price DB sorted out properly, then 
the default in the transfer dialog will be correct in most cases which 
means the extra work for the user resulting from showing it is minimal. 
At least I wouldn't mind, and it would give me a chance to verify that 
the price chosen made sense.


When I'm entering foreign currency purchases (e.g., using a CAD or GBP 
credit card) that's pretty much what I do already.  All my expense 
accounts are USD and I update the Price DB daily so I generally just 
accept the default conversion from the transfer dialog in this case. 
It really doesn't slow me down much.  I think the problem comes if the 
default isn't good enough.


Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Rounding in the price db.

[Mike Alexander]

--On August 12, 2015 at 10:23:09 AM +0200 Geert Janssens 
geert.gnuc...@kobaltwit.be wrote:



I'm not really sure about the price entered vs F::Q price. I would
imagine that if you are tracking stock, you would be interested in
the  exact real price you bought/sold it, which is not necessarily
exactly  the price you get from F::Q. Wouldn't that mean that the
price entered  should be preferred over F::Q prices at least in some
cases ?


I was thinking about multiple currency transactions when I said I would 
prefer F::Q prices to transaction prices in the price DB.  For stock, 
etc., transactions your point makes sense.  In that case the price 
implied by a buy or sell is more likely to be a real price that 
should be recorded in the price DB.  On the other hand, my experience 
is that the transaction price is more likely to match the F::Q price 
for stock transactions so it matters less which you use.  Considering 
all this, I guess I still prefer to use F::Q prices over transaction 
prices in the price DB if both exist.


To address another point, the Advanced Portfolio report only uses the 
price DB to calculate the current value of the commodity (and other 
things derived from current value such as unrealized gain).  Basis and 
realized gain are always calculated from actual transactions and their 
implied prices.  I'm not sure about other reports.


   Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Rounding in the price db.

[Mike Alexander]

--On August 11, 2015 at 9:20:43 AM +0100 John Ralls 
jra...@ceridwen.us wrote:



Hmm. That’s doable in master with careful rounding control. I’m
pretty sure it would produce overflows in maint because the old
int128 math is really int128-int64 and it barfs if any intermediate
result won’t fit in int64_t. On the other hand, the current
implementation with no rounding is likely to produce overflows if one
tries to do a large transaction in Dinars-Dobe (or Bitcoin to a lot
of minor currencies, for that matter). ISTM arbitrarily setting a
10^-10 rounding isn’t quite right, though. If we’re going to do
something like that, and see the next para, it would be more correct
to round to the number of digits which make a 1-scu change in the
smaller currency or perhaps one more digit than that. 10 digits might
not be enough for converting STD to BTC with its 10^-6 scu.

The whole issue of restricting denominators to powers of 10, implied
by “significant digits” and “decimal places is also worthy of
discussion. Both terms are meaningless with rational numbers unless
one imposes that constraint. Once one does, though, significant
digits is no more difficult to implement than decimal places: One
just rounds the numerator to x digits after doing the denominator
conversion, adjusting the denominator to account for any reduction of
the digits in the numerator.

None of which does anything to help deal with the small moves in
prices due to rounding in the amount and value to their respective
SCUs getting recorded in the pricedb. Mike’s recommendation to
capture the user input works in the case where the user inputs (or
retrieves from F::Q) a price, but not if she uses the
other-account-amount entry.


This is what I meant.  If she enters two amounts then there's no choice 
but to compute a price.



IIRC I implemented the price db storage
as after-rounding-to-scu so that there’d be only one place in the
code where it was written out. That can be changed easily enough and
might reduce some of the complaints about the presented price jumping
around when entering a bunch of foreign currency transactions. *That*
change could go in maint. I clearly hadn’t sufficiently thought
through rounding when I started this thread. It seems sufficiently
complicated that any implementation should go into master instead.


Yes, thinking of it as a rational number problem instead of a decimal 
number problem probably helps.



We seem to agree that there need be only one price per day in the
price db, but no one’s said anything about what price that should
be. I’m inclined towards any new price replaces the existing one
unless they’re the same. What about source? Should a F::Q price
take precedence over a transfer dialog price or vice-versa?


My first guess is to take the last price entered and prefer F::Q price 
to a calculated price.  I keep daily F::Q prices back for quite a while 
and when I enter historical transactions it's sometimes annoying when 
it adds a new price for that day.  I usually go back and delete the 
transaction price which is often different from the F::Q price due to 
banks adjusting the rate to their advantage.  I really wouldn't want it 
to delete the F::Q price for that day.  Furthermore if I use F::Q to 
get quotes new F::Q prices should replace transaction prices for that 
day, if any.


It's complicated.  I've always said that I hate dealing with numbers 
and I certainly don't pretend to be an expert on this.


Mike



___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Rounding in the price db.

[Mike Alexander]

--On August 10, 2015 at 4:27:40 PM +0100 John Ralls 
jra...@ceridwen.us wrote:



I seem to remember that banks here use 6 significant digits in
exchange  rates, which is not quite the same as 6 places after the
decimal point.  Your USD-Sao Tomean Dobra illustrates this: due to
the extreme value  difference between the two you need 10 digits
after the decimal point,  but only 6 of them are significant (ie not
a leading zero).

Is this something our gnc_numeric code supports and can be used by
the  price db ?


We have “significant digits” rounding code, but IIRC it’s
really six decimal places. If we were to have real significant digits
we’d need to be careful to keep it away from the debits-and-credits
code or it could make a real mess.


Six significant digits would probably do, although I would argue for a 
few more to properly convert large quantities of a commodity.  However 
10 decimal places is easier to implement and is probably just as good. 
That seems to be what the XE conversion tables at 
http://www.xe.com/currencytables/ use. Even that falls down 
converting gold to Sao Tomean Dobra, but it works ok for any real 
currency.  It gives 6 digits of accuracy converting between Kuwaiti 
Dinar and Sao Tomean Dobra which is the biggest ratio I can find.


 Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: gnucash master: Multiple changes pushed

[Mike Alexander]

 On Aug 10, 2015, at 3:05 AM, John Ralls jra...@ceridwen.us wrote:
 
 I pushed the change Alex suggested to maint and that got my other change 
 synced to Github so everything should be OK now.

I pushed changes to maint and master that I think fix things up.  They replace 
my change to gnu-fq-helper.in on master with your version of the same fix and 
put the parts of my change 142844db that relevant to maint on that branch.

  Mike
 
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Rounding in the price db.

[Mike Alexander]

--On August 9, 2015 at 8:35:04 PM +0100 John Ralls jra...@ceridwen.us 
wrote:



A recent IRC conversation and a discussion (face to face!) with a
user got me thinking about the prices entered from the transfer
dialog into the price db. The current situation is that when the user
creates a transaction between accounts with differing currencies and
commodities, the transfer dialog comes up to get either a price or
the value in the “other” currency. The user duly enters a number
and GC does the appropriate calculation, rounding the value to the
max denominator for the “other” account’s currency/commodity,
then recalculating the “true” price without the rounding and
saves the result in the price db. The user could do several entries
with the same nominal exchange rate and get a different exact
fraction for each because of the rounding of the value.

Does that really make sense? Should we round the price we store in
the price db to some reasonable fraction? What fraction? The smallest
commodity unit (scu) of the “value” currency/commodity seems
reasonable to me. For example, if the entry is of  how many US
Dollars buys a Euro or a share of Ford stock, the price should be
rounded to the scu of US Dollars, i.e. cents. OTOH some mutual funds
price in mils ($.0001), so perhaps we should round the price db
entries to


No, that doesn't make sense.  I've been bothered by this for some time.

However, I don't think rounding to the SCU of either commodity is 
correct.  Suppose the SCU of A and B is 2 and the official exchange 
rate from A to B is 1.500555 (my bank quotes rates to 6 decimal places 
when I buy foreign currency).  If I first convert 2.00 A to 3.00 B by 
entering this exchange rate in the transfer dialog it would record an 
exchange rate of 1.50.  Later a value of 2000 A would be converted by 
default to 3000 B, not 3001.11.  I think you need to record more digits 
than the SCU.  Ideally you should record exactly what is entered in the 
transfer dialog if the user enters a price rather than a destination 
value.  If they enter a destination value then you probably should 
record at least 6 places after the decimal point.


Things get tricky if the ratio is very small.  The current rate for USD 
per Sao Tomean Dobra per USD is 0.444615.  if you rounded this to 
0.44, you lose a lot of precision.  If someone enters a transaction 
of 1000 Dobra to USD .04 (unlikely, but valid) then the recorded rate 
to 2 places would be zero which is clearly a bad idea.


I solved this problem in the Advanced Portfolio report by recording 
prices derived from transactions with 8 decimal places.  This seemed 
large enough to avoid losing precision for very small prices, but small 
enough to avoid overflows in computations.  Perhaps we should do that 
for the price DB too.



Related but somewhat separate, does it make sense to record more than
one price per day in the price db? If so, should we suppress saving
the price if it’s unchanged, perhaps within some tolerance, of the
previous entry on the same day? If not, do we keep the first entry or
overwrite every entry so that we end up with the last (entered) entry
for a particular day?

The point would be to not store a bunch of data that we don’t use
and to present users with more useful defaults in the transfer dialog
when they’re entering a bunch of transactions for the same day and
the same currencies/commodities. What are the possible bad impacts,
particularly in reports, especially the Advanced Portfolio Report?


Recording only one price per day probably is a good idea.  I think 
there may already be code that won't record a price if the same price 
is there on the same day, but I'm not sure what the definition of 
same is.


Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: gnucash master: Multiple changes pushed

[Mike Alexander]

--On August 9, 2015 at 10:36:52 AM +0100 John Ralls 
jra...@ceridwen.us wrote:



m

On Aug 8, 2015, at 5:00 PM, gnucash-patches-requ...@gnucash.org
wrote: commit 1e16ba6c14ccec6222a8fe798e7824285fd49cac
Author: Mike Alexander m...@umich.edu
Date:   Fri Aug 7 19:43:12 2015 -0400

   Quote the date returned by get_quote_time so Scheme can read it.

commit 655c04bd68d984272d8593ebbcff10f3dbaac93b
Author: Mike Alexander m...@umich.edu
Date:   Thu Aug 6 17:43:57 2015 -0400

   Avoid passing invalid arguments to functions to get rid of some
   non-fatal asserts.



Mike,

The first one and possibly the second should have been made in maint
and then merged to master.
http://wiki.gnucash.org/wiki/Git#Branching_and_Merging. Just because
there’s no bug report doesn’t mean it isn’t a bug!

Oddly, I did fix the first last Sunday and it sure looks like I
pushed it (20242c5 Put the quotes back around the date string in
gnc-fq-helper.). I just pulled a fresh clone from Code and it’s
there, but it doesn’t show up in Github and didn’t get pulled in
to Geert’s merge — which might have happened because Geert pulls
from Github and only pushes to Code.


You're right, I apologize.  For the first one I thought your change 
d2798b8c was only on master, but I see that it was also on maint.  For 
the second one, I just forgot.  They were just little changes I found 
in passing while looking for other problems and I didn't really think 
about where to put them.  I'll try to do better in the future.


I just tried to fix this, but I think there is something wrong with the 
remote repositories.  The GitHub and Code repositories seem to be out 
of synch.  If I pull maint from GitHub and then try to push a change to 
Code, Git tells me I'm out of date and need to do a pull.  I'm set up 
to pull from GitHub and push to Code, and they seem to be out of synch. 
This is probably why your change 20242c5 isn't on GitHub.  I could pull 
from Code and push there, but I'm afraid I'll just make things worse. 
If someone fixes the Git confusion, I'll fix my mistakes.


  Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: io-gncxml-v1.c

[Mike Alexander]

 On Jun 14, 2015, at 3:17 AM, John Ralls jra...@ceridwen.us wrote:
 
 Mike,
 
 I’m working on extracting the KVP operations and this file has a bunch. From 
 history it looks like you’ve been maintaining it recently. Do you have a test 
 file?

I recall vaguely doing some work on it a few years ago, but I don't think it 
was more than fixing a few minor bugs.  Right now I'm out of the country and 
away from my development machine, but I don't think I have much in the way of 
test files.  I'll be home around the end of June and will let you know if I'm 
remembering wrong.

  Mike
 
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Difficulty building on MacOSX

[Mike Alexander]

--On May 5, 2015 at 6:18:09 PM -0700 John Ralls jra...@ceridwen.us 
wrote:



I obviously haven’t encountered that problem, and ISTM our
environments for this purpose should be pretty much identical, at
least when you’re using the Xcode compiler. The one possibility
that comes to mind is optimization: I’m building with -O0 so that I
can easily use the debugger.

You can cd to src/libqof/qof and make check; you should get the same
crash on test-gnc-timezone if you have google test or on test-qof if
you don’t. That will provide a convenient program to run the
debugger on to see if you can get some more info.

The stack trace looks like the problem is coming from trying to reuse
a supposedly freed block, so the real bug is probably somewhere else.
Try setting $MallocScribble and see if it will crash where the real
bug is.


I just pushed a change that seems to fix this.  The thing about my 
setup that caused the failure is that (for reasons I haven't 
investigated) sizeof(time_t) is 8 but I have a version 1 tzfile.  The 
code that parsed the timezone file was using sizeof(time_t) for the 
size of the offsets in the file, but a version 1 file has 4 byte 
offsets no matter what sizeof(time_t) is.  This meant that it was byte 
swapping stuff that wasn't in the input buffer (as well as putting junk 
in the transitions list, of course).


I also pushed a trivial change to support version 3 tzfile format. 
Version 3 is essentially identical to version 2 and the differences 
don't seem to matter to this code.


If you think my change is inappropriate let me know and I'll back it 
out.


  Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Difficulty building on MacOSX

[Mike Alexander]

--On May 7, 2015 at 7:19:49 AM -0700 John Ralls jra...@ceridwen.us 
wrote:



Surely you didn’t rebuild libc just to get a 64-bit time_t. What
motivated you? Since you went to the work of replacing libc and the
headers in your SDK (and have to redo it every time Apple pushes an
Xcode upgrade), why not upgrade your TZ database as well?



No, I didn't rebuild libc or anything like that.

When I compile gnc-timezone.cpp the command used is

clang++ -DHAVE_CONFIG_H -I. -I../../../../../gnucash/src/libqof/qof 
-I../../.. -I../../../../../gnucash/lib/libc 
-I../../../../../gnucash/src -D_REENTRANT -I/opt/local/include/glib-2.0 
-I/opt/local/lib/glib-2.0/include -I/opt/local/include 
-I/opt/local/include -DG_LOG_DOMAIN=\qof\ -I/opt/local/include 
-DDUMP_FUNCTIONS -Werror -Wall -Wno-unused -Wno-deprecated-register 
-std=gnu++11 -g -O0 -MT gnc-timezone.lo -MD -MP -MF 
.deps/gnc-timezone.Tpo -c 
../../../../../gnucash/src/libqof/qof/gnc-timezone.cpp  -fno-common 
-DPIC -o .libs/gnc-timezone.o


In this compilation, time_t is defined in 
/usr/include/sys/_types/_time_t.h as


  typedef __darwin_time_t time_t;

__darwin_time_t is defined in /usr/include/i386/_types.h as

  typedef long __darwin_time_t;

I'm using

Apple LLVM version 6.1.0 (clang-602.0.49) (based on LLVM 3.6.0svn)
Target: x86_64-apple-darwin14.3.0
Thread model: posix

to compile this.   It's located in /usr/bin/clang++.

In that compilation sizeof(long) is 8.

/etc/localtime is a symlink to /usr/share/zoneinfo/America/Detroit 
which was last changed on Nov. 16.  I've never touched that file, or 
either of the include files, myself.  I don't know why my machine is 
different from yours.


 Mike



___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Difficulty building on MacOSX

[Mike Alexander]

--On May 5, 2015 at 6:18:09 PM -0700 John Ralls jra...@ceridwen.us 
wrote:



I obviously haven’t encountered that problem, and ISTM our
environments for this purpose should be pretty much identical, at
least when you’re using the Xcode compiler. The one possibility
that comes to mind is optimization: I’m building with -O0 so that I
can easily use the debugger.


So am I, so that isn't the difference.



You can cd to src/libqof/qof and make check; you should get the same
crash on test-gnc-timezone if you have google test or on test-qof if
you don’t. That will provide a convenient program to run the
debugger on to see if you can get some more info.

The stack trace looks like the problem is coming from trying to reuse
a supposedly freed block, so the real bug is probably somewhere else.
Try setting $MallocScribble and see if it will crash where the real
bug is.


I ran test-qof under lldb and stopped at malloc_error_break.  Using 
this with various malloc-related environment variables gave me a stack 
traces for where the offending block was allocated and one for where it 
was released.  I've put these in a file in 
https://www.dropbox.com/sh/wghtw6xnqv4ygc5/AACtTnDB1oKgSnFoD_E6zREAa?dl=0 
along with the crash reports.  It looks like it's a character string 
allocated by IANAParser, but I haven't looked further yet.  It's quite 
possible that this string is only an innocent victim of some code that 
thinks it is changing something else.


  Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Difficulty building on MacOSX

[Mike Alexander]

I'm having trouble building the X window version of Gnucash with the 
recent changes to use Boost for data and time on MacOSX 10.10.3 with 
XCode 6.3.1.  I've tried using both clang as provided by XCode and GCC 
4.8 as provided by MacPorts and get similar failures.  In each case 
either the first or second attempt to run guild as part of the make 
fails with the error


*** error for object 0x7fa6a8413178: incorrect checksum for freed 
object - object was probably modified after being freed.


This happens when some guile code (either unittest-support.scm or 
core-utils.scm) tries to load a GnuCash extension which causes the 
timezone tables to be initialized.  I've had at least two different 
stack traces leading up to this failure.  An example of each is in 
https://www.dropbox.com/sh/wghtw6xnqv4ygc5/AACtTnDB1oKgSnFoD_E6zREAa?dl=0. 



I'll keep looking for this, but if anyone has any suggestions, I'd like 
to hear them.  Let me know if you want more information.


   Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: gnucash master: Book-Currency Feature step 1

[Mike Alexander]

--On February 9, 2015 at 10:48:30 PM -0800 Alex Aycinena 
alex.aycin...@gmail.com wrote:



That's exactly the approach planned. The book options dialogue will
be a choice of three currency accounting approaches: trading accounts,
book-currency, neither. If trading accounts is selected, it should
work just as now. If neither is selected, it should work exactly as
now when trading accounts is not selected.


That sounds ok, I was just slightly confused by your comment on the 
recent commit.  I guess you were just talking about reading the old 
file format, not more than that.  I just wanted to be sure we were all 
in agreement.


 Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: gnucash master: Book-Currency Feature step 1

[Mike Alexander]

 On Feb 9, 2015, at 4:18 PM, J. Alex Aycinena alex.aycin...@code.gnucash.org 
 wrote:
 
 Updatedvia  https://github.com/Gnucash/gnucash/commit/b22c6b6e 
 (commit)
   from  https://github.com/Gnucash/gnucash/commit/9c8405da (commit)
 
 
 
 commit b22c6b6eb9ab86273bfc7225cc50504f28158bc8
 Author: Alex Aycinena alex.aycin...@gmail.com
 Date:   Mon Feb 9 12:52:29 2015 -0800
 
Book-Currency Feature step 1
 
The changes made are:
 
 libqof/qof/qofbookslots.h - define Currency Accounting Method option; 
 keep
 the current Trading Accounts option (in order to read prior files)
 libqof/qof/qofbook.cpp  .h - define function to determine if book 
 uses
 book-currency, modify function to determine if book uses trading
 accounts and add gobject properties accordingly
 libqof/qof/test/test-qofbook.c - define test function to determine if 
 book
 uses book-currency and modify test function for trading accounts
 engine/test/utest-Split.cpp - modify test function for trading 
 accounts
 to use new currency-accounting and trading book properties
 engine/test/utest-Transaction.c - modify test function for trading 
 accounts
 to use new currency-accounting and trading book properties
 


Your comments imply that you might one day remove the trading account option.  
Please don’t, and don’t deprecate the use of it.  Trading accounts have a 
number of advantages over forcing the use of a book currency and should not be 
removed.  Peter Selinger’s tutorial mainly talks about a “reference currency” 
to show how it doesn’t work as well as trading accounts.  In no way is he 
arguing that a reference currency is superior to trading accounts, although 
some situations may require the use of a reference currency anyway.  

I missed the discussion last October about this issue since I was out of the 
country without Internet access, but I fail to see why the issues you’re 
addressing couldn’t be fixed without such a drastic change.  If you want to 
implement this approach, go ahead, but do it in a way that doesn’t interfere 
with the way things work now.  That’s the approach I took with trading accounts 
(if you turn them off GnuCash works like they had never been implemented) and 
you should do the same here.

 Mike
 
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Byte order mark always make some files modified

[Mike Alexander]

--On December 28, 2014 at 7:19:42 PM +0800 Chenxiong Qi 
qcxh...@gmail.com wrote:



And what sort of machine is this on?  Windows, Mac, or Linux?  I have
core.autocrlf set to input and it seems to work ok for me on my
Mac. You might want to reread the section on End-of-line
conversion in the git help attributes man page.


Linux. I'm running Fedora 21.


I don't know what the problem is.  I just tried this again using Git 
2.2.1 on MacOSX 10.9.5 and it works fine.  I cloned the github 
repository and switched back and forth between maint and master a 
number of times and it worked fine.  The *.win32-bin.txt and *.mt940 
files have DOS line endings in my clone as they should.  What version 
of Git are you using?  Perhaps that is the problem.  There doesn't seem 
to be anything wrong with the repository's copy of GnuCash.


  Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Byte order mark always make some files modified

[Mike Alexander]

--On December 27, 2014 at 9:22:12 AM -0800 John Ralls 
jra...@ceridwen.us wrote:





On Dec 27, 2014, at 3:50 AM, Chenxiong Qi qcxh...@gmail.com wrote:

Hi,

Following files are always marked as modified in my repos when swtich
between maint and master back and forth, sometimes these modified
files prevent git from git-checkout, meanwhile they cannot be stashed
or checked out with `checkout --`.

modified:   doc/README-de.win32-bin.txt
modified:   doc/README-fr.win32-bin.txt
modified:   doc/README-it.win32-bin.txt
modified:   doc/README-nl.win32-bin.txt
modified:   doc/README-zh_CN.win32-bin.txt
modified:   doc/README-zh_TW.win32-bin.txt
modified:   doc/README.win32-bin.txt
modified:   doc/examples/downloaded.mt940

I searched archived mails in gnucash-devel, set `git config --global
core.autocrlf input`. However, this only affects the first time when
checkout branch from master to maint. Above issue happens again when
checkout back to master from maint. Really confused, have no idea why
this happens and not sure whether it's an CRLF-related issue.


Those files' line endings are supposed to be controlled by
.gitattributes, which overrides the core.autocrlf setting. I tweaked
the globs in .gitattributes 2 Dec., and Mike Alexander removed the
line endings from the repo on the 8th. This worked correctly for me
just now, with no core.autocrlf setting. Did you experience the
problem with a fresh checkout or an older one?

README.win32-bin.txt and downloaded.mt940 don't have BOMs.


And what sort of machine is this on?  Windows, Mac, or Linux?  I have 
core.autocrlf set to input and it seems to work ok for me on my Mac. 
You might want to reread the section on End-of-line conversion in the 
git help attributes man page.


  Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: git always make some files modified

[Mike Alexander]

--On December 27, 2014 at 9:16:56 PM +0100 Christian Stimming 
christ...@cstimming.de wrote:



My solution is as follows:

- Remove the .gitattributes file
- Commit this removal into git (which makes the modified-message go
away),
- And reset the master branch back to the original master.


Removing .gitattributes is a very bad idea.  It will mess up the line 
endings in your repository.  It will only affect you if you don't push 
things to other repositories, but it is still not a good idea.  Anyone 
who pushes things to other repositories should not remove that file (or 
change it unless they are very careful).


Git repositories are supposed to have all text files normalized to LF 
line endings.  Then autocrlf controls what happens to them in your 
checked out copy.  One of the big problems is determining what files 
are text files and so should be normalized.  If you remove 
.gitattributes then you will cause git to misidentify text files and 
mess up line endings in things you commit to the repository.  This is 
particularly true on Windows machines, but can be a problem on Linux 
machines or Macs too.


Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: python-bindings: coredump whil executing gnucash.Session

[Mike Alexander]

--On December 14, 2014 at 7:02:15 AM -0800 John Ralls 
jra...@ceridwen.us wrote:



Did you regenerate the swig files? You'll need to do that often on
master. If you build in a separate directory you'll need to `git
clean -fdx` in your source directory then re-run autogen.sh and
configure. If you build in-source `make distclean` should do it.


Is that really true?  I've been building in a separate build directory 
for years and haven't ever done that.  If I want a clean build I just 
move the existing build directory aside and create a new one.  This has 
always seemed to work.  It appears to me that the generated swig C 
source files are put in the build directory, not the source directory, 
so blowing away the build directory causes them to be rebuilt.  If you 
build in the source directory a 'make distclean' is no doubt required.


   Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: commit 78fea12afc5f0db5a137d0766e92994232f60a78

[Mike Alexander]

--On December 13, 2014 at 12:13:23 PM -0800 John Ralls 
jra...@ceridwen.us wrote:



I do use MacPorts and I see that it has a really weird patch to
Glib's configure file which affects this area.  I need to figure out
what they think they are doing and what effect it may be having on
this problem. MacPorts essentially ignores configure.ac and patches
configure, which makes it harder to understand what is going on.

For  the moment I patched Apple's headers to look at __LP64__ when
defining the int64 types and this works.  It's obviously not a
permanent solution.


I think patching Glib is more appropriate.
https://bugzilla.gnome.org/show_bug.cgi?id=741488 has the patch if
you want to use it. It took an extra day thanks to Guile not building
with the 10.10 SDK. That’s
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19372.


I never meant the patch to Apple's headers to be more than an 
experiment since I didn't think it would be possible to get Apple to 
make that change, even if it is a good idea.  It does seem better to 
use long instead of long long fir 64 bit types when __LP64__ is 
set, but it doesn't really matter so long as things are consistent and 
getting a patch into GLib will be a lot easier.


Thanks for the patch, it seems to work fine.  As I mentioned, MacPorts 
has a bunch of changes in GLib's configure already, but most of them 
seem to be from years ago and are probably no longer relevant.  I threw 
them all away and regenerated configure from configure.ac and it built 
without problem.  GnuCash then built without error using this GLib.


 Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: commit 78fea12afc5f0db5a137d0766e92994232f60a78

[Mike Alexander]

--On December 11, 2014 at 8:31:03 PM -0800 John Ralls 
jra...@ceridwen.us wrote:



Thanks for reverting.


No problem, I shouldn't have committed it in the first place.



You may well be the only one who's building 64-bit on a mac. I think
pretty much everyone does on Linux, but Linux seems to declare
int64_t conditionally as long on 64-bit processors while, as you
noted, Apple defines it unconditionally as long long. No doubt they
did so to make their lives easier when they were doing the transition
from Intel Core2 to Core2 duo a few years ago and saw no reason to
change once the transition was complete. Glib's configure ignores C
defines entirely and uses AC_CHECK_SIZEOF(long) and
AC_CHECK_SIZEOF(long long) to decide; if $ac_cv_sizeof_long == 8 then
it wins.

The simplest fix would be to patch glib's configure.ac to let
$ac_cv_sizeof_long_long win if building on darwin, but I don't know
if that will fly with the Glib crew. I'll file a bug tomorrow and see
how it goes. In the meantime you can patch it locally on your
machine. You're using MacPorts IIRC. They might be more accepting of
that patch if Glib won't take it.


I do use MacPorts and I see that it has a really weird patch to Glib's 
configure file which affects this area.  I need to figure out what they 
think they are doing and what effect it may be having on this problem. 
MacPorts essentially ignores configure.ac and patches configure, which 
makes it harder to understand what is going on.


For  the moment I patched Apple's headers to look at __LP64__ when 
defining the int64 types and this works.  It's obviously not a 
permanent solution.


 Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: commit 78fea12afc5f0db5a137d0766e92994232f60a78

[Mike Alexander]

--On December 11, 2014 at 11:37:19 AM -0800 John Ralls 
jra...@ceridwen.us wrote:



Please ask on the list or file a bug before making changes to the new
code. Bear in mind that I’m developing in OS X too and using Xcode
tools daily though not the IDE, so problems you’re encountering are
perhaps not so much related to Xcode as to building 64-bit, or
perhaps to a warning that you’ve set that I haven’t. GC built
fine for me before your changes with Xcode 6.1.1. I’m also testing
this on Debian Wheezy and Jessie, Alex is testing on F19 and F20, and
the build VM tests Win32. All compiled correctly without your changes.

Regarding standard library macros (e.g. PRIu64) vs. Glib macros
(G_GINT64_FORMAT) and types (e.g. int64_t vs. gint64), maybe you
didn’t register the discussion early last year [1] that kicked off
the C++ development: We’re moving away from Glib, and it
shouldn’t be used in new code. We need to figure out how to tune
your development environment either via configure or Xcode so that it
works with the standard library macros; it might be as simple as
configuring std=C++11 in the Xcode IDE.


Sorry about that, I've reverted that commit.  I should have cleared it 
first.


I don't use the IDE to build, I never have done so.  I build the X 
Window version using the same automake tools as everyone else.  I only 
use the IDE to debug GnuCash, not build it.  I didn't change anything 
in configure or the makefiles related to warnings or the C++ standard 
level.  I do configure with --enable-compile-warnings=yes, but removing 
that didn't change anything since these errors are not warnings.


The problem I was having is because there is a difference between 
gint64 and int64_t which causes clang++ to not consider them to be 
interchangeable.  It appears that if __LP64__ is defined Glib declares 
gint64 as typedef signed long while int64_t is declared 
unconditionally as typedef long long.  It's correct that clang 
doesn't consider these equivalent, although they are both 64 bit 
integers in that environment.  Why is this affecting me and not others? 
Am I the only one building in a 64 bit environment?


This is the first time I've taken a look at the C++ work so far, and 
I'm happy with what I see.  It looks quite good and I'll try to help 
without causing too much grief.


Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: C++ work

[Mike Alexander]

--On September 9, 2014 at 3:12:33 PM -0400 Aaron Laws 
dartm...@gmail.com wrote:



Currently, c++ work is starting at the deepest point (the part of the
code that is relied on by everything), qof, so that a C api has to be
maintained until everything that relies on QOF has a way of accessing
the c++ interfaces. This means that c++ and C interfaces need to be
created and maintained in parallel until everything's ready to
switch. This has grated on me for quite a while, but I see it as a
very difficult problem. I don't see a quick way to fix it.


I don't agree that this is such a big problem.  Keeping existing C 
interfaces to code rewritten in C++ isn't that big a deal.  Sure, it 
adds a little bit of work, but not much compared to the work of 
converting the code to a proper class hierarchy in C++.  I went through 
all this in another project of similar size to GnuCash so I have some 
experience with the problems involved.  The approach we took there was 
mostly to rewrite things in C++ if they were going to be rewritten or 
extensively modified anyway.  Sometimes things were rewritten because 
the benefits of using C++ seemed great enough to be worth the effort, 
but we certainly didn't try to avoid C dependencies on C++ code.  When 
I left the company it was about half converted.  That was 8 years ago, 
and I suspect there are still parts that are in C because it isn't 
worth the cost to rewrite them.


   Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Namespaces, file names,...

[Mike Alexander]

On Sep 5, 2014, at 11:26 AM, Buddha Buck blaisepas...@gmail.com wrote:
 
 When every method name has to be Hungarian-notated to work with
 pseudonamespaces, short namespace abbreviations make sense.
 
 But with true namespaces, using directives, and namespace aliasing
 available in C++, I see benefits in readability to use full length names
 for namespaces (like GnuCash::Account instead of gncAccount or
 Gnc::Account. There's less likelyhood of collision with, say, a library
 written to support the General Nutrition Centers (GNC) chain of stores.
 
 While I personally prefer snake_case to camelCase, I think that the
 idiomatic C++ tends to use camelCase.

I agree with both points.  I think the namespace name should be spelled out in 
full and camel case is more often used in C++.  I happen to prefer it, but 
perhaps that’s just because I’ve used it more.

Mike
 
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Gnucash c++

[Mike Alexander]

On Aug 11, 2014, at 4:52 PM, Christian Stimming christ...@cstimming.de wrote:

 thanks for investing time in Gnucash and also in its development towards more 
 future-proof programming technologies. I was a bit puzzled about the benefit 
 of switching the normal compiling from C to C++, just by itself. IMHO, 
 there 
 is of course an immediate benefit if the data structures move from plain C 
 structs to C++ classes, with constructor/destructor and such. If you plan to 
 do such a transition with any of gnucash's data structures, of course every 
 code using those will have to be C++. However, just changing this into C++ 
 doesn't also solve the problem here: The usage of the C structs in the code 
 is 
 just that: C structs, with foo_new() and foo_delete() functions and maybe 
 even 
 glib's reference counting. To really use C++ classes instead, every single 
 usage of those old C idioms will have to be replaced by proper C++ 
 constructs. 
 IMHO, just switching the C compiling to C++ doesn't quite bring you much 
 gain here. Do you think it helps you much? I have some doubts. I see some 
 more 
 benefit when changing individual data structures to C++, then switching the 
 old C functions into wrappers that make the new C++ behaviour available to 
 the 
 C side. This means the existing C code can continue to compile in C, and the 
 next steps would rather be to open the possibility for new C++ code such as 
 unittests and maybe new GUI code in C++ (or python or something similar). 
 IMHO 
 this would be more benefitial. What do others think?

What you say is correct, of course, but I don’t think it’s a complete waste of 
time to switch over to C++ for most compiles.  It has a few advantages I can 
think of:

1. C++ compilers enforce more strict standards for C code and will catch 
problems the C compiler may not catch.

2. When some data structure is converted to a C++ class then there’s a good 
chance that any code that uses it will already be C++ code.

3. Header files won’t have to have as many #ifdef __cplusplus ... endif 
constructs.

These are all pretty minor, but they are useful, I think.  There are probably 
other advantages too.  On the other hand it will be a lot of work to get 
everything to work in C++.  I think only the easy part has been done so far.  
The last time I worked on something like this we did what you suggest, 
Christian, and it worked well.  Whether it’s worth doing this is not obvious to 
me.

   Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: -Wno-deprecated-register with Clang

[Mike Alexander]

--On July 2, 2014 at 3:39:07 PM -0400 Mike Alexander m...@umich.edu 
wrote:



I found my stupid mistake and GnuCash now builds ok with clang for me
(a lot faster too).  I do both an optimized and a debug build.  The
optimized build works ok, but the debug build crashes trying to
produce a vendor listing report.  It's an EXC_BAD_ACCESS exception
and the top of the stack is

0   libgnc-qof.1.dylib  0x0001069cc83d
qof_book_get_collection + 61 (qofbook.cpp:510)
1   libgncmod-engine.dylib  0x00010663a404 gncVendorLookup + 68
(gncVendor.h:116)
2   libgncmod-engine.dylib  0x00010663a2aa
gncOwnerGetOwnerFromLot + 282 (gncOwner.c:630)
3   libgncmod-engine.dylib  0x00010660b20e
_wrap_gncOwnerGetOwnerFromLot + 126 (swig-engine.c:32929)

I don't have time to look into this right now since real life is
intruding.  If no one else finds it, I'll either fix it or file a bug
report later.


I just pushed a fix for this in bacc0b2.  gncOwnerGetOwnerFromLot was 
clobbering a stack variable by storing a 64 bit owner type in a 32 bit 
variable.


   Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: -Wno-deprecated-register with Clang

[Mike Alexander]

--On July 2, 2014 at 9:58:28 AM +0200 John Ralls jra...@ceridwen.us 
wrote:



Yeah, I build all the time with Clang, otherwise I wouldn't have
needed to make that change. I'm using Xcode 5.0 because of a problem
with 5.1,  though I don't remember exactly what. I think it might
even have been something on another package. I'll have to change that
to a version-based check, I guess. I think Geert builds with Clang on
Linux from time to time so that he can run the static analyzer.

I'm adding -stdlib=libstdc++ to CXXFLAGS for building WebKitGtk
because the old version I build doesn't like libc++. It's a real PITA
to get a new version to build on OSX because the WebKitGtk team are
pretty hostile to the non-X11 backends and enjoy deliberately
breaking them. That's the only place I use -stdlib, but IIRC you're
using MacPorts and they might be using it on something that we couple
with more tightly; libofx comes to mind as one possibility.


Yes, I use MacPorts to install the prereqs for GnuCash.  In Mavericks 
they switched entirely to using libc++, I think because Apple did.  You 
can't use libstdc++ to build things with MacPorts anymore.  The version 
of WebKit they include builds ok that way, but I build the X11 version 
not the native version.


I found my stupid mistake and GnuCash now builds ok with clang for me 
(a lot faster too).  I do both an optimized and a debug build.  The 
optimized build works ok, but the debug build crashes trying to produce 
a vendor listing report.  It's an EXC_BAD_ACCESS exception and the top 
of the stack is


0   libgnc-qof.1.dylib  0x0001069cc83d qof_book_get_collection 
+ 61 (qofbook.cpp:510)
1   libgncmod-engine.dylib  0x00010663a404 gncVendorLookup + 68 
(gncVendor.h:116)
2   libgncmod-engine.dylib  0x00010663a2aa gncOwnerGetOwnerFromLot 
+ 282 (gncOwner.c:630)
3   libgncmod-engine.dylib  0x00010660b20e 
_wrap_gncOwnerGetOwnerFromLot + 126 (swig-engine.c:32929)


I don't have time to look into this right now since real life is 
intruding.  If no one else finds it, I'll either fix it or file a bug 
report later.


  Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: gnucash master: Multiple changes pushed

[Mike Alexander]

--On July 2, 2014 at 5:01:10 PM -0400 John Ralls 
jra...@code.gnucash.org wrote:



Updated  via  https://github.com/Gnucash/gnucash/commit/29923b1f
(commit) via  https://github.com/Gnucash/gnucash/commit/773326b7
(commit)from  https://github.com/Gnucash/gnucash/commit/d3389828
(commit)



commit 29923b1fd9676123411ec69a271e4b5801e8ad92
Author: John Ralls jra...@ceridwen.us
Date:   Wed Jul 2 22:55:36 2014 +0200

Replace '\0' with NULL as char* argument, avoiding warning.

commit 773326b7c554df5ea3e6f3255d7bed4480587273
Author: John Ralls jra...@ceridwen.us
Date:   Wed Jul 2 22:54:39 2014 +0200

Fix typo from 859866a.


Why did you put the two calls to AC_MSG_RESULT back?  They don't match 
any calls to AC_MSG_CHECKING so they just print OK or No without 
any apparent reason.  The action-if-true and action-if-false parameters 
to AC_COMPILE_IFELSE are optional according to the writeup and my 
testing.


  Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

-Wno-deprecated-register with Clang

[Mike Alexander]

I just tried to compile GnuCash with clang and it failed because of the 
change in 4aed8b3 to not set -Wno-deprecated-register with clang.  The 
version of clang in Mavericks (MacOSX 10.9) certainly recognizes this 
parameter and won't compile GnuCash without it.  Which version of clang 
were you using which failed to recognize it?  I'm using Apple LLVM 
version 5.1 (clang-503.0.40) (based on LLVM 3.4svn).


Has anyone built GnuCash successfully with clang?  Even after I fixed 
that problem, I got lots of link errors that looked like a mismatched 
C++ library.  I haven't looked into this much yet, but it's probably 
because Mavericks replaced libstdc++ with libc++.  Since gcc works fine 
for me, it's not a high priority to fix this.


Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Rethinking Numeric

[Mike Alexander]

--On May 31, 2014 11:19:23 PM +0200 Christian Stimming 
christ...@cstimming.de wrote:



But back to your initial question: You said we occasionally
encounter  overflow errors. I don't understand (yet) what the
actual problem is. With  our current rational numbers and int64_t
numerator we have approx. 19 decimal  digits of precision (see [2]
for the digits of a 64 bit signed integer), if I  consider the
numerator as fully used.

Are 19 significant decimal digits not enough? Are there thinkable
cases when  they are not enough? I tend to think the problem is
rather found in our  rational number's rounding, which is not the
suitable rounding method for our  financial application domain. If
this is the problem, a different data type  that does the rounding
always according to decimal numbers, and not according  to (in normal
float/double calculations) binary floating point numbers, or (in
gnc_numeric) according to rational numbers with some potentially
unknown  denominator.



Rounding along with division is at least part of the cause of 
overflows.  I dealt with a number of overflow problems in the advanced 
portfolio report that were the result of the interaction of rounding 
and division.  I solved it by essentially converting things to a 
decimal notation.  I picked a (hopefully) sufficiently large decimal 
denominator and rounded relevant calculations to that denominator.  I 
think this is more or less what you're suggesting in the context of the 
current implementation.


Division of rational numbers, which is what GnuCash does, involves the 
calculation of GCDs (or something similar) which can produce very large 
integers.  Years ago I was involved with the Reduce algebraic system 
which implements arbitrary precision rational numbers mainly for this 
reason.  It needs to exactly divide rational numbers with no rounding 
so it can, for example, factor polynomials.  All but the most trivial 
calculations quickly exceeded the range of hardware integers.  We don't 
need that, of course.  Our problem seems to be that we're using 
rational numbers where it isn't really necessary.


Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: gnucash master: A truly ancient bug, discovered with an Xcode-5.1 compiler warning.

[Mike Alexander]

--On May 19, 2014 7:07:46 PM -0400 John Ralls jra...@code.gnucash.org 
wrote:



Updated  via  https://github.com/Gnucash/gnucash/commit/eabaee8e
(commit)from  https://github.com/Gnucash/gnucash/commit/6e62ce99
(commit)



commit eabaee8eb58c557743b8b1b476b4145b97eb9836
Author: John Ralls jra...@ceridwen.us
Date:   Thu May 15 17:04:26 2014 -0700

A truly ancient bug, discovered with an Xcode-5.1 compiler
warning.


Should this go on maint?

 Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: private-kvp merge reverted other changes since November.

[Mike Alexander]

On May 13, 2014, at 11:47 AM, John Ralls jra...@ceridwen.us wrote:
 
 Yeah, it would be silly to merge after every commit. One strategy might be to 
 frequently merge from master and then revert the merge if there are no 
 conflicts. ISTM that relying on rerere in the face of ongoing development on 
 both branches risks replaying what was the right answer with last week's code 
 but isn't with this week's, so if there are conflicts resolved in the merge 
 it stays so that new code builds on the resolved result rather than 
 continuing to diverge.
 
 I think merges into master should happen for every module (e.g. gncguid, 
 gncdate) so that conflicts coming into master are limited to a single 
 subject. That should make it a bit easier for the person doing the merge to 
 keep track of which way to resolve the conflicts.
 
 That will still produce a ladder appearance in gitk and friends. I don't find 
 that objectionable, but apparently some people do.

I agree that merging master into topic branches frequently is a good idea.  I 
have a long running branch where I do all my real work and merge master into it 
often.  This seems to work well.  I've also created more short lived topic 
branches and used them like this.

In the other direction, I think that once a topic branch is merged to master it 
should be abandoned (unless it is used to fix bugs in the stuff that was just 
merged).  A new topic branch should be created for subsequent changes, even if 
they are related to the changes that were just merged (such as the same changes 
to a different part of the code).  I'm not sure if this is what you're 
suggesting or not, but it would seem to avoid a ladder appearance (if I know 
what you mean by that).

   Mike
 
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: private-kvp merge reverted other changes since November.

[Mike Alexander]

--On May 13, 2014 9:17:58 PM +0100 Colin Law clan...@gmail.com wrote:


It’s not. I see no reason to abandon a branch just because it’s
merged into master, and if you really have a long-running branch
where you do all of your work, neither do you. It won’t avoid the
ladder look, either. There will just be a bunch of shortish branches
instead of one long one.


If you want to work in that way I suggest having a look at git rebase.
 Rather than merging the branch into master this effectively moves the
base of the branch along to the current master and makes the tree look
much simpler.


That's what I do.  I rebase my branches onto master each time it is 
updated.  This seems to work well and keeps the tree much simpler.


  Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: private-kvp merge reverted other changes since November.

[Mike Alexander]

--On May 13, 2014 9:35:58 PM -0700 John Ralls jra...@ceridwen.us 
wrote:



That's the SVN way. We discussed this back in March [1] and decided
that we're not going to do that anymore. If you want to revisit that
you need a better argument than that's the way I've always done it,
considering that the Git community at large doesn't seem to consider
it a best practice.


So what has been wrong with my commits that you would like me to fix? 
Do you think absolutely everything should go on a branch and get merged 
into master or maint?  Or is it ok to put small changes directly on 
master or maint?


 Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: private-kvp merge reverted other changes since November.

[Mike Alexander]

--On May 6, 2014 10:41:17 PM -0400 John Ralls jra...@ceridwen.us 
wrote:



That's unfortunate. It means that our strategy of merging instead of
rebasing isn't going to work.

I can't address the details until I get back from travel next week --
and it may take a few more days of catch up after that -- but I'll go
through the diffs in detail and try to repair any regressions.

Going forward we need to consider how to handle long-running and
deferred branches (private-kvp was an instance of the latter: I wrote
most of it in November and early December, but decided that we were
too close to final release to merge it). Obviously more and better
tests would help, but that's not a realistic prescription. Any ideas,
anyone?



There has to be a good way to do this since otherwise using topic 
branches in GIT would never work right.  Clearly this isn't the case so 
there must be something we don't understand.


Rather than try to find and fix all the problems created, it might be 
better to revert the merge and figure out how to do it correctly. 
There are a lot of problems with the merge and fixing them all is going 
to be difficult and error prone.


  Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: private-kvp merge reverted other changes since November.

[Mike Alexander]

--On May 7, 2014 6:41:03 PM +0200 Geert Janssens 
janssens-ge...@telenet.be wrote:



On Tuesday 06 May 2014 18:16:51 Mike Alexander wrote:

Just to pick another random example, the merge also removed a call to
qof_instance_set_dirty in gnc_template_register_save_xfrm_cell which
is in register/ledger-core/split-register-model-save.c.  This call
was added in 613ba0d on December 7.  This is only one of a number of
changes I noticed.

I think that is actually correct. From how I understand John's work
qof_instance_set takes care of properly dirtying the kvp. If I'm
mistaken here then there are many places in the new code that no
longer  mark kvp's as dirty.


Perhaps it is a desired change, but then it should have been on the 
private-kvp branch.  Instead it was introduced as a side effect of the 
merge back to master.  Even if the call to qof_instance_set_dirty is 
not needed after the private-kvp changes, it won't hurt anything.  If 
it is to be removed it should be removed explicitly, not as a side 
effect of the merge.


   Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

private-kvp merge reverted other changes since November.

[Mike Alexander]

I get this error when I quit GnuCash (master branch) with a report open:

Backtrace:
In ice-9/boot-9.scm:
157: 7 [catch #t #catch-closure 116f3a2c0 ...]
In unknown file:
  ?: 6 [apply-smob/1 #catch-closure 116f3a2c0]
  ?: 5 [call-with-input-string gnc:report-generate-restore-forms ...]
In ice-9/boot-9.scm:
2320: 4 [save-module-excursion #procedure 1185f6600 at 
ice-9/eval-string.scm:65:9 ()]

In ice-9/eval-string.scm:
 44: 3 [read-and-eval #input: string 1181f69c0 #:lang ...]
 37: 2 [lp gnc:report-generate-restore-forms]
In ice-9/eval.scm:
393: 1 [eval #memoized gnc:report-generate-restore-forms ()]
In unknown file:
  ?: 0 [memoize-variable-access! # #]

ERROR: In procedure memoize-variable-access!:
ERROR: Unbound variable: gnc:report-generate-restore-forms


This appears to be because the merge of the private-kvp branch 
accidentally reverted part of 87c9a3b from Feb. 27.  That commit 
changed the function name to gnc:report-serialize, but the private-kvp 
merge changed the call back but not the definition.  This is only one 
of many changes reverted by this merge.  Since the private-kvp branch 
is based on c2d93cbe from November 3, other changes since then have 
been reverted also.


I compared two diffs: git diff c2d93cbe 207bedb and git diff acad5a0 
f49983b which I think should be functionally much the same.  The first 
is changes made on the private-kvp branch and the second is changes 
made by the merge from it to master.  In fact the two diffs are quite 
different.  Among the differences is the one that causes the error 
quoted above.


Just to pick another random example, the merge also removed a call to 
qof_instance_set_dirty in gnc_template_register_save_xfrm_cell which is 
in register/ledger-core/split-register-model-save.c.  This call was 
added in 613ba0d on December 7.  This is only one of a number of 
changes I noticed.


 Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Crash paying invoice

[Mike Alexander]

--On May 5, 2014 12:11:32 AM -0400 John Ralls jra...@ceridwen.us 
wrote:



Turned out not to be so hard after all, there's just an extra level
of indirection required. Fixed in edd85fa.


Thanks for fixing it.  It seems to work fine for me now.

  Mike

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Crash paying invoice

[Mike Alexander]

Something in private-kvp branch that was merged in a day or two ago 
seems to have broken part of the business features.  If I try to pay an 
invoice for a customer I get a crash in the code that creates the 
payment window.  The top of the stack is


* thread #1: tid = 0x81af3a, 0x000f7c32 
libgobject-2.0.0.dylib`g_type_check_instance_cast + 38, queue = 
'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, 
address=0x10002)


   frame #0: 0x000f7c32 
libgobject-2.0.0.dylib`g_type_check_instance_cast + 38


 * frame #1: 0x0001139eb92e 
libgncmod-business-gnome.dylib`gnc_payment_dialog_owner_changed(pw=0x7fe33b2ceff0) 
+ 78 at dialog-payment.c:461


   frame #2: 0x0001139ed0c6 
libgncmod-business-gnome.dylib`new_payment_window(owner=0x7fff52546fc0, 
book=0x7fe33c83c6d0, invoice=0x7fe33a13a7f0) + 2812 at 
dialog-payment.c:991


   frame #3: 0x0001139ed504 
libgncmod-business-gnome.dylib`gnc_ui_payment_new_with_invoice(owner=0x7fe33f26f350, 
book=0x7fe33c83c6d0, invoice=0x7fe33a13a7f0) + 110 at 
dialog-payment.c:1101


   frame #4: 0x0001139e0cd5 
libgncmod-business-gnome.dylib`gnc_invoice_window_payment_cb(widget=0x, 
data=0x7fe33f26f1f0) + 139 at dialog-invoice.c:1078


   frame #5: 0x0001139f4f90 
libgncmod-business-gnome.dylib`gnc_plugin_page_invoice_cmd_pay_invoice(action=0x7fe33a144070, 
plugin_page=0x7fe33a9004b0) + 294 at gnc-plugin-page-invoice.c:908


I verified that f49983b fails and acad5a0 works.  I tried to do a 
bisect to find the problem, but many of the intermediate commits won't 
build cleanly on my machine so I gave up.  When I reverted that merge 
things started working again.  Hopefully this will be easy to 
reproduce, but if you need more info let me know.


Mike


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: gnucash maint: Multiple changes pushed

[Mike Alexander]

On May 2, 2014, at 10:01 AM, Geert Janssens janssens-ge...@telenet.be wrote:

 I have pushed a commit that only adds _FORTIFY_SOURCE if compilation with 
 optimization is detected.
 
 Feel free to improve it if you have a cleaner solution.
 
 Geert

Thanks for fixing it.  Just out of curiosity, what version of gcc are you 
using?  I didn't get this error using 4.7 on MacOSX.  I'll have to admit that I 
didn't really know whether adding this unconditionally was the right thing or 
not, but that's what the code was trying to do so that's what I made it do.

 Mike
 

smime.p7s
Description: S/MIME cryptographic signature
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: gnucash maint: Add _FORTIFY_SOURCE directive only when compiling with optimization enabled

[Mike Alexander]

On May 2, 2014, at 9:58 AM, Geert Janssens gjanss...@code.gnucash.org wrote:
 Updatedvia  https://github.com/Gnucash/gnucash/commit/ca480862 
 (commit)
   from  https://github.com/Gnucash/gnucash/commit/08c59b58 (commit)
 
 
 
 commit ca48086287045c0af08b0dde62ce121e00a8e0c0
 Author: Geert Janssens janssens-ge...@telenet.be
 Date:   Fri May 2 15:55:48 2014 +0200
 
Add _FORTIFY_SOURCE directive only when compiling with optimization enabled


This change seems to add _FORTIFY_SOURCE for any level of -O, including -O0.  
Am I reading it correctly?  Does this work?  If so, that's why I didn't see the 
problem.  My build script always supplies a -O parameter, either -O0 or -O2 for 
debug or optimized builds respectively.  The error message seems to imply that 
you need an optimization level greater than 0 for _FORTIFY_SOURCE.

   Mike
 

smime.p7s
Description: S/MIME cryptographic signature
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel