Re: OFX import issue: Was: Is ofx import buildable in master?
Le 2023-04-24 à 08 h 24, Thomas Baumgart via KMyMoney-devel a écrit : On Montag, 24. April 2023 00:38:27 CEST Jack via KMyMoney-devel wrote: For Jonatan's loan case, it is used as a placeholder in the principal and interest splits of a scheduled loan repayment transaction. They should be replaced with the actual values when the scheduled transaction is entered. Maybe something failed at that point and the value got into a real transaction unchanged. For the test file scenario one probably needs to have a KMM file with a loan, a scheduled repayment transaction which is due and an import file that contains that transaction. Then importing the file should show the problem. I expect this to happen somewhere in the match of scheduled transactions in https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/converter/mymoneystatementreader.cpp#L1361-L1369 which calls ScheduledTransactionMatchFinder::findMatchInMatchCandidatesList https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/converter/scheduledtransactionmatchfinder.cpp#L28-L44 which then calls KMyMoneyUtils::scheduledTransaction() https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/kmymoneyutils.cpp#L340-L355 and that should take care of calculating the distribution using calculateAutoLoan() at https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/kmymoneyutils.cpp#L270-L277 using MyMoneyForecast::calculateAutoLoan() at https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/mymoney/mymoneyforecast.cpp#L1612-L1683 Using the original data and a bit of debugging would be the easiest to find out what is going on. Not sure if Jonatan is capable of doing that. @Jonatan: if possible, keep a copy of the KMM file and the imported statement aside from your real data so that you can always re-run this. Ok that took a bit of effort being able to (almost) debug c++ again. Here are my finding. Not sure how clear my idea ares thougth. in https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/converter/mymoneystatementreader.cpp#L1361-L1369 before calling handleMatchingOfScheduledTransaction matchedSplit has m_value matching the loan periodicPayment matchedSchedule.m_transaction has 3 split: 0 "Amortization" matching the loan periodicPayment 1 "Amortization" LLONG_MIN 2 "Interest" LLONG_MIN transactionUnderImport has 3 split 0 "Amortization" matching the loan periodicPayment 1 "Amortization" matching the principal paid 2 "Interest" matching the interest paid That transactionUnderImport seems to be the one properly calculated within calculateAutoLoan Then within handleMatchingOfScheduledTransaction, it seems to save schedule which is matchedSchedule with some missing info. Now where I'm unsure. is it inside handleMatchingOfScheduledTransaction that the wrong data is used, or should matchedSchedule also contain the rigth info. I could give another shot at debugging it if you need more info. I can also open a bug now that it's confirmed if you want to continue in the bug tracker. Last thing, I tried reproducing it in a clean file, but I cant make the transaction match thus it's hard for me to provide a way to reproduce it on your end. I could provide the ofx file thought and some info from my .kmy save if it can be usefull. Thanks for the info you did provide it did get me looking in the rigth area. Jonatan Cloutier
Re: OFX import issue: Was: Is ofx import buildable in master?
Jonatan Cloutier Le 2023-04-24 à 08 h 24, Thomas Baumgart via KMyMoney-devel a écrit : On Montag, 24. April 2023 00:38:27 CEST Jack via KMyMoney-devel wrote: On 2023.04.23 11:35, Jonatan Cloutier via KMyMoney-devel wrote: [snip.] On the other side, I was able to confirm that there seems to be a real issue in regards to loan scheduled transaction, when matched to an import. My mortage just jump to a higher amount than the us debt! (by exactly 92 233 720 368 547 758,07 which is actually LLONG_MAX/100 ??). I'll try reproducing that in a test file. Let me know if there is any specific information that might be valuable. I'll open a bug when I feel I have enough info and that I'm not the one that created the issue with either my custom import or some manual change I would have done in my file. I was about to file a wishlist for matching the ending balance of the brokerage account when importing an investment account (using libofx) and noticed something strange. The ofxlog.txt included the snippets "234.56" and "TotalMarketValueTotal Market ValueDOLLAR123.45" (amounts edited) which match the balance of the brokerage account and the value of the investment account. However, in kmm-statement-2023-04-23T22-16-58.913.txt, the line beginning No conversion or calculation gets confused. It's certainly not libofx but the KMyMoney statement reader. That value you see is a constant provided with MyMoneyMoney::autoCalc. Actually it is not LLONG_MAX/100 but (LLONG_MIN+1)/100, but Jonatan's guess was close :) I noticed it is used by the statement reader. Here are some references: https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/mymoney/mymoneymoney.cpp#L61 https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/mymoney/mymoneymoney.cpp#L383-L386 https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/mymoney/mymoneystatement.h#L101 https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/converter/mymoneystatementreader.cpp#L575 So for the case of the statement file it simply means, that no balance information was extracted from the statement. For Jonatan's loan case, it is used as a placeholder in the principal and interest splits of a scheduled loan repayment transaction. They should be replaced with the actual values when the scheduled transaction is entered. Maybe something failed at that point and the value got into a real transaction unchanged. For the test file scenario one probably needs to have a KMM file with a loan, a scheduled repayment transaction which is due and an import file that contains that transaction. Then importing the file should show the problem. I expect this to happen somewhere in the match of scheduled transactions in https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/converter/mymoneystatementreader.cpp#L1361-L1369 which calls ScheduledTransactionMatchFinder::findMatchInMatchCandidatesList https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/converter/scheduledtransactionmatchfinder.cpp#L28-L44 which then calls KMyMoneyUtils::scheduledTransaction() https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/kmymoneyutils.cpp#L340-L355 and that should take care of calculating the distribution using calculateAutoLoan() at https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/kmymoneyutils.cpp#L270-L277 using MyMoneyForecast::calculateAutoLoan() at https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/mymoney/mymoneyforecast.cpp#L1612-L1683 Using the original data and a bit of debugging would be the easiest to find out what is going on. Not sure if Jonatan is capable of doing that. @Jonatan: if possible, keep a copy of the KMM file and the imported statement aside from your real data so that you can always re-run this. I did take a backup of both the file before I manually fix all of the previous one and before I did the last import. I'm actually waiting for this week payment so that I can have a smaller import to debug. I'm not quite setup to do proper c++ debug, but with the pointer given above, I might try with some debug print. I'll let you know my finding, probably toward the weekend. I also confirm it's not ofx, since my custom import use the kmm statement import directly, although, I'll debug with real ofx file as this is the real supported way to do it.
Re: OFX import issue: Was: Is ofx import buildable in master?
On Montag, 24. April 2023 00:38:27 CEST Jack via KMyMoney-devel wrote: > On 2023.04.23 11:35, Jonatan Cloutier via KMyMoney-devel wrote: > [snip.] > > On the other side, I was able to confirm that there seems to be a > > real issue in regards to loan scheduled transaction, when matched to > > an import. My mortage just jump to a higher amount than the us debt! > > (by exactly 92 233 720 368 547 758,07 which is actually LLONG_MAX/100 > > ??). I'll try reproducing that in a test file. Let me know if there > > is any specific information that might be valuable. I'll open a bug > > when I feel I have enough info and that I'm not the one that created > > the issue with either my custom import or some manual change I would > > have done in my file. > I was about to file a wishlist for matching the ending balance of the > brokerage account when importing an investment account (using libofx) > and noticed something strange. > > The ofxlog.txt included the snippets "234.56" and > "TotalMarketValueTotal Market > ValueDOLLAR123.45" (amounts edited) which match the > balance of the brokerage account and the value of the investment > account. However, in kmm-statement-2023-04-23T22-16-58.913.txt, the > line beginning 'closingbalance="-9223372036854775807/100"' which is the same amount > you noticed. It looks like some numeric conversion is getting > confused, but I don't know if it's libofx or KMM itself. No conversion or calculation gets confused. It's certainly not libofx but the KMyMoney statement reader. That value you see is a constant provided with MyMoneyMoney::autoCalc. Actually it is not LLONG_MAX/100 but (LLONG_MIN+1)/100, but Jonatan's guess was close :) I noticed it is used by the statement reader. Here are some references: https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/mymoney/mymoneymoney.cpp#L61 https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/mymoney/mymoneymoney.cpp#L383-L386 https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/mymoney/mymoneystatement.h#L101 https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/converter/mymoneystatementreader.cpp#L575 So for the case of the statement file it simply means, that no balance information was extracted from the statement. For Jonatan's loan case, it is used as a placeholder in the principal and interest splits of a scheduled loan repayment transaction. They should be replaced with the actual values when the scheduled transaction is entered. Maybe something failed at that point and the value got into a real transaction unchanged. For the test file scenario one probably needs to have a KMM file with a loan, a scheduled repayment transaction which is due and an import file that contains that transaction. Then importing the file should show the problem. I expect this to happen somewhere in the match of scheduled transactions in https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/converter/mymoneystatementreader.cpp#L1361-L1369 which calls ScheduledTransactionMatchFinder::findMatchInMatchCandidatesList https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/converter/scheduledtransactionmatchfinder.cpp#L28-L44 which then calls KMyMoneyUtils::scheduledTransaction() https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/kmymoneyutils.cpp#L340-L355 and that should take care of calculating the distribution using calculateAutoLoan() at https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/kmymoneyutils.cpp#L270-L277 using MyMoneyForecast::calculateAutoLoan() at https://invent.kde.org/office/kmymoney/-/blob/master/kmymoney/mymoney/mymoneyforecast.cpp#L1612-L1683 Using the original data and a bit of debugging would be the easiest to find out what is going on. Not sure if Jonatan is capable of doing that. @Jonatan: if possible, keep a copy of the KMM file and the imported statement aside from your real data so that you can always re-run this. -- Regards Thomas Baumgart - Software has no brain - use your own! - signature.asc Description: This is a digitally signed message part.
OFX import issue: Was: Is ofx import buildable in master?
On 2023.04.23 11:35, Jonatan Cloutier via KMyMoney-devel wrote: [snip.] On the other side, I was able to confirm that there seems to be a real issue in regards to loan scheduled transaction, when matched to an import. My mortage just jump to a higher amount than the us debt! (by exactly 92 233 720 368 547 758,07 which is actually LLONG_MAX/100 ??). I'll try reproducing that in a test file. Let me know if there is any specific information that might be valuable. I'll open a bug when I feel I have enough info and that I'm not the one that created the issue with either my custom import or some manual change I would have done in my file. I was about to file a wishlist for matching the ending balance of the brokerage account when importing an investment account (using libofx) and noticed something strange. The ofxlog.txt included the snippets "234.56" and "TotalMarketValueTotal Market ValueDOLLAR123.45" (amounts edited) which match the balance of the brokerage account and the value of the investment account. However, in kmm-statement-2023-04-23T22-16-58.913.txt, the line beginning 'closingbalance="-9223372036854775807/100"' which is the same amount you noticed. It looks like some numeric conversion is getting confused, but I don't know if it's libofx or KMM itself.
OFX Import Issue
I'm running a fresh copy of KMM compiled from the master branch. When I import my paycheck from my bank it matches with similar transaction from about a year ago. All of the spits are wrong. it should be matching with a recent transaction but it appear to pick the one with the closest amount. I have set the Import setting to match transactions within 14 days but that seems to be ignored. Changing this to 60 days does not help. On the data entry settings tab I set the Same transaction if amount differs by less than 15%. The most recent transaction is about 5% more. Brendan Coupe
Re: [Kmymoney-devel] Trying to debug: Yet another OFX import issue
On Monday 09 February 2015 20:49:35 Jack wrote:
> On 2015.02.05 02:16, Thomas Baumgart wrote:
> >>> I looked into the sources and found out, that we keep the date of
> >>> the last transaction from the previous import as that date. I
> >>> remember that for HBCI we even subtract a few days from that to be
> >>> on the safe side as banks might add data later on (we've seen this
> >>> happen here via HBCI). Would that explain the behavior you
> >>> experience?
> >>
> >> I think I found the issue. The date of last update is not kept in
> >> the ONLINEBANKING part, where I first thought it would be, but in
> >> the keyvaluepairs of the account for lastImportedTransactionDate
> >> key. My credit card and bank accounts all have an appropriate value
> >> for that. However, none of my investment accounts have that
> >> keyvaluepair in the kmy file. This may be another difference
> >> between banking/credit card and investment accounts that need to be
> >> thought about again. I don't know if I can follow the code well
> >> enough to dig up where this date gets updated during an ofx import
> >> and why it is apparently handled differently for different types of
> >> accounts. I am guessing that what happens is if KMM can't find a
> >> lastImportedTransactionDate, then it defaults to either 60 days ago,
> >> or the number of days in the settings, even if the account is set to
> >> use last update date.
> >>
> > > Have I missed something, or is this a bug?
> >
> > If you don't find that value in the KVP section of the account, then
> > this is a bug. Here's how I find where it gets set:
> >
> > kmymoney/converter/mymoneystatementreader.cpp:
> > m_account.setValue("lastImportedTransactionDate",
> > s.m_dateEnd.toString(Qt::ISODate));
>
> I did file a bug (https://bugs.kde.org/show_bug.cgi?id=343878) and have
> managed to get KMM running in debug mode in kdevelop. For checking,
> credit card, and investment accounts (all with OFX direct access set
> up) the program does get to that section of code. However, only for
> the credit card account is there a valid date in s.m_dateEnd, so it
> looks like for the other two account types, it does not set
> lastImportedTransactionDate. This is a bit odd, in that I DO see that
> date in the .kmy file for my checking account. I have not yet tracked
> if there are other paths to this section of code. I'm also new enough
> with kdevelop that I can't figure out the values of all variables (such
> as s.m_closingBalance) so I can't fully follow and understand the
> program flow.
>
> Any suggestions on how to further diagnose/troubleshoot this?
Hovering over the variable in KDevelop should show its value in a tooltip type
of window. 'ddd' might be another choice of debugger frontend. I've used it in
the past a few times, but not recently.
I did a 'git grep m_dateEnd' in the top directory of our tree which shows only
a single occurrence in the OFX importer. That might be a good spot to look
into next.
--
Regards
Thomas Baumgart
GPG-FP: E55E D592 F45F 116B 8429 4F99 9C59 DB40 B75D D3BA
-
"I know, a 'real Linux geek' doesn't walk around the wall: He bangs
his head against it until it tumbles down!" -- wobo on FLUG ml
-
signature.asc
Description: This is a digitally signed message part.
___
KMyMoney-devel mailing list
KMyMoney-devel@kde.org
https://mail.kde.org/mailman/listinfo/kmymoney-devel
[Kmymoney-devel] Trying to debug: Yet another OFX import issue
On 2015.02.05 02:16, Thomas Baumgart wrote:
I looked into the sources and found out, that we keep the date of
the last transaction from the previous import as that date. I
remember that for HBCI we even subtract a few days from that to be
on the safe side as banks might add data later on (we've seen this
happen here via HBCI). Would that explain the behavior you
experience?
>
I think I found the issue. The date of last update is not kept in
the ONLINEBANKING part, where I first thought it would be, but in
the keyvaluepairs of the account for lastImportedTransactionDate
key. My credit card and bank accounts all have an appropriate value
for that. However, none of my investment accounts have that
keyvaluepair in the kmy file. This may be another difference
between banking/credit card and investment accounts that need to be
thought about again. I don't know if I can follow the code well
enough to dig up where this date gets updated during an ofx import
and why it is apparently handled differently for different types of
accounts. I am guessing that what happens is if KMM can't find a
lastImportedTransactionDate, then it defaults to either 60 days ago,
or the number of days in the settings, even if the account is set to
use last update date.
>
> Have I missed something, or is this a bug?
If you don't find that value in the KVP section of the account, then
this is a bug. Here's how I find where it gets set:
kmymoney/converter/mymoneystatementreader.cpp:
m_account.setValue("lastImportedTransactionDate",
s.m_dateEnd.toString(Qt::ISODate));
I did file a bug (https://bugs.kde.org/show_bug.cgi?id=343878) and have
managed to get KMM running in debug mode in kdevelop. For checking,
credit card, and investment accounts (all with OFX direct access set
up) the program does get to that section of code. However, only for
the credit card account is there a valid date in s.m_dateEnd, so it
looks like for the other two account types, it does not set
lastImportedTransactionDate. This is a bit odd, in that I DO see that
date in the .kmy file for my checking account. I have not yet tracked
if there are other paths to this section of code. I'm also new enough
with kdevelop that I can't figure out the values of all variables (such
as s.m_closingBalance) so I can't fully follow and understand the
program flow.
Any suggestions on how to further diagnose/troubleshoot this?
Jack
___
KMyMoney-devel mailing list
KMyMoney-devel@kde.org
https://mail.kde.org/mailman/listinfo/kmymoney-devel
Re: [Kmymoney-devel] Yet another OFX import issue
Jack,
On Wednesday 04 February 2015 18:03:24 Jack wrote:
> On 2015.02.04 15:19, Thomas Baumgart wrote:
> > Hi Jack,
> >
> > On Monday 02 February 2015 18:04:33 Jack wrote:
> > > Using OFX direct connect, I just updated an investment account. The
> > > most recent transactions imported were for 2014/12/31. The online
> > > settings have start date of import set to last update. If I delete
> >
> > one
> >
> > > of the transactions from 2014/12/31 and update the account again, it
> > > re-imports that transaction. Since the last update was today, I
> >
> > would
> >
> > > not expect it to import anything from over a month ago.
> > >
> > > Looking into the log (ofxlog.txt) the relevant line looks like
> > > "20141202" and this does not seem to change to today's date
> > > after an import, whether or not it actually imports any
> >
> > transactions as
> >
> > > new. If I explicitly set a start date of import, that date does
> >
> > show
> >
> > > up in in the next update, but if I set it back to date of
> > > last update, it goes back to 20141202. The clock on my PC is
> >
> > correct.
> >
> > > This is with git head (Version 4.7.90-826bb6aa03) as of a few weeks
> > > ago. I'll try to do some more testing, but would appreciate any
> > > suggestions as to where I might look for anything else that could
> > > affect this. I wouldn't expect a bug, as I don't recall any
> >
> > changes in
> >
> > > this area recently - have I missed something?
> > >
> > >
> > > Another minor odd point is that if it DOES import a transaction, I
> >
> > get
> >
> > > the popup showing how many transactions were downloaded, how many
> >
> > were
> >
> > > duplicates, etc. If all the downloaded transactions are
> >
> > duplicates, I
> >
> > > don't get the popup at all.
> > >
> > > Thanks for any ideas.
> >
> > I looked into the sources and found out, that we keep the date of the
> > last
> > transaction from the previous import as that date. I remember that
> > for HBCI we
> > even subtract a few days from that to be on the safe side as banks
> > might add
> > data later on (we've seen this happen here via HBCI). Would that
> > explain the
> > behavior you experience?
>
> I think I found the issue. The date of last update is not kept in the
> ONLINEBANKING part, where I first thought it would be, but in the
> keyvaluepairs of the account for lastImportedTransactionDate key. My
> credit card and bank accounts all have an appropriate value for that.
> However, none of my investment accounts have that keyvaluepair in the
> kmy file. This may be another difference between banking/credit card
> and investment accounts that need to be thought about again. I don't
> know if I can follow the code well enough to dig up where this date
> gets updated during an ofx import and why it is apparently handled
> differently for different types of accounts. I am guessing that what
> happens is if KMM can't find a lastImportedTransactionDate, then it
> defaults to either 60 days ago, or the number of days in the settings,
> even if the account is set to use last update date.
>
> Have I missed something, or is this a bug?
If you don't find that value in the KVP section of the account, then this is a
bug. Here's how I find where it gets set:
thb@thb-nb:~/devel/kmymoney (master)$ git grep "lastImportedTransactionDate" |
grep setValue
kmymoney/converter/mymoneystatementreader.cpp:
m_account.setValue("lastImportedTransactionDate",
s.m_dateEnd.toString(Qt::ISODate));
kmymoney/mymoney/mymoneyfiletest.cpp:
a.setValue("lastImportedTransactionDate", QDate(2011, 12,
1).toString(Qt::ISODate));
kmymoney/mymoney/mymoneyfiletest.cpp:
a.setValue("lastImportedTransactionDate", QDate(2011, 12,
1).toString(Qt::ISODate));
So it's only one spot, the other two are testcases.
--
Regards
Thomas Baumgart
GPG-FP: E55E D592 F45F 116B 8429 4F99 9C59 DB40 B75D D3BA
-
90 percent of all errors are sitting
60 cm in front of the display.
-
signature.asc
Description: This is a digitally signed message part.
___
KMyMoney-devel mailing list
KMyMoney-devel@kde.org
https://mail.kde.org/mailman/listinfo/kmymoney-devel
Re: [Kmymoney-devel] Yet another OFX import issue
On 2015.02.04 15:19, Thomas Baumgart wrote: Hi Jack, On Monday 02 February 2015 18:04:33 Jack wrote: > Using OFX direct connect, I just updated an investment account. The > most recent transactions imported were for 2014/12/31. The online > settings have start date of import set to last update. If I delete one > of the transactions from 2014/12/31 and update the account again, it > re-imports that transaction. Since the last update was today, I would > not expect it to import anything from over a month ago. > > Looking into the log (ofxlog.txt) the relevant line looks like > "20141202" and this does not seem to change to today's date > after an import, whether or not it actually imports any transactions as > new. If I explicitly set a start date of import, that date does show > up in in the next update, but if I set it back to date of > last update, it goes back to 20141202. The clock on my PC is correct. > This is with git head (Version 4.7.90-826bb6aa03) as of a few weeks > ago. I'll try to do some more testing, but would appreciate any > suggestions as to where I might look for anything else that could > affect this. I wouldn't expect a bug, as I don't recall any changes in > this area recently - have I missed something? > > > Another minor odd point is that if it DOES import a transaction, I get > the popup showing how many transactions were downloaded, how many were > duplicates, etc. If all the downloaded transactions are duplicates, I > don't get the popup at all. > > Thanks for any ideas. I looked into the sources and found out, that we keep the date of the last transaction from the previous import as that date. I remember that for HBCI we even subtract a few days from that to be on the safe side as banks might add data later on (we've seen this happen here via HBCI). Would that explain the behavior you experience? I think I found the issue. The date of last update is not kept in the ONLINEBANKING part, where I first thought it would be, but in the keyvaluepairs of the account for lastImportedTransactionDate key. My credit card and bank accounts all have an appropriate value for that. However, none of my investment accounts have that keyvaluepair in the kmy file. This may be another difference between banking/credit card and investment accounts that need to be thought about again. I don't know if I can follow the code well enough to dig up where this date gets updated during an ofx import and why it is apparently handled differently for different types of accounts. I am guessing that what happens is if KMM can't find a lastImportedTransactionDate, then it defaults to either 60 days ago, or the number of days in the settings, even if the account is set to use last update date. Have I missed something, or is this a bug? Jack ___ KMyMoney-devel mailing list KMyMoney-devel@kde.org https://mail.kde.org/mailman/listinfo/kmymoney-devel
Re: [Kmymoney-devel] Yet another OFX import issue
On 2015.02.04 15:19, Thomas Baumgart wrote: Hi Jack, On Monday 02 February 2015 18:04:33 Jack wrote: > Using OFX direct connect, I just updated an investment account. The > most recent transactions imported were for 2014/12/31. The online > settings have start date of import set to last update. If I delete one > of the transactions from 2014/12/31 and update the account again, it > re-imports that transaction. Since the last update was today, I would > not expect it to import anything from over a month ago. > > Looking into the log (ofxlog.txt) the relevant line looks like > "20141202" and this does not seem to change to today's date > after an import, whether or not it actually imports any transactions as > new. If I explicitly set a start date of import, that date does show > up in in the next update, but if I set it back to date of > last update, it goes back to 20141202. The clock on my PC is correct. > This is with git head (Version 4.7.90-826bb6aa03) as of a few weeks > ago. I'll try to do some more testing, but would appreciate any > suggestions as to where I might look for anything else that could > affect this. I wouldn't expect a bug, as I don't recall any changes in > this area recently - have I missed something? > > > Another minor odd point is that if it DOES import a transaction, I get > the popup showing how many transactions were downloaded, how many were > duplicates, etc. If all the downloaded transactions are duplicates, I > don't get the popup at all. > > Thanks for any ideas. I looked into the sources and found out, that we keep the date of the last transaction from the previous import as that date. I remember that for HBCI we even subtract a few days from that to be on the safe side as banks might add data later on (we've seen this happen here via HBCI). Would that explain the behavior you experience? If you mean adding a few days, no, that is not the problem. If I update twice in a row, the second time should then have a date no more than a few days ago, not near the beginning of December, about 60 days ago, which is what I'm seeing now. I suspect something has changed, since I don't recall this problem in the past. Where is that date of last import stored in the KMM file? I don't recall seeing it. I'll have to look again - but I do recall seeing some date in the ONLINEBANKING section in a format different from 20150203, where I would have expected all dates to be there in the same format. Just to explain why this is so obvious to me right now - you can recall that I've had lots of issues with OFX imports from Merrill Lynch. In some cases, it is simply easier for me to create a new transaction and then delete the downloaded one, instead of trying to modify the downloaded one. When I do this, it will download them again. For this reason, I delay deleting the originals until another day with another download, just so it wont try to download them again. My download is now starting 60 days ago, instead of a few days ago, and still 60 days ago for a second (or more) downloads in a row today. Jack ___ KMyMoney-devel mailing list KMyMoney-devel@kde.org https://mail.kde.org/mailman/listinfo/kmymoney-devel
Re: [Kmymoney-devel] Yet another OFX import issue
Hi Jack, On Monday 02 February 2015 18:04:33 Jack wrote: > Using OFX direct connect, I just updated an investment account. The > most recent transactions imported were for 2014/12/31. The online > settings have start date of import set to last update. If I delete one > of the transactions from 2014/12/31 and update the account again, it > re-imports that transaction. Since the last update was today, I would > not expect it to import anything from over a month ago. > > Looking into the log (ofxlog.txt) the relevant line looks like > "20141202" and this does not seem to change to today's date > after an import, whether or not it actually imports any transactions as > new. If I explicitly set a start date of import, that date does show > up in in the next update, but if I set it back to date of > last update, it goes back to 20141202. The clock on my PC is correct. > This is with git head (Version 4.7.90-826bb6aa03) as of a few weeks > ago. I'll try to do some more testing, but would appreciate any > suggestions as to where I might look for anything else that could > affect this. I wouldn't expect a bug, as I don't recall any changes in > this area recently - have I missed something? > > > Another minor odd point is that if it DOES import a transaction, I get > the popup showing how many transactions were downloaded, how many were > duplicates, etc. If all the downloaded transactions are duplicates, I > don't get the popup at all. > > Thanks for any ideas. I looked into the sources and found out, that we keep the date of the last transaction from the previous import as that date. I remember that for HBCI we even subtract a few days from that to be on the safe side as banks might add data later on (we've seen this happen here via HBCI). Would that explain the behavior you experience? -- Regards Thomas Baumgart GPG-FP: E55E D592 F45F 116B 8429 4F99 9C59 DB40 B75D D3BA - I can break things just fine by myself, I don't need (KDE) 4.2 to help me. -- Bill Suit - signature.asc Description: This is a digitally signed message part. ___ KMyMoney-devel mailing list KMyMoney-devel@kde.org https://mail.kde.org/mailman/listinfo/kmymoney-devel
Re: [Kmymoney-devel] Yet another OFX import issue
I was referring just to the preceding sentence. Allan Sent from my Samsung Galaxy smartphone. >> >> Too bad there isn't an ofx server out there where you can set up test >> accounts just for this sort of testing. >> > Sounds like one for the OFX list? > I don't think so. The choice of start date of download is specified on the Edit Account Dialog, Online Settings Tab, Import Details subtab. The three choices are "Last update", "Today minus" with a spinbox for number of days, or "Pick date" with a date selector. Also note that that DTSTART is part of the OFX request, not the response. That all sounds like KMM stuff, not something buried within libofx. Jack ___ KMyMoney-devel mailing list KMyMoney-devel@kde.org https://mail.kde.org/mailman/listinfo/kmymoney-devel
Re: [Kmymoney-devel] Yet another OFX import issue
On 2015.02.03 18:46, aga wrote: On 03/02/15 22:41, Jack wrote: On 2015.02.03 06:14, aga wrote: On 02/02/15 23:04, Jack wrote: Using OFX direct connect, I just updated an investment account. The most recent transactions imported were for 2014/12/31. The online settings have start date of import set to last update. If I delete one of the transactions from 2014/12/31 and update the account again, it re-imports that transaction. Since the last update was today, I would not expect it to import anything from over a month ago. Looking into the log (ofxlog.txt) the relevant line looks like "20141202" and this does not seem to change to today's date after an import, whether or not it actually imports any transactions as new. This being a new day, has 20141202" been updated to yesterday's date? Not to yesterday's date, but to 20141203, which I'm guessing is 60 days before today. It looks like it's using the "Today minus [60] days" setting instead of "Last update." I believe that setting is in the kmm file as an index, not a text value. Do we have an "off by one" issue? Not a typical one, as setting to a specific date works fine. Setting to today minus 5 days works (20150129). Setting that back to 10 days and then selecting "Last update" reverts to 20141203. So it looks like it is using the "default" of today minus 60 days if "Last update" is checked. I'll have to see if I can create a smaller test case. Too bad there isn't an ofx server out there where you can set up test accounts just for this sort of testing. Sounds like one for the OFX list? I don't think so. The choice of start date of download is specified on the Edit Account Dialog, Online Settings Tab, Import Details subtab. The three choices are "Last update", "Today minus" with a spinbox for number of days, or "Pick date" with a date selector. Also note that that DTSTART is part of the OFX request, not the response. That all sounds like KMM stuff, not something buried within libofx. Jack ___ KMyMoney-devel mailing list KMyMoney-devel@kde.org https://mail.kde.org/mailman/listinfo/kmymoney-devel
Re: [Kmymoney-devel] Yet another OFX import issue
On 03/02/15 22:41, Jack wrote: On 2015.02.03 06:14, aga wrote: On 02/02/15 23:04, Jack wrote: Using OFX direct connect, I just updated an investment account. The most recent transactions imported were for 2014/12/31. The online settings have start date of import set to last update. If I delete one of the transactions from 2014/12/31 and update the account again, it re-imports that transaction. Since the last update was today, I would not expect it to import anything from over a month ago. Looking into the log (ofxlog.txt) the relevant line looks like "20141202" and this does not seem to change to today's date after an import, whether or not it actually imports any transactions as new. This being a new day, has 20141202" been updated to yesterday's date? Not to yesterday's date, but to 20141203, which I'm guessing is 60 days before today. It looks like it's using the "Today minus [60] days" setting instead of "Last update." I believe that setting is in the kmm file as an index, not a text value. Do we have an "off by one" issue? Not a typical one, as setting to a specific date works fine. Setting to today minus 5 days works (20150129). Setting that back to 10 days and then selecting "Last update" reverts to 20141203. So it looks like it is using the "default" of today minus 60 days if "Last update" is checked. I'll have to see if I can create a smaller test case. Too bad there isn't an ofx server out there where you can set up test accounts just for this sort of testing. Sounds like one for the OFX list? Allan Jack ___ KMyMoney-devel mailing list KMyMoney-devel@kde.org https://mail.kde.org/mailman/listinfo/kmymoney-devel
Re: [Kmymoney-devel] Yet another OFX import issue
On 2015.02.03 06:14, aga wrote: On 02/02/15 23:04, Jack wrote: Using OFX direct connect, I just updated an investment account. The most recent transactions imported were for 2014/12/31. The online settings have start date of import set to last update. If I delete one of the transactions from 2014/12/31 and update the account again, it re-imports that transaction. Since the last update was today, I would not expect it to import anything from over a month ago. Looking into the log (ofxlog.txt) the relevant line looks like "20141202" and this does not seem to change to today's date after an import, whether or not it actually imports any transactions as new. This being a new day, has 20141202" been updated to yesterday's date? Not to yesterday's date, but to 20141203, which I'm guessing is 60 days before today. It looks like it's using the "Today minus [60] days" setting instead of "Last update." I believe that setting is in the kmm file as an index, not a text value. Do we have an "off by one" issue? Not a typical one, as setting to a specific date works fine. Setting to today minus 5 days works (20150129). Setting that back to 10 days and then selecting "Last update" reverts to 20141203. So it looks like it is using the "default" of today minus 60 days if "Last update" is checked. I'll have to see if I can create a smaller test case. Too bad there isn't an ofx server out there where you can set up test accounts just for this sort of testing. Jack ___ KMyMoney-devel mailing list KMyMoney-devel@kde.org https://mail.kde.org/mailman/listinfo/kmymoney-devel
Re: [Kmymoney-devel] Yet another OFX import issue
On 02/02/15 23:04, Jack wrote: Using OFX direct connect, I just updated an investment account. The most recent transactions imported were for 2014/12/31. The online settings have start date of import set to last update. If I delete one of the transactions from 2014/12/31 and update the account again, it re-imports that transaction. Since the last update was today, I would not expect it to import anything from over a month ago. Looking into the log (ofxlog.txt) the relevant line looks like "20141202" and this does not seem to change to today's date after an import, whether or not it actually imports any transactions as new. This being a new day, has 20141202" been updated to yesterday's date? Allan If I explicitly set a start date of import, that date does show up in in the next update, but if I set it back to date of last update, it goes back to 20141202. The clock on my PC is correct. This is with git head (Version 4.7.90-826bb6aa03) as of a few weeks ago. I'll try to do some more testing, but would appreciate any suggestions as to where I might look for anything else that could affect this. I wouldn't expect a bug, as I don't recall any changes in this area recently - have I missed something? Another minor odd point is that if it DOES import a transaction, I get the popup showing how many transactions were downloaded, how many were duplicates, etc. If all the downloaded transactions are duplicates, I don't get the popup at all. Thanks for any ideas. Jack ___ KMyMoney-devel mailing list KMyMoney-devel@kde.org https://mail.kde.org/mailman/listinfo/kmymoney-devel
[Kmymoney-devel] Yet another OFX import issue
Using OFX direct connect, I just updated an investment account. The most recent transactions imported were for 2014/12/31. The online settings have start date of import set to last update. If I delete one of the transactions from 2014/12/31 and update the account again, it re-imports that transaction. Since the last update was today, I would not expect it to import anything from over a month ago. Looking into the log (ofxlog.txt) the relevant line looks like "20141202" and this does not seem to change to today's date after an import, whether or not it actually imports any transactions as new. If I explicitly set a start date of import, that date does show up in in the next update, but if I set it back to date of last update, it goes back to 20141202. The clock on my PC is correct. This is with git head (Version 4.7.90-826bb6aa03) as of a few weeks ago. I'll try to do some more testing, but would appreciate any suggestions as to where I might look for anything else that could affect this. I wouldn't expect a bug, as I don't recall any changes in this area recently - have I missed something? Another minor odd point is that if it DOES import a transaction, I get the popup showing how many transactions were downloaded, how many were duplicates, etc. If all the downloaded transactions are duplicates, I don't get the popup at all. Thanks for any ideas. Jack ___ KMyMoney-devel mailing list KMyMoney-devel@kde.org https://mail.kde.org/mailman/listinfo/kmymoney-devel
