Re: [GNC] Import Transaction Matcher issue in v4

[Fross, Michael]

Hello everyone,

Just confirming the issue with Citibank QFX files.  They are changing the
FITID for the same transactions in the QFX downloads.  I'll reach out to
them, but I'm not optimistic

The download on Friday yielded the following for a specific transaction:

CREDIT
2020072112
54.00
202007210003
ACH Electronic Credit



The Download of the same account today for the same transaction yielded
this in the QFX file:

CREDIT
2020072112
54.00
202007210004
ACH Electronic Credit




On Mon, Jul 27, 2020 at 9:33 PM David Carlson 
wrote:

> Michael,
>
> Exactly.  The QFX file only contains the information that the bank would
> put on your monthly account statement along with the necessary
> infrastructure to identify which bank created the file and which account
> and the time frame which the file describes and sometimes some additional
> information that GnuCash may or may not use.  GnuCash has to synthesize the
> rest in order to create a list of full double-entry qualified
> transactions.  That is why the import wizard is so difficult for users to
> work with and even more difficult for developers to code.
>
> Unfortunately most of us are very sloppy technically in talking of the
> file as if it is a transaction list when it is really more like a bank
> statement.
>
> On Mon, Jul 27, 2020 at 8:57 PM Fross, Michael  wrote:
>
>> Hello David,
>>
>> In looking at the QFX file, I only see one side of the transaction.  This
>> file has 3 distinct transactions ( records) that are not related
>> and only one account ID is listed.
>>
>> Michael
>>
>> On Mon, Jul 27, 2020 at 5:56 PM David Carlson <
>> david.carlson@gmail.com> wrote:
>>
>>> There is another use case which GnuCash also needs to handle.  Some OFX
>>> files may contain both sides of an inter-account transfer between two
>>> accounts within the same bank because the OFX file can include multiple
>>> accounts.  In fact, I do that on a regular basis at one of my banks.  I do
>>> not test GnuCash's handling of these because I want to get the separate
>>> notes associated with each side and merge them together into the same
>>> transaction in my data file.  Thus I deliberately call both of them New
>>> rather than try to match them within the import process.  I have not
>>> checked to see if they have unique FITID's, but they would be under
>>> different ACCOUNTID's  and both can be accepted as New in release 2.6.19
>>> and prior, which is the latest version that I use.  I suppose ideally I
>>> would like gnuCash to detect this case and automatically apply the
>>> corresponding notes to both sides of the  transfer, so I don't have to do
>>> it manually.
>>>
>>> David Carlson
>>>
>>> On Mon, Jul 27, 2020 at 5:23 PM Fross, Michael 
>>> wrote:
>>>
 Got it. Thanks.  Now I understand why it used to work.  And I certainly
 wouldn't expect GnuCash to correct a bank's horrible behavior.  I'll
 keep
 an eye on the FITID and see if that's what they are doing.

 Thanks everyone for your help.  Much appreciated.

 Michael

 On Mon, Jul 27, 2020 at 5:13 PM Jean Laroche  wrote:

 > Ah, OK I get it now!
 > Yes, you should check that from one day to the next, the FTID returned
 > by citibank remains the same (for the same transaction). If not, then
 > that's going to be a problem.
 > What has changed recently with GC is that it not will not match a new
 > imported transaction with one from the register that has an online id.
 > Previous versions allowed that, so in your case, it's quite possible
 > that from one day to the next, the same transaction would be given a
 > different ID, but GC matched it to the same register transaction that
 > was previously matched.
 > So, on day 1, you'd match OFX1->Reg1, and on day two, you'd still
 match
 > OFX2 -> reg1, even though reg1 had already been matched to OFX1 the
 > previous day. So you'd see OFX2 appear as "match" and not "Add".
 >
 > The correct behavior is that OFX2 actually has the same ID as OFX1,
 and
 > it's entirely skipped by GC. But it seems that your bank is messing
 > their online ID, and this is what's causing the issue for you, along
 > with the new GC behavior.
 >
 > I'm not sure how to fix your problem. It's clearly a bank issue but
 > that's not terribly helpful as far as your problem goes. We should
 > probably add an option to defeat the new GC import behavior,
 > specifically for users like you whose bank send unreliable FTIDs/
 >
 > FYI, the new GC behavior was added because the old behavior was
 causing
 > issues with people who had recurring daily transactions: imagine
 buying
 > a cup of coffee every day: in that case GC will match all daily cups
 of
 > coffee to the same register transaction (within a date range, of
 course)
 > instead of adding the new one every day (which really is the 

Re: [GNC] Import Transaction Matcher issue in v4

[David Carlson]

Michael,

Exactly.  The QFX file only contains the information that the bank would
put on your monthly account statement along with the necessary
infrastructure to identify which bank created the file and which account
and the time frame which the file describes and sometimes some additional
information that GnuCash may or may not use.  GnuCash has to synthesize the
rest in order to create a list of full double-entry qualified
transactions.  That is why the import wizard is so difficult for users to
work with and even more difficult for developers to code.

Unfortunately most of us are very sloppy technically in talking of the file
as if it is a transaction list when it is really more like a bank
statement.

On Mon, Jul 27, 2020 at 8:57 PM Fross, Michael  wrote:

> Hello David,
>
> In looking at the QFX file, I only see one side of the transaction.  This
> file has 3 distinct transactions ( records) that are not related
> and only one account ID is listed.
>
> Michael
>
> On Mon, Jul 27, 2020 at 5:56 PM David Carlson 
> wrote:
>
>> There is another use case which GnuCash also needs to handle.  Some OFX
>> files may contain both sides of an inter-account transfer between two
>> accounts within the same bank because the OFX file can include multiple
>> accounts.  In fact, I do that on a regular basis at one of my banks.  I do
>> not test GnuCash's handling of these because I want to get the separate
>> notes associated with each side and merge them together into the same
>> transaction in my data file.  Thus I deliberately call both of them New
>> rather than try to match them within the import process.  I have not
>> checked to see if they have unique FITID's, but they would be under
>> different ACCOUNTID's  and both can be accepted as New in release 2.6.19
>> and prior, which is the latest version that I use.  I suppose ideally I
>> would like gnuCash to detect this case and automatically apply the
>> corresponding notes to both sides of the  transfer, so I don't have to do
>> it manually.
>>
>> David Carlson
>>
>> On Mon, Jul 27, 2020 at 5:23 PM Fross, Michael  wrote:
>>
>>> Got it. Thanks.  Now I understand why it used to work.  And I certainly
>>> wouldn't expect GnuCash to correct a bank's horrible behavior.  I'll keep
>>> an eye on the FITID and see if that's what they are doing.
>>>
>>> Thanks everyone for your help.  Much appreciated.
>>>
>>> Michael
>>>
>>> On Mon, Jul 27, 2020 at 5:13 PM Jean Laroche  wrote:
>>>
>>> > Ah, OK I get it now!
>>> > Yes, you should check that from one day to the next, the FTID returned
>>> > by citibank remains the same (for the same transaction). If not, then
>>> > that's going to be a problem.
>>> > What has changed recently with GC is that it not will not match a new
>>> > imported transaction with one from the register that has an online id.
>>> > Previous versions allowed that, so in your case, it's quite possible
>>> > that from one day to the next, the same transaction would be given a
>>> > different ID, but GC matched it to the same register transaction that
>>> > was previously matched.
>>> > So, on day 1, you'd match OFX1->Reg1, and on day two, you'd still match
>>> > OFX2 -> reg1, even though reg1 had already been matched to OFX1 the
>>> > previous day. So you'd see OFX2 appear as "match" and not "Add".
>>> >
>>> > The correct behavior is that OFX2 actually has the same ID as OFX1, and
>>> > it's entirely skipped by GC. But it seems that your bank is messing
>>> > their online ID, and this is what's causing the issue for you, along
>>> > with the new GC behavior.
>>> >
>>> > I'm not sure how to fix your problem. It's clearly a bank issue but
>>> > that's not terribly helpful as far as your problem goes. We should
>>> > probably add an option to defeat the new GC import behavior,
>>> > specifically for users like you whose bank send unreliable FTIDs/
>>> >
>>> > FYI, the new GC behavior was added because the old behavior was causing
>>> > issues with people who had recurring daily transactions: imagine buying
>>> > a cup of coffee every day: in that case GC will match all daily cups of
>>> > coffee to the same register transaction (within a date range, of
>>> course)
>>> > instead of adding the new one every day (which really is the correct
>>> way
>>> > to do it).
>>> >
>>> >
>>> >
>>> > Jean
>>> >
>>> >
>>> > On 7/27/20 2:45 PM, Fross, Michael wrote:
>>> > > Hi Jean.  Let me try to better explain my issue.
>>> > >
>>> > > On most days, I download QFX files from my Checking and Credit Card
>>> > > accounts and import them.  In this way I clear transactions and I can
>>> > > see what's going on.  The next day I'll do so again, and the importer
>>> > > skips those that have already been matched per your comment.s  I've
>>> been
>>> > > doing this for years and it's worked well.
>>> > >
>>> > > Since I updated to V4 the transactions that would have matched
>>> > > previously now come up with "Match Missing!" in the importer.  I
>>> don't
>>> > > want to add 

Re: [GNC] Import Transaction Matcher issue in v4

[Fross, Michael]

Hello David,

In looking at the QFX file, I only see one side of the transaction.  This
file has 3 distinct transactions ( records) that are not related
and only one account ID is listed.

Michael

On Mon, Jul 27, 2020 at 5:56 PM David Carlson 
wrote:

> There is another use case which GnuCash also needs to handle.  Some OFX
> files may contain both sides of an inter-account transfer between two
> accounts within the same bank because the OFX file can include multiple
> accounts.  In fact, I do that on a regular basis at one of my banks.  I do
> not test GnuCash's handling of these because I want to get the separate
> notes associated with each side and merge them together into the same
> transaction in my data file.  Thus I deliberately call both of them New
> rather than try to match them within the import process.  I have not
> checked to see if they have unique FITID's, but they would be under
> different ACCOUNTID's  and both can be accepted as New in release 2.6.19
> and prior, which is the latest version that I use.  I suppose ideally I
> would like gnuCash to detect this case and automatically apply the
> corresponding notes to both sides of the  transfer, so I don't have to do
> it manually.
>
> David Carlson
>
> On Mon, Jul 27, 2020 at 5:23 PM Fross, Michael  wrote:
>
>> Got it. Thanks.  Now I understand why it used to work.  And I certainly
>> wouldn't expect GnuCash to correct a bank's horrible behavior.  I'll keep
>> an eye on the FITID and see if that's what they are doing.
>>
>> Thanks everyone for your help.  Much appreciated.
>>
>> Michael
>>
>> On Mon, Jul 27, 2020 at 5:13 PM Jean Laroche  wrote:
>>
>> > Ah, OK I get it now!
>> > Yes, you should check that from one day to the next, the FTID returned
>> > by citibank remains the same (for the same transaction). If not, then
>> > that's going to be a problem.
>> > What has changed recently with GC is that it not will not match a new
>> > imported transaction with one from the register that has an online id.
>> > Previous versions allowed that, so in your case, it's quite possible
>> > that from one day to the next, the same transaction would be given a
>> > different ID, but GC matched it to the same register transaction that
>> > was previously matched.
>> > So, on day 1, you'd match OFX1->Reg1, and on day two, you'd still match
>> > OFX2 -> reg1, even though reg1 had already been matched to OFX1 the
>> > previous day. So you'd see OFX2 appear as "match" and not "Add".
>> >
>> > The correct behavior is that OFX2 actually has the same ID as OFX1, and
>> > it's entirely skipped by GC. But it seems that your bank is messing
>> > their online ID, and this is what's causing the issue for you, along
>> > with the new GC behavior.
>> >
>> > I'm not sure how to fix your problem. It's clearly a bank issue but
>> > that's not terribly helpful as far as your problem goes. We should
>> > probably add an option to defeat the new GC import behavior,
>> > specifically for users like you whose bank send unreliable FTIDs/
>> >
>> > FYI, the new GC behavior was added because the old behavior was causing
>> > issues with people who had recurring daily transactions: imagine buying
>> > a cup of coffee every day: in that case GC will match all daily cups of
>> > coffee to the same register transaction (within a date range, of course)
>> > instead of adding the new one every day (which really is the correct way
>> > to do it).
>> >
>> >
>> >
>> > Jean
>> >
>> >
>> > On 7/27/20 2:45 PM, Fross, Michael wrote:
>> > > Hi Jean.  Let me try to better explain my issue.
>> > >
>> > > On most days, I download QFX files from my Checking and Credit Card
>> > > accounts and import them.  In this way I clear transactions and I can
>> > > see what's going on.  The next day I'll do so again, and the importer
>> > > skips those that have already been matched per your comment.s  I've
>> been
>> > > doing this for years and it's worked well.
>> > >
>> > > Since I updated to V4 the transactions that would have matched
>> > > previously now come up with "Match Missing!" in the importer.  I don't
>> > > want to add them as they would create duplicates.  Transactions that
>> are
>> > > not cleared seem to be able to be matched.  But those that are
>> > > previously cleared get the "Match Missing!" issue.
>> > >
>> > > Now, based on what you told me, the QFX file for a problem transaction
>> > > has a FITID that is one higher than the ONLINE_ID in the gnucash file
>> > > for the same transaction.  So the Match Missing items seems to be
>> > correct.
>> > >
>> > > If I duplicate the transaction, delete the original, it matches which
>> I
>> > > assume is because there is not an ONLINE_ID associated with the new
>> one.
>> > >
>> > > What I don't understand is if there is an issue with GNUCash, which is
>> > > why I asked if others are having the same issue, or if it is/was an
>> > > issue with CITIBANK QFX files.  If they changed the FITID for some
>> > > reason that would cause 

Re: [GNC] Import Transaction Matcher issue in v4

[David Carlson]

There is another use case which GnuCash also needs to handle.  Some OFX
files may contain both sides of an inter-account transfer between two
accounts within the same bank because the OFX file can include multiple
accounts.  In fact, I do that on a regular basis at one of my banks.  I do
not test GnuCash's handling of these because I want to get the separate
notes associated with each side and merge them together into the same
transaction in my data file.  Thus I deliberately call both of them New
rather than try to match them within the import process.  I have not
checked to see if they have unique FITID's, but they would be under
different ACCOUNTID's  and both can be accepted as New in release 2.6.19
and prior, which is the latest version that I use.  I suppose ideally I
would like gnuCash to detect this case and automatically apply the
corresponding notes to both sides of the  transfer, so I don't have to do
it manually.

David Carlson

On Mon, Jul 27, 2020 at 5:23 PM Fross, Michael  wrote:

> Got it. Thanks.  Now I understand why it used to work.  And I certainly
> wouldn't expect GnuCash to correct a bank's horrible behavior.  I'll keep
> an eye on the FITID and see if that's what they are doing.
>
> Thanks everyone for your help.  Much appreciated.
>
> Michael
>
> On Mon, Jul 27, 2020 at 5:13 PM Jean Laroche  wrote:
>
> > Ah, OK I get it now!
> > Yes, you should check that from one day to the next, the FTID returned
> > by citibank remains the same (for the same transaction). If not, then
> > that's going to be a problem.
> > What has changed recently with GC is that it not will not match a new
> > imported transaction with one from the register that has an online id.
> > Previous versions allowed that, so in your case, it's quite possible
> > that from one day to the next, the same transaction would be given a
> > different ID, but GC matched it to the same register transaction that
> > was previously matched.
> > So, on day 1, you'd match OFX1->Reg1, and on day two, you'd still match
> > OFX2 -> reg1, even though reg1 had already been matched to OFX1 the
> > previous day. So you'd see OFX2 appear as "match" and not "Add".
> >
> > The correct behavior is that OFX2 actually has the same ID as OFX1, and
> > it's entirely skipped by GC. But it seems that your bank is messing
> > their online ID, and this is what's causing the issue for you, along
> > with the new GC behavior.
> >
> > I'm not sure how to fix your problem. It's clearly a bank issue but
> > that's not terribly helpful as far as your problem goes. We should
> > probably add an option to defeat the new GC import behavior,
> > specifically for users like you whose bank send unreliable FTIDs/
> >
> > FYI, the new GC behavior was added because the old behavior was causing
> > issues with people who had recurring daily transactions: imagine buying
> > a cup of coffee every day: in that case GC will match all daily cups of
> > coffee to the same register transaction (within a date range, of course)
> > instead of adding the new one every day (which really is the correct way
> > to do it).
> >
> >
> >
> > Jean
> >
> >
> > On 7/27/20 2:45 PM, Fross, Michael wrote:
> > > Hi Jean.  Let me try to better explain my issue.
> > >
> > > On most days, I download QFX files from my Checking and Credit Card
> > > accounts and import them.  In this way I clear transactions and I can
> > > see what's going on.  The next day I'll do so again, and the importer
> > > skips those that have already been matched per your comment.s  I've
> been
> > > doing this for years and it's worked well.
> > >
> > > Since I updated to V4 the transactions that would have matched
> > > previously now come up with "Match Missing!" in the importer.  I don't
> > > want to add them as they would create duplicates.  Transactions that
> are
> > > not cleared seem to be able to be matched.  But those that are
> > > previously cleared get the "Match Missing!" issue.
> > >
> > > Now, based on what you told me, the QFX file for a problem transaction
> > > has a FITID that is one higher than the ONLINE_ID in the gnucash file
> > > for the same transaction.  So the Match Missing items seems to be
> > correct.
> > >
> > > If I duplicate the transaction, delete the original, it matches which I
> > > assume is because there is not an ONLINE_ID associated with the new
> one.
> > >
> > > What I don't understand is if there is an issue with GNUCash, which is
> > > why I asked if others are having the same issue, or if it is/was an
> > > issue with CITIBANK QFX files.  If they changed the FITID for some
> > > reason that would cause this.  Since no one else has said they are
> > > having the issue, I'm assuming this is not a GNUCash issue.
> > >
> > > Does that better explain the issue?
> > >
> > > I really appreciate the time to respond and help me through this.
> > >
> > > Michael
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Mon, Jul 27, 2020 at 3:50 PM Jean Laroche  > > 

Re: [GNC] Import Transaction Matcher issue in v4

[David Carlson]

Fross, Michael,

Are you trying to match bank payments to credit card accounts from QFX/OFX
downloads from both sides?  If so, the FITID's might accidentally match but
the FID and/or BANKID and/or  ACCTID of the bank/credit card transaction
split line in the import file should be different from the corresponding
transaction split imported from the other financial institution.  Thus,
even if there is a FITID associated to a different split line in the same
transaction, it should not prevent matching to an incoming transaction,
and, hopefully, GnuCash would not choke on transactions imported from both
sides.

On Mon, Jul 27, 2020 at 4:53 PM Fross, Michael  wrote:

> Hi Christopher.  Alas I do not keep the QFX files.  I'll hang on to them
> for now and see if the FIDID does change.
>
> On Mon, Jul 27, 2020 at 4:20 PM Christopher Lam  >
> wrote:
>
> > As I understand the ofx spec*, the fitid should be an invariant for the
> > bank and account. By any chance do you have older qfx files to compare?
> >
> > * https://www.ofx.net/downloads.html
> >
> > On Tue, 28 Jul 2020, 4:34 am Fross, Michael,  wrote:
> >
> >> Thanks Jean / John for your thoughts.  There is a register entry that
> >> matches, IMHO, very closely.   I increased the Match Display Threshold
> >> from
> >> 1 to 3, and then to 6 (which appears to be the highest value allowed.)
> >> Every transaction from the import says "Match Missing."
> >>
> >> Digging around a bit, for the transaction in question, the QFX file
> >> contains the FITID of 202007210003:
> >>
> >> 
> >> CREDIT
> >> 2020072112
> >> 54.00
> >> 202007210003
> >> ACH Electronic Credit
> >> Expenses
> >> 
> >>
> >> My GNUCash file contains, for the same transaction has the online id
> being
> >> 202007210002
> >> 
> >>   
> >> online_id
> >> 202007210002
> >>   
> >>
> >> The online_ID is ...002 instead of ...003.  Changing the QFX file to
> match
> >> the online_id value seemed to work.  Now my question is why would this
> be
> >> different for *lots* of transactions.  Everything worked normally in v3,
> >> but this would not have changed as part of the release.  I'll check a
> few
> >> more problem transactions and see if I can detect a pattern.  Perhaps
> >> Citibank is paying games
> >>
> >> Michael
> >>
> >> On Sun, Jul 26, 2020 at 4:57 PM jean laroche  wrote:
> >>
> >> > To get a match you have to have a transaction in the register that's
> >> > sufficiently similar to the one you're importing, and that has not
> been
> >> > imported/matched before.
> >> > In your case, it could be one of these reasons (I can't see the
> image):
> >> > - There's no matching transaction in your register (no existing
> >> > transaction has amount close, and a date close to the imported one)
> >> > - There's a matching transaction but it's already been matched to an
> >> > imported transaction at some point so it's not available to be matched
> >> > to the new imported one.
> >> > - There's a matching transaction that's available, but the match score
> >> > is below the threshold that allows the transaction to be shown as a
> >> > potential match. Too large a date mismatch can cause that.
> >> >
> >> > Can you check whether you're in one of these 3 cases? If you're in
> case
> >> > 3, you can lower the minimum matching threshold in the preferences and
> >> > see if that helps.
> >> > J.
> >> >
> >> >
> >> > On 7/26/2020 2:44 PM, John Ralls wrote:
> >> > > If there's no matching transaction already in the account then
> there's
> >> > nothing to clear. In that case only adding or not makes sense.
> >> > >
> >> > > Regards,
> >> > > John Ralls
> >> > >
> >> > >
> >> > >> On Jul 26, 2020, at 1:56 PM, Fross, Michael 
> >> wrote:
> >> > >>
> >> > >> Hello all,
> >> > >>
> >> > >> I sent this earlier this month and didn't see any reply so I
> thought
> >> I
> >> > >> would try again.   Has anyone else seen these issues?  I use
> Citibank
> >> > and
> >> > >> perhaps it's a Citibank issue, but I did not have this problem on
> v2
> >> or
> >> > v3.
> >> > >>
> >> > >> Thanks all.  I appreciate the help.
> >> > >>
> >> > >> Michael
> >> > >>
> >> > >>
> >> > >> On Sat, Jul 4, 2020 at 9:48 AM Fross, Michael 
> >> > wrote:
> >> > >>
> >> > >>> Hello all,
> >> > >>>
> >> > >>> I typically download QFX files from my banks every day or two,
> >> import
> >> > them
> >> > >>> to clear them in Gnucash.  Worked great.  However, ever since
> >> > upgrading to
> >> > >>> v4, the importer seems to have trouble matching.  Most of the
> >> imported
> >> > >>> transactions are listed in the importer as (A)dd, but when I
> select
> >> > >>> (C)lear for them it says match missing.
> >> > >>>
> >> > >>> This has occurred for several accounts.  Here is a simple credit
> >> card
> >> > >>> example, although for my checking account, there are dozens like
> >> > this.  The
> >> > >>> top portion shows the register with the Sprint bill cleared.  The
> >> date,
> >> > >>> 

Re: [GNC] Import Transaction Matcher issue in v4

[Fross, Michael]

Got it. Thanks.  Now I understand why it used to work.  And I certainly
wouldn't expect GnuCash to correct a bank's horrible behavior.  I'll keep
an eye on the FITID and see if that's what they are doing.

Thanks everyone for your help.  Much appreciated.

Michael

On Mon, Jul 27, 2020 at 5:13 PM Jean Laroche  wrote:

> Ah, OK I get it now!
> Yes, you should check that from one day to the next, the FTID returned
> by citibank remains the same (for the same transaction). If not, then
> that's going to be a problem.
> What has changed recently with GC is that it not will not match a new
> imported transaction with one from the register that has an online id.
> Previous versions allowed that, so in your case, it's quite possible
> that from one day to the next, the same transaction would be given a
> different ID, but GC matched it to the same register transaction that
> was previously matched.
> So, on day 1, you'd match OFX1->Reg1, and on day two, you'd still match
> OFX2 -> reg1, even though reg1 had already been matched to OFX1 the
> previous day. So you'd see OFX2 appear as "match" and not "Add".
>
> The correct behavior is that OFX2 actually has the same ID as OFX1, and
> it's entirely skipped by GC. But it seems that your bank is messing
> their online ID, and this is what's causing the issue for you, along
> with the new GC behavior.
>
> I'm not sure how to fix your problem. It's clearly a bank issue but
> that's not terribly helpful as far as your problem goes. We should
> probably add an option to defeat the new GC import behavior,
> specifically for users like you whose bank send unreliable FTIDs/
>
> FYI, the new GC behavior was added because the old behavior was causing
> issues with people who had recurring daily transactions: imagine buying
> a cup of coffee every day: in that case GC will match all daily cups of
> coffee to the same register transaction (within a date range, of course)
> instead of adding the new one every day (which really is the correct way
> to do it).
>
>
>
> Jean
>
>
> On 7/27/20 2:45 PM, Fross, Michael wrote:
> > Hi Jean.  Let me try to better explain my issue.
> >
> > On most days, I download QFX files from my Checking and Credit Card
> > accounts and import them.  In this way I clear transactions and I can
> > see what's going on.  The next day I'll do so again, and the importer
> > skips those that have already been matched per your comment.s  I've been
> > doing this for years and it's worked well.
> >
> > Since I updated to V4 the transactions that would have matched
> > previously now come up with "Match Missing!" in the importer.  I don't
> > want to add them as they would create duplicates.  Transactions that are
> > not cleared seem to be able to be matched.  But those that are
> > previously cleared get the "Match Missing!" issue.
> >
> > Now, based on what you told me, the QFX file for a problem transaction
> > has a FITID that is one higher than the ONLINE_ID in the gnucash file
> > for the same transaction.  So the Match Missing items seems to be
> correct.
> >
> > If I duplicate the transaction, delete the original, it matches which I
> > assume is because there is not an ONLINE_ID associated with the new one.
> >
> > What I don't understand is if there is an issue with GNUCash, which is
> > why I asked if others are having the same issue, or if it is/was an
> > issue with CITIBANK QFX files.  If they changed the FITID for some
> > reason that would cause this.  Since no one else has said they are
> > having the issue, I'm assuming this is not a GNUCash issue.
> >
> > Does that better explain the issue?
> >
> > I really appreciate the time to respond and help me through this.
> >
> > Michael
> >
> >
> >
> >
> >
> >
> > On Mon, Jul 27, 2020 at 3:50 PM Jean Laroche  > > wrote:
> >
> > There's a misunderstanding here.
> > This is what GC does:
> > - It looks at the OFX transaction, which always has an FITID.
> > - IF GC sees this FITID in one of the register transactions, it skips
> > the OFX transaction, because it's already been imported (when a
> > transaction gets imported from the ofx, the online_id is copied into
> > the
> > register transaction, that's how GC remembers it's already imported
> the
> > transaction).
> > - If the FITID is not found in the register, GC assumes that this is
> a
> > new transaction and attempts to match it with existing transactions
> in
> > your register. BUT:
> >  . It only looks at register transactions that have NO FITID
> (i.e.,
> > register transactions that you entered manually, typically)
> >  . Among those, it looks for transactions that match pretty
> closely
> > for the date and the amount.
> >  . It picks the one with the highest matching score, if that
> > score is
> > above a user-adjustable threshold.
> >
> > So in the case you outline below, if the register transaction has the
> > same 

Re: [GNC] Import Transaction Matcher issue in v4

[Jean Laroche]

Ah, OK I get it now!
Yes, you should check that from one day to the next, the FTID returned 
by citibank remains the same (for the same transaction). If not, then 
that's going to be a problem.
What has changed recently with GC is that it not will not match a new 
imported transaction with one from the register that has an online id. 
Previous versions allowed that, so in your case, it's quite possible 
that from one day to the next, the same transaction would be given a 
different ID, but GC matched it to the same register transaction that 
was previously matched.
So, on day 1, you'd match OFX1->Reg1, and on day two, you'd still match 
OFX2 -> reg1, even though reg1 had already been matched to OFX1 the 
previous day. So you'd see OFX2 appear as "match" and not "Add".


The correct behavior is that OFX2 actually has the same ID as OFX1, and 
it's entirely skipped by GC. But it seems that your bank is messing 
their online ID, and this is what's causing the issue for you, along 
with the new GC behavior.


I'm not sure how to fix your problem. It's clearly a bank issue but 
that's not terribly helpful as far as your problem goes. We should 
probably add an option to defeat the new GC import behavior, 
specifically for users like you whose bank send unreliable FTIDs/


FYI, the new GC behavior was added because the old behavior was causing 
issues with people who had recurring daily transactions: imagine buying 
a cup of coffee every day: in that case GC will match all daily cups of 
coffee to the same register transaction (within a date range, of course) 
instead of adding the new one every day (which really is the correct way 
to do it).




Jean


On 7/27/20 2:45 PM, Fross, Michael wrote:

Hi Jean.  Let me try to better explain my issue.

On most days, I download QFX files from my Checking and Credit Card 
accounts and import them.  In this way I clear transactions and I can 
see what's going on.  The next day I'll do so again, and the importer 
skips those that have already been matched per your comment.s  I've been 
doing this for years and it's worked well.


Since I updated to V4 the transactions that would have matched 
previously now come up with "Match Missing!" in the importer.  I don't 
want to add them as they would create duplicates.  Transactions that are 
not cleared seem to be able to be matched.  But those that are 
previously cleared get the "Match Missing!" issue.


Now, based on what you told me, the QFX file for a problem transaction 
has a FITID that is one higher than the ONLINE_ID in the gnucash file 
for the same transaction.  So the Match Missing items seems to be correct.


If I duplicate the transaction, delete the original, it matches which I 
assume is because there is not an ONLINE_ID associated with the new one.


What I don't understand is if there is an issue with GNUCash, which is 
why I asked if others are having the same issue, or if it is/was an 
issue with CITIBANK QFX files.  If they changed the FITID for some 
reason that would cause this.  Since no one else has said they are 
having the issue, I'm assuming this is not a GNUCash issue.


Does that better explain the issue?

I really appreciate the time to respond and help me through this.

Michael






On Mon, Jul 27, 2020 at 3:50 PM Jean Laroche > wrote:


There's a misunderstanding here.
This is what GC does:
- It looks at the OFX transaction, which always has an FITID.
- IF GC sees this FITID in one of the register transactions, it skips
the OFX transaction, because it's already been imported (when a
transaction gets imported from the ofx, the online_id is copied into
the
register transaction, that's how GC remembers it's already imported the
transaction).
- If the FITID is not found in the register, GC assumes that this is a
new transaction and attempts to match it with existing transactions in
your register. BUT:
     . It only looks at register transactions that have NO FITID (i.e.,
register transactions that you entered manually, typically)
     . Among those, it looks for transactions that match pretty closely
for the date and the amount.
     . It picks the one with the highest matching score, if that
score is
above a user-adjustable threshold.

So in the case you outline below, if the register transaction has the
same FITID as your ofx transaction, the ofx transaction will NOT be
imported (it will be just skipped).

I think I don't quite understand the problem you're having. Is it that
you're importing OFX transaction and they're not matching register
transactions that you entered manually?
Or is it that they're being imported despite the fact they were
imported
previously?

J.

On 7/27/20 1:31 PM, Fross, Michael wrote:
 > Thanks Jean / John for your thoughts.  There is a register entry
that
 > matches, IMHO, very closely.   I increased the Match 

Re: [GNC] Import Transaction Matcher issue in v4

[John Ralls]

No. Section 3.2.3 of the OFX 1.6 spec:
"An FI (or its Service Provider) assigns an  to uniquely identify a 
financial transaction
that can appear in an account statement. Its primary purpose is to allow a 
client to detect
duplicate responses. Open Financial Exchange intends  for use in 
statement download
applications, where every transaction (not just those that are 
client-originated or serveroriginated)
requires a unique ID."

Regards,
John Ralls



> On Jul 27, 2020, at 2:20 PM, Christopher Lam  
> wrote:
> 
> As I understand the ofx spec*, the fitid should be an invariant for the
> bank and account. By any chance do you have older qfx files to compare?
> 
> * https://www.ofx.net/downloads.html
> 
> On Tue, 28 Jul 2020, 4:34 am Fross, Michael,  wrote:
> 
>> Thanks Jean / John for your thoughts.  There is a register entry that
>> matches, IMHO, very closely.   I increased the Match Display Threshold from
>> 1 to 3, and then to 6 (which appears to be the highest value allowed.)
>> Every transaction from the import says "Match Missing."
>> 
>> Digging around a bit, for the transaction in question, the QFX file
>> contains the FITID of 202007210003:
>> 
>> 
>> CREDIT
>> 2020072112
>> 54.00
>> 202007210003
>> ACH Electronic Credit
>> Expenses
>> 
>> 
>> My GNUCash file contains, for the same transaction has the online id being
>> 202007210002
>>
>>  
>>online_id
>>202007210002
>>  
>> 
>> The online_ID is ...002 instead of ...003.  Changing the QFX file to match
>> the online_id value seemed to work.  Now my question is why would this be
>> different for *lots* of transactions.  Everything worked normally in v3,
>> but this would not have changed as part of the release.  I'll check a few
>> more problem transactions and see if I can detect a pattern.  Perhaps
>> Citibank is paying games
>> 
>> Michael
>> 
>> On Sun, Jul 26, 2020 at 4:57 PM jean laroche  wrote:
>> 
>>> To get a match you have to have a transaction in the register that's
>>> sufficiently similar to the one you're importing, and that has not been
>>> imported/matched before.
>>> In your case, it could be one of these reasons (I can't see the image):
>>> - There's no matching transaction in your register (no existing
>>> transaction has amount close, and a date close to the imported one)
>>> - There's a matching transaction but it's already been matched to an
>>> imported transaction at some point so it's not available to be matched
>>> to the new imported one.
>>> - There's a matching transaction that's available, but the match score
>>> is below the threshold that allows the transaction to be shown as a
>>> potential match. Too large a date mismatch can cause that.
>>> 
>>> Can you check whether you're in one of these 3 cases? If you're in case
>>> 3, you can lower the minimum matching threshold in the preferences and
>>> see if that helps.
>>> J.
>>> 
>>> 
>>> On 7/26/2020 2:44 PM, John Ralls wrote:
 If there's no matching transaction already in the account then there's
>>> nothing to clear. In that case only adding or not makes sense.
 
 Regards,
 John Ralls
 
 
> On Jul 26, 2020, at 1:56 PM, Fross, Michael 
>> wrote:
> 
> Hello all,
> 
> I sent this earlier this month and didn't see any reply so I thought I
> would try again.   Has anyone else seen these issues?  I use Citibank
>>> and
> perhaps it's a Citibank issue, but I did not have this problem on v2
>> or
>>> v3.
> 
> Thanks all.  I appreciate the help.
> 
> Michael
> 
> 
> On Sat, Jul 4, 2020 at 9:48 AM Fross, Michael 
>>> wrote:
> 
>> Hello all,
>> 
>> I typically download QFX files from my banks every day or two, import
>>> them
>> to clear them in Gnucash.  Worked great.  However, ever since
>>> upgrading to
>> v4, the importer seems to have trouble matching.  Most of the
>> imported
>> transactions are listed in the importer as (A)dd, but when I select
>> (C)lear for them it says match missing.
>> 
>> This has occurred for several accounts.  Here is a simple credit card
>> example, although for my checking account, there are dozens like
>>> this.  The
>> top portion shows the register with the Sprint bill cleared.  The
>> date,
>> amount, and name (mostly) match.
>> 
>> [image: image.png]
>> 
>> Not sure if there is just something wrong with my setup or not.
>>> Perhaps a
>> bug?  Are others experiencing this?  Any ideas to get the matcher
>>> matching
>> again?  Something need to get cleared out?
>> 
>> For those of us in the US, happy Independence Day.  Thank you all for
>>> your
>> assistance.
>> 
>> Michael
>> 
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> 

Re: [GNC] Import Transaction Matcher issue in v4

[Fross, Michael]

Hi Christopher.  Alas I do not keep the QFX files.  I'll hang on to them
for now and see if the FIDID does change.

On Mon, Jul 27, 2020 at 4:20 PM Christopher Lam 
wrote:

> As I understand the ofx spec*, the fitid should be an invariant for the
> bank and account. By any chance do you have older qfx files to compare?
>
> * https://www.ofx.net/downloads.html
>
> On Tue, 28 Jul 2020, 4:34 am Fross, Michael,  wrote:
>
>> Thanks Jean / John for your thoughts.  There is a register entry that
>> matches, IMHO, very closely.   I increased the Match Display Threshold
>> from
>> 1 to 3, and then to 6 (which appears to be the highest value allowed.)
>> Every transaction from the import says "Match Missing."
>>
>> Digging around a bit, for the transaction in question, the QFX file
>> contains the FITID of 202007210003:
>>
>> 
>> CREDIT
>> 2020072112
>> 54.00
>> 202007210003
>> ACH Electronic Credit
>> Expenses
>> 
>>
>> My GNUCash file contains, for the same transaction has the online id being
>> 202007210002
>> 
>>   
>> online_id
>> 202007210002
>>   
>>
>> The online_ID is ...002 instead of ...003.  Changing the QFX file to match
>> the online_id value seemed to work.  Now my question is why would this be
>> different for *lots* of transactions.  Everything worked normally in v3,
>> but this would not have changed as part of the release.  I'll check a few
>> more problem transactions and see if I can detect a pattern.  Perhaps
>> Citibank is paying games
>>
>> Michael
>>
>> On Sun, Jul 26, 2020 at 4:57 PM jean laroche  wrote:
>>
>> > To get a match you have to have a transaction in the register that's
>> > sufficiently similar to the one you're importing, and that has not been
>> > imported/matched before.
>> > In your case, it could be one of these reasons (I can't see the image):
>> > - There's no matching transaction in your register (no existing
>> > transaction has amount close, and a date close to the imported one)
>> > - There's a matching transaction but it's already been matched to an
>> > imported transaction at some point so it's not available to be matched
>> > to the new imported one.
>> > - There's a matching transaction that's available, but the match score
>> > is below the threshold that allows the transaction to be shown as a
>> > potential match. Too large a date mismatch can cause that.
>> >
>> > Can you check whether you're in one of these 3 cases? If you're in case
>> > 3, you can lower the minimum matching threshold in the preferences and
>> > see if that helps.
>> > J.
>> >
>> >
>> > On 7/26/2020 2:44 PM, John Ralls wrote:
>> > > If there's no matching transaction already in the account then there's
>> > nothing to clear. In that case only adding or not makes sense.
>> > >
>> > > Regards,
>> > > John Ralls
>> > >
>> > >
>> > >> On Jul 26, 2020, at 1:56 PM, Fross, Michael 
>> wrote:
>> > >>
>> > >> Hello all,
>> > >>
>> > >> I sent this earlier this month and didn't see any reply so I thought
>> I
>> > >> would try again.   Has anyone else seen these issues?  I use Citibank
>> > and
>> > >> perhaps it's a Citibank issue, but I did not have this problem on v2
>> or
>> > v3.
>> > >>
>> > >> Thanks all.  I appreciate the help.
>> > >>
>> > >> Michael
>> > >>
>> > >>
>> > >> On Sat, Jul 4, 2020 at 9:48 AM Fross, Michael 
>> > wrote:
>> > >>
>> > >>> Hello all,
>> > >>>
>> > >>> I typically download QFX files from my banks every day or two,
>> import
>> > them
>> > >>> to clear them in Gnucash.  Worked great.  However, ever since
>> > upgrading to
>> > >>> v4, the importer seems to have trouble matching.  Most of the
>> imported
>> > >>> transactions are listed in the importer as (A)dd, but when I select
>> > >>> (C)lear for them it says match missing.
>> > >>>
>> > >>> This has occurred for several accounts.  Here is a simple credit
>> card
>> > >>> example, although for my checking account, there are dozens like
>> > this.  The
>> > >>> top portion shows the register with the Sprint bill cleared.  The
>> date,
>> > >>> amount, and name (mostly) match.
>> > >>>
>> > >>> [image: image.png]
>> > >>>
>> > >>> Not sure if there is just something wrong with my setup or not.
>> > Perhaps a
>> > >>> bug?  Are others experiencing this?  Any ideas to get the matcher
>> > matching
>> > >>> again?  Something need to get cleared out?
>> > >>>
>> > >>> For those of us in the US, happy Independence Day.  Thank you all
>> for
>> > your
>> > >>> assistance.
>> > >>>
>> > >>> Michael
>> > >>>
>> > >> ___
>> > >> gnucash-user mailing list
>> > >> gnucash-user@gnucash.org
>> > >> To update your subscription preferences or to unsubscribe:
>> > >> https://lists.gnucash.org/mailman/listinfo/gnucash-user
>> > >> If you are using Nabble or Gmane, please see
>> > https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
>> > >> -
>> > >> Please remember to CC this list on all your replies.
>> > >> You can 

Re: [GNC] Import Transaction Matcher issue in v4

[Fross, Michael]

Hi Jean.  Let me try to better explain my issue.

On most days, I download QFX files from my Checking and Credit Card
accounts and import them.  In this way I clear transactions and I can see
what's going on.  The next day I'll do so again, and the importer skips
those that have already been matched per your comment.s  I've been doing
this for years and it's worked well.

Since I updated to V4 the transactions that would have matched previously
now come up with "Match Missing!" in the importer.  I don't want to add
them as they would create duplicates.  Transactions that are not cleared
seem to be able to be matched.  But those that are previously cleared get
the "Match Missing!" issue.

Now, based on what you told me, the QFX file for a problem transaction has
a FITID that is one higher than the ONLINE_ID in the gnucash file for the
same transaction.  So the Match Missing items seems to be correct.

If I duplicate the transaction, delete the original, it matches which I
assume is because there is not an ONLINE_ID associated with the new one.

What I don't understand is if there is an issue with GNUCash, which is why
I asked if others are having the same issue, or if it is/was an issue with
CITIBANK QFX files.  If they changed the FITID for some reason that would
cause this.  Since no one else has said they are having the issue, I'm
assuming this is not a GNUCash issue.

Does that better explain the issue?

I really appreciate the time to respond and help me through this.

Michael






On Mon, Jul 27, 2020 at 3:50 PM Jean Laroche  wrote:

> There's a misunderstanding here.
> This is what GC does:
> - It looks at the OFX transaction, which always has an FITID.
> - IF GC sees this FITID in one of the register transactions, it skips
> the OFX transaction, because it's already been imported (when a
> transaction gets imported from the ofx, the online_id is copied into the
> register transaction, that's how GC remembers it's already imported the
> transaction).
> - If the FITID is not found in the register, GC assumes that this is a
> new transaction and attempts to match it with existing transactions in
> your register. BUT:
> . It only looks at register transactions that have NO FITID (i.e.,
> register transactions that you entered manually, typically)
> . Among those, it looks for transactions that match pretty closely
> for the date and the amount.
> . It picks the one with the highest matching score, if that score is
> above a user-adjustable threshold.
>
> So in the case you outline below, if the register transaction has the
> same FITID as your ofx transaction, the ofx transaction will NOT be
> imported (it will be just skipped).
>
> I think I don't quite understand the problem you're having. Is it that
> you're importing OFX transaction and they're not matching register
> transactions that you entered manually?
> Or is it that they're being imported despite the fact they were imported
> previously?
>
> J.
>
> On 7/27/20 1:31 PM, Fross, Michael wrote:
> > Thanks Jean / John for your thoughts.  There is a register entry that
> > matches, IMHO, very closely.   I increased the Match Display Threshold
> > from 1 to 3, and then to 6 (which appears to be the highest value
> > allowed.)  Every transaction from the import says "Match Missing."
> >
> > Digging around a bit, for the transaction in question, the QFX file
> > contains the FITID of 202007210003:
> >
> > 
> > CREDIT
> > 2020072112
> > 54.00
> > 202007210003
> > ACH Electronic Credit
> > Expenses
> > 
> >
> > My GNUCash file contains, for the same transaction has the online id
> > being 202007210002
> >  
> >
> >  online_id
> >  202007210002
> >
> >
> > The online_ID is ...002 instead of ...003.  Changing the QFX file to
> > match the online_id value seemed to work.  Now my question is why would
> > this be different for *lots* of transactions.  Everything worked
> > normally in v3, but this would not have changed as part of the release.
> > I'll check a few more problem transactions and see if I can detect a
> > pattern.  Perhaps Citibank is paying games
> >
> > Michael
> >
> > On Sun, Jul 26, 2020 at 4:57 PM jean laroche  > > wrote:
> >
> > To get a match you have to have a transaction in the register that's
> > sufficiently similar to the one you're importing, and that has not
> been
> > imported/matched before.
> > In your case, it could be one of these reasons (I can't see the
> image):
> > - There's no matching transaction in your register (no existing
> > transaction has amount close, and a date close to the imported one)
> > - There's a matching transaction but it's already been matched to an
> > imported transaction at some point so it's not available to be
> matched
> > to the new imported one.
> > - There's a matching transaction that's available, but the match
> score
> > is below the threshold 

Re: [GNC] Import Transaction Matcher issue in v4

[Christopher Lam]

As I understand the ofx spec*, the fitid should be an invariant for the
bank and account. By any chance do you have older qfx files to compare?

* https://www.ofx.net/downloads.html

On Tue, 28 Jul 2020, 4:34 am Fross, Michael,  wrote:

> Thanks Jean / John for your thoughts.  There is a register entry that
> matches, IMHO, very closely.   I increased the Match Display Threshold from
> 1 to 3, and then to 6 (which appears to be the highest value allowed.)
> Every transaction from the import says "Match Missing."
>
> Digging around a bit, for the transaction in question, the QFX file
> contains the FITID of 202007210003:
>
> 
> CREDIT
> 2020072112
> 54.00
> 202007210003
> ACH Electronic Credit
> Expenses
> 
>
> My GNUCash file contains, for the same transaction has the online id being
> 202007210002
> 
>   
> online_id
> 202007210002
>   
>
> The online_ID is ...002 instead of ...003.  Changing the QFX file to match
> the online_id value seemed to work.  Now my question is why would this be
> different for *lots* of transactions.  Everything worked normally in v3,
> but this would not have changed as part of the release.  I'll check a few
> more problem transactions and see if I can detect a pattern.  Perhaps
> Citibank is paying games
>
> Michael
>
> On Sun, Jul 26, 2020 at 4:57 PM jean laroche  wrote:
>
> > To get a match you have to have a transaction in the register that's
> > sufficiently similar to the one you're importing, and that has not been
> > imported/matched before.
> > In your case, it could be one of these reasons (I can't see the image):
> > - There's no matching transaction in your register (no existing
> > transaction has amount close, and a date close to the imported one)
> > - There's a matching transaction but it's already been matched to an
> > imported transaction at some point so it's not available to be matched
> > to the new imported one.
> > - There's a matching transaction that's available, but the match score
> > is below the threshold that allows the transaction to be shown as a
> > potential match. Too large a date mismatch can cause that.
> >
> > Can you check whether you're in one of these 3 cases? If you're in case
> > 3, you can lower the minimum matching threshold in the preferences and
> > see if that helps.
> > J.
> >
> >
> > On 7/26/2020 2:44 PM, John Ralls wrote:
> > > If there's no matching transaction already in the account then there's
> > nothing to clear. In that case only adding or not makes sense.
> > >
> > > Regards,
> > > John Ralls
> > >
> > >
> > >> On Jul 26, 2020, at 1:56 PM, Fross, Michael 
> wrote:
> > >>
> > >> Hello all,
> > >>
> > >> I sent this earlier this month and didn't see any reply so I thought I
> > >> would try again.   Has anyone else seen these issues?  I use Citibank
> > and
> > >> perhaps it's a Citibank issue, but I did not have this problem on v2
> or
> > v3.
> > >>
> > >> Thanks all.  I appreciate the help.
> > >>
> > >> Michael
> > >>
> > >>
> > >> On Sat, Jul 4, 2020 at 9:48 AM Fross, Michael 
> > wrote:
> > >>
> > >>> Hello all,
> > >>>
> > >>> I typically download QFX files from my banks every day or two, import
> > them
> > >>> to clear them in Gnucash.  Worked great.  However, ever since
> > upgrading to
> > >>> v4, the importer seems to have trouble matching.  Most of the
> imported
> > >>> transactions are listed in the importer as (A)dd, but when I select
> > >>> (C)lear for them it says match missing.
> > >>>
> > >>> This has occurred for several accounts.  Here is a simple credit card
> > >>> example, although for my checking account, there are dozens like
> > this.  The
> > >>> top portion shows the register with the Sprint bill cleared.  The
> date,
> > >>> amount, and name (mostly) match.
> > >>>
> > >>> [image: image.png]
> > >>>
> > >>> Not sure if there is just something wrong with my setup or not.
> > Perhaps a
> > >>> bug?  Are others experiencing this?  Any ideas to get the matcher
> > matching
> > >>> again?  Something need to get cleared out?
> > >>>
> > >>> For those of us in the US, happy Independence Day.  Thank you all for
> > your
> > >>> assistance.
> > >>>
> > >>> Michael
> > >>>
> > >> ___
> > >> gnucash-user mailing list
> > >> gnucash-user@gnucash.org
> > >> To update your subscription preferences or to unsubscribe:
> > >> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> > >> If you are using Nabble or Gmane, please see
> > https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> > >> -
> > >> Please remember to CC this list on all your replies.
> > >> You can do this by using Reply-To-List or Reply-All.
> > > ___
> > > gnucash-user mailing list
> > > gnucash-user@gnucash.org
> > > To update your subscription preferences or to unsubscribe:
> > > https://lists.gnucash.org/mailman/listinfo/gnucash-user
> > > If you are using Nabble or 

Re: [GNC] Import Transaction Matcher issue in v4

[Jean Laroche]

There's a misunderstanding here.
This is what GC does:
- It looks at the OFX transaction, which always has an FITID.
- IF GC sees this FITID in one of the register transactions, it skips 
the OFX transaction, because it's already been imported (when a 
transaction gets imported from the ofx, the online_id is copied into the 
register transaction, that's how GC remembers it's already imported the 
transaction).
- If the FITID is not found in the register, GC assumes that this is a 
new transaction and attempts to match it with existing transactions in 
your register. BUT:
   . It only looks at register transactions that have NO FITID (i.e., 
register transactions that you entered manually, typically)
   . Among those, it looks for transactions that match pretty closely 
for the date and the amount.
   . It picks the one with the highest matching score, if that score is 
above a user-adjustable threshold.


So in the case you outline below, if the register transaction has the 
same FITID as your ofx transaction, the ofx transaction will NOT be 
imported (it will be just skipped).


I think I don't quite understand the problem you're having. Is it that 
you're importing OFX transaction and they're not matching register 
transactions that you entered manually?
Or is it that they're being imported despite the fact they were imported 
previously?


J.

On 7/27/20 1:31 PM, Fross, Michael wrote:
Thanks Jean / John for your thoughts.  There is a register entry that 
matches, IMHO, very closely.   I increased the Match Display Threshold 
from 1 to 3, and then to 6 (which appears to be the highest value 
allowed.)  Every transaction from the import says "Match Missing."


Digging around a bit, for the transaction in question, the QFX file 
contains the FITID of 202007210003:



CREDIT
2020072112
54.00
202007210003
ACH Electronic Credit
Expenses


My GNUCash file contains, for the same transaction has the online id 
being 202007210002

         
           
             online_id
             202007210002
           

The online_ID is ...002 instead of ...003.  Changing the QFX file to 
match the online_id value seemed to work.  Now my question is why would 
this be different for *lots* of transactions.  Everything worked 
normally in v3, but this would not have changed as part of the release.  
I'll check a few more problem transactions and see if I can detect a 
pattern.  Perhaps Citibank is paying games


Michael

On Sun, Jul 26, 2020 at 4:57 PM jean laroche > wrote:


To get a match you have to have a transaction in the register that's
sufficiently similar to the one you're importing, and that has not been
imported/matched before.
In your case, it could be one of these reasons (I can't see the image):
- There's no matching transaction in your register (no existing
transaction has amount close, and a date close to the imported one)
- There's a matching transaction but it's already been matched to an
imported transaction at some point so it's not available to be matched
to the new imported one.
- There's a matching transaction that's available, but the match score
is below the threshold that allows the transaction to be shown as a
potential match. Too large a date mismatch can cause that.

Can you check whether you're in one of these 3 cases? If you're in case
3, you can lower the minimum matching threshold in the preferences and
see if that helps.
J.


On 7/26/2020 2:44 PM, John Ralls wrote:
 > If there's no matching transaction already in the account then
there's nothing to clear. In that case only adding or not makes sense.
 >
 > Regards,
 > John Ralls
 >
 >
 >> On Jul 26, 2020, at 1:56 PM, Fross, Michael mailto:mich...@fross.org>> wrote:
 >>
 >> Hello all,
 >>
 >> I sent this earlier this month and didn't see any reply so I
thought I
 >> would try again.   Has anyone else seen these issues?  I use
Citibank and
 >> perhaps it's a Citibank issue, but I did not have this problem
on v2 or v3.
 >>
 >> Thanks all.  I appreciate the help.
 >>
 >> Michael
 >>
 >>
 >> On Sat, Jul 4, 2020 at 9:48 AM Fross, Michael mailto:mich...@fross.org>> wrote:
 >>
 >>> Hello all,
 >>>
 >>> I typically download QFX files from my banks every day or two,
import them
 >>> to clear them in Gnucash.  Worked great.  However, ever since
upgrading to
 >>> v4, the importer seems to have trouble matching.  Most of the
imported
 >>> transactions are listed in the importer as (A)dd, but when I select
 >>> (C)lear for them it says match missing.
 >>>
 >>> This has occurred for several accounts.  Here is a simple
credit card
 >>> example, although for my checking account, there are dozens
like this.  The
 >>> top portion shows the register with the Sprint bill cleared. 
The date,

   

Re: [GNC] Import Transaction Matcher issue in v4

[Fross, Michael]

Thanks Jean / John for your thoughts.  There is a register entry that
matches, IMHO, very closely.   I increased the Match Display Threshold from
1 to 3, and then to 6 (which appears to be the highest value allowed.)
Every transaction from the import says "Match Missing."

Digging around a bit, for the transaction in question, the QFX file
contains the FITID of 202007210003:


CREDIT
2020072112
54.00
202007210003
ACH Electronic Credit
Expenses


My GNUCash file contains, for the same transaction has the online id being
202007210002

  
online_id
202007210002
  

The online_ID is ...002 instead of ...003.  Changing the QFX file to match
the online_id value seemed to work.  Now my question is why would this be
different for *lots* of transactions.  Everything worked normally in v3,
but this would not have changed as part of the release.  I'll check a few
more problem transactions and see if I can detect a pattern.  Perhaps
Citibank is paying games

Michael

On Sun, Jul 26, 2020 at 4:57 PM jean laroche  wrote:

> To get a match you have to have a transaction in the register that's
> sufficiently similar to the one you're importing, and that has not been
> imported/matched before.
> In your case, it could be one of these reasons (I can't see the image):
> - There's no matching transaction in your register (no existing
> transaction has amount close, and a date close to the imported one)
> - There's a matching transaction but it's already been matched to an
> imported transaction at some point so it's not available to be matched
> to the new imported one.
> - There's a matching transaction that's available, but the match score
> is below the threshold that allows the transaction to be shown as a
> potential match. Too large a date mismatch can cause that.
>
> Can you check whether you're in one of these 3 cases? If you're in case
> 3, you can lower the minimum matching threshold in the preferences and
> see if that helps.
> J.
>
>
> On 7/26/2020 2:44 PM, John Ralls wrote:
> > If there's no matching transaction already in the account then there's
> nothing to clear. In that case only adding or not makes sense.
> >
> > Regards,
> > John Ralls
> >
> >
> >> On Jul 26, 2020, at 1:56 PM, Fross, Michael  wrote:
> >>
> >> Hello all,
> >>
> >> I sent this earlier this month and didn't see any reply so I thought I
> >> would try again.   Has anyone else seen these issues?  I use Citibank
> and
> >> perhaps it's a Citibank issue, but I did not have this problem on v2 or
> v3.
> >>
> >> Thanks all.  I appreciate the help.
> >>
> >> Michael
> >>
> >>
> >> On Sat, Jul 4, 2020 at 9:48 AM Fross, Michael 
> wrote:
> >>
> >>> Hello all,
> >>>
> >>> I typically download QFX files from my banks every day or two, import
> them
> >>> to clear them in Gnucash.  Worked great.  However, ever since
> upgrading to
> >>> v4, the importer seems to have trouble matching.  Most of the imported
> >>> transactions are listed in the importer as (A)dd, but when I select
> >>> (C)lear for them it says match missing.
> >>>
> >>> This has occurred for several accounts.  Here is a simple credit card
> >>> example, although for my checking account, there are dozens like
> this.  The
> >>> top portion shows the register with the Sprint bill cleared.  The date,
> >>> amount, and name (mostly) match.
> >>>
> >>> [image: image.png]
> >>>
> >>> Not sure if there is just something wrong with my setup or not.
> Perhaps a
> >>> bug?  Are others experiencing this?  Any ideas to get the matcher
> matching
> >>> again?  Something need to get cleared out?
> >>>
> >>> For those of us in the US, happy Independence Day.  Thank you all for
> your
> >>> assistance.
> >>>
> >>> Michael
> >>>
> >> ___
> >> gnucash-user mailing list
> >> gnucash-user@gnucash.org
> >> To update your subscription preferences or to unsubscribe:
> >> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> >> If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> >> -
> >> Please remember to CC this list on all your replies.
> >> You can do this by using Reply-To-List or Reply-All.
> > ___
> > gnucash-user mailing list
> > gnucash-user@gnucash.org
> > To update your subscription preferences or to unsubscribe:
> > https://lists.gnucash.org/mailman/listinfo/gnucash-user
> > If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> > -
> > Please remember to CC this list on all your replies.
> > You can do this by using Reply-To-List or Reply-All.
>
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see
> 

Re: [GNC] Import Transaction Matcher issue in v4

[Adrien Monteleone]

Ah, forgot about the ID, thanks.

Regards,
Adrien

On 7/27/20 12:31 PM, Jean Laroche wrote:

No,
 >> - There's a matching transaction but it's already been matched to an
 >> imported transaction at some point so it's not available to be matched
 >> to the new imported one.

This means, if you re-import the same file, the register transactions 
that were previously imported are marked as having been matched/imported 
(they have an online-id), and they're not available to be matched.
But more importantly, the ofx transaction is recognized as having been 
imported before because its online id is already in the register so the 
ofx transaction is skipped entirely.

Jean


___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Import Transaction Matcher issue in v4

[Jean Laroche]

No,
>> - There's a matching transaction but it's already been matched to an
>> imported transaction at some point so it's not available to be matched
>> to the new imported one.

This means, if you re-import the same file, the register transactions 
that were previously imported are marked as having been matched/imported 
(they have an online-id), and they're not available to be matched.
But more importantly, the ofx transaction is recognized as having been 
imported before because its online id is already in the register so the 
ofx transaction is skipped entirely.

Jean


On 7/27/20 9:52 AM, Adrien Monteleone wrote:
This appears to mean that if a subsequent import file contained a 
duplicate transaction for some reason, (or the user accidentally 
selected an already imported file) then duplicates would be imported. Or 
am I reading that wrong?


Regards,
Adrien

On 7/26/20 4:56 PM, jean laroche wrote:

- There's a matching transaction but it's already been matched to an 
imported transaction at some point so it's not available to be matched 
to the new imported one.


___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.

-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Import Transaction Matcher issue in v4

[Adrien Monteleone]

This appears to mean that if a subsequent import file contained a 
duplicate transaction for some reason, (or the user accidentally 
selected an already imported file) then duplicates would be imported. Or 
am I reading that wrong?


Regards,
Adrien

On 7/26/20 4:56 PM, jean laroche wrote:

- There's a matching transaction but it's already been matched to an 
imported transaction at some point so it's not available to be matched 
to the new imported one.


___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Import Transaction Matcher issue in v4

[jean laroche]

To get a match you have to have a transaction in the register that's 
sufficiently similar to the one you're importing, and that has not been 
imported/matched before.

In your case, it could be one of these reasons (I can't see the image):
- There's no matching transaction in your register (no existing 
transaction has amount close, and a date close to the imported one)
- There's a matching transaction but it's already been matched to an 
imported transaction at some point so it's not available to be matched 
to the new imported one.
- There's a matching transaction that's available, but the match score 
is below the threshold that allows the transaction to be shown as a 
potential match. Too large a date mismatch can cause that.


Can you check whether you're in one of these 3 cases? If you're in case 
3, you can lower the minimum matching threshold in the preferences and 
see if that helps.

J.


On 7/26/2020 2:44 PM, John Ralls wrote:

If there's no matching transaction already in the account then there's nothing 
to clear. In that case only adding or not makes sense.

Regards,
John Ralls



On Jul 26, 2020, at 1:56 PM, Fross, Michael  wrote:

Hello all,

I sent this earlier this month and didn't see any reply so I thought I
would try again.   Has anyone else seen these issues?  I use Citibank and
perhaps it's a Citibank issue, but I did not have this problem on v2 or v3.

Thanks all.  I appreciate the help.

Michael


On Sat, Jul 4, 2020 at 9:48 AM Fross, Michael  wrote:


Hello all,

I typically download QFX files from my banks every day or two, import them
to clear them in Gnucash.  Worked great.  However, ever since upgrading to
v4, the importer seems to have trouble matching.  Most of the imported
transactions are listed in the importer as (A)dd, but when I select
(C)lear for them it says match missing.

This has occurred for several accounts.  Here is a simple credit card
example, although for my checking account, there are dozens like this.  The
top portion shows the register with the Sprint bill cleared.  The date,
amount, and name (mostly) match.

[image: image.png]

Not sure if there is just something wrong with my setup or not.  Perhaps a
bug?  Are others experiencing this?  Any ideas to get the matcher matching
again?  Something need to get cleared out?

For those of us in the US, happy Independence Day.  Thank you all for your
assistance.

Michael


___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.


___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Import Transaction Matcher issue in v4

[John Ralls]

If there's no matching transaction already in the account then there's nothing 
to clear. In that case only adding or not makes sense.

Regards,
John Ralls


> On Jul 26, 2020, at 1:56 PM, Fross, Michael  wrote:
> 
> Hello all,
> 
> I sent this earlier this month and didn't see any reply so I thought I
> would try again.   Has anyone else seen these issues?  I use Citibank and
> perhaps it's a Citibank issue, but I did not have this problem on v2 or v3.
> 
> Thanks all.  I appreciate the help.
> 
> Michael
> 
> 
> On Sat, Jul 4, 2020 at 9:48 AM Fross, Michael  wrote:
> 
>> Hello all,
>> 
>> I typically download QFX files from my banks every day or two, import them
>> to clear them in Gnucash.  Worked great.  However, ever since upgrading to
>> v4, the importer seems to have trouble matching.  Most of the imported
>> transactions are listed in the importer as (A)dd, but when I select
>> (C)lear for them it says match missing.
>> 
>> This has occurred for several accounts.  Here is a simple credit card
>> example, although for my checking account, there are dozens like this.  The
>> top portion shows the register with the Sprint bill cleared.  The date,
>> amount, and name (mostly) match.
>> 
>> [image: image.png]
>> 
>> Not sure if there is just something wrong with my setup or not.  Perhaps a
>> bug?  Are others experiencing this?  Any ideas to get the matcher matching
>> again?  Something need to get cleared out?
>> 
>> For those of us in the US, happy Independence Day.  Thank you all for your
>> assistance.
>> 
>> Michael
>> 
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see 
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.

___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Import Transaction Matcher issue in v4

[Fross, Michael]

Hello all,

I sent this earlier this month and didn't see any reply so I thought I
would try again.   Has anyone else seen these issues?  I use Citibank and
perhaps it's a Citibank issue, but I did not have this problem on v2 or v3.

Thanks all.  I appreciate the help.

Michael


On Sat, Jul 4, 2020 at 9:48 AM Fross, Michael  wrote:

> Hello all,
>
> I typically download QFX files from my banks every day or two, import them
> to clear them in Gnucash.  Worked great.  However, ever since upgrading to
> v4, the importer seems to have trouble matching.  Most of the imported
> transactions are listed in the importer as (A)dd, but when I select
> (C)lear for them it says match missing.
>
> This has occurred for several accounts.  Here is a simple credit card
> example, although for my checking account, there are dozens like this.  The
> top portion shows the register with the Sprint bill cleared.  The date,
> amount, and name (mostly) match.
>
> [image: image.png]
>
> Not sure if there is just something wrong with my setup or not.  Perhaps a
> bug?  Are others experiencing this?  Any ideas to get the matcher matching
> again?  Something need to get cleared out?
>
> For those of us in the US, happy Independence Day.  Thank you all for your
> assistance.
>
> Michael
>
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

[GNC] Import Transaction Matcher issue in v4

[Fross, Michael]

Hello all,

I typically download QFX files from my banks every day or two, import them
to clear them in Gnucash.  Worked great.  However, ever since upgrading to
v4, the importer seems to have trouble matching.  Most of the imported
transactions are listed in the importer as (A)dd, but when I select
(C)lear for them it says match missing.

This has occurred for several accounts.  Here is a simple credit card
example, although for my checking account, there are dozens like this.  The
top portion shows the register with the Sprint bill cleared.  The date,
amount, and name (mostly) match.

[image: image.png]

Not sure if there is just something wrong with my setup or not.  Perhaps a
bug?  Are others experiencing this?  Any ideas to get the matcher matching
again?  Something need to get cleared out?

For those of us in the US, happy Independence Day.  Thank you all for your
assistance.

Michael
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see 
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.