Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Wayne Stambaugh]

I finally got a chance to test this and here is what I found.  Please
note, my testing was limited to simply adding (foo bar) to each section
and object in the board file.  I did not add anything more complex such
as (foo (bar 1.15)(baz "A baz string")).  All valid syntax would need to
be supported.

The "general" and "page" sections and "via" object drop the unknown
syntax on file save.

The "title_block" and "layer" sections and "net" and "segment" objects
load with invalid file error.

The "setup" section and "net_class", "module", "dimension", "gr_text",
and "zones" objects work but it can move the location of the unknown syntax.

The "gr_line" object works when I add at the end of the list of tokens
but not immediately after the "gr_line" token.

Given these issues, I'm going to defer to 6.0.  If we decide to go this
route, I would like to write the parser to handle any case of unknown
syntax with one giant caveat, any third party syntax that causes a name
space clash with any changes to our file format looses the argument.  We
retain the right to use whatever syntax we want without having to figure
out what anyone else is doing so extend the kicad file at your own risk.
 If this isn't acceptable, then I'm going to go with the original
properties concept and any property name space clashes can be hashed out
between the third parties.

Cheers,

Wayne

On 3/27/2018 1:34 PM, Jeff Young wrote:
> 5.0, 6.0 or abandon?
> 
>> On 20 Mar 2018, at 16:47, Jeff Young > > wrote:
>>
>> Hi Wayne,
>>
>> This solution /*is*/ pretty much “property” tokens except that:
>>
>> 1) it’s more flexible (properties can be arbitrary s-expressions)
>> 2) it handles 3rd-party-needs and our own with a single infrastructure
>>
>> I was going to add (3) it requires name-spacing, but “property” tokens
>> will require name spacing anyway (to keep 2 different 3rd parties from
>> colliding).  The only difference is that we get the default name-space
>> to ourselves.
>>
>> In fact, I think the “property” angle is a much better angle to look
>> at it from.  Rather than think of it as a file-format issue, think of
>> it as s-expression meta-data for BOARD_ITEMS.  If we later decide that
>> "(hole-to-hole-clearance 0.25)" should define a distance used by DRC,
>> then great.  Until then, it’s just an opaque s-expression.
>>
>> I also think it demonstrates that the risk of wiping out stuff editing
>> a future board with an older version isn’t really there.  Just like
>> “properties”, we’ll round-trip whatever meta-data was in the board.
>>  Sure, new BOARD_ITEMS that you add won’t have the future data, but
>> why would they?  
>>
>> (And if we remain concerned José’s force-save-as solution is an
>> excellent idea.)
>>
>> The code complexity it adds to the parser is tiny.  It turned out to
>> be far easier to implement than anticipated.
>>
>> Cheers,
>> Jeff.
>>
>>
>>> On 20 Mar 2018, at 14:48, Wayne Stambaugh >> > wrote:
>>>
>>> Jeff,
>>>
>>> I wanted some time to think about this.  It has been discussed before.
>>> While I'm not completely opposed to it, I still haven't found a more
>>> compelling argument to convince me that it is a better idea than using
>>> strict parsing.  As a user, it does have a certain appeal.  As a
>>> developer, it opens a Pandora's box of issues that once they are in
>>> play, could be extremely difficult to reverse.  Please see my responses
>>> below.
>>>
>>> On 3/20/2018 9:40 AM, Jeff Young wrote:
 @Wayne, did you have any thoughts on this iteration?

> On 20 Mar 2018, at 10:22, Jeff Young  
> > wrote:
>
> Oh, and I don’t think this materially alters the support equation: we
> already have to deal with hand-edited boards, so what’s in the file is
> never guaranteed to be something Kicad put there.
>>>
>>> True, but this is one of the reasons that the board parser is strict.
>>> It is immediately obvious what doesn't belong in the file.  Also, in the
>>> past users have tricked eeschema and pcbnew into a feature that didn't
>>> technically exist by taking advantage of the loose file parsing.  Then
>>> when something changed internally and their clever hacks were broken, we
>>> ended up with bug reports.  I do not want to repeat this again.
>>>
>
>> On 20 Mar 2018, at 10:19, Jeff Young > 
>> > wrote:
>>
>> Hi Seth,
>>
>> The original version spit out log entries for skipped elements.  This
>> version follows the XML/browser convention of silently ignoring.
>>  Even though this isn’t an XML format, I think we need to recognise
>> that we live in an XML world and XML processing conventions set
>> expectations.
>>>
>>> After reading this, I am even more thankful that we didn't choose XML
>>> for our file format.  Gleefully ignoring file syntax that the parser
>>> doesn't understand IMO is a bad

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Wayne Stambaugh]

I agree with this.  The current pad parsing code should not be open
ended even if there are plans for future expansion.

On 3/27/2018 1:51 PM, Seth Hillbrand wrote:
> Hi Jeff-
> 
> Can we split this patch?  I think the addition of strict checking for
> the pads should be in 5.  
> 
> -S
> 
> 2018-03-27 10:34 GMT-07:00 Jeff Young  >:
> 
> 5.0, 6.0 or abandon?
> 
> 
>> On 20 Mar 2018, at 16:47, Jeff Young > > wrote:
>>
>> Hi Wayne,
>>
>> This solution /*is*/ pretty much “property” tokens except that:
>>
>> 1) it’s more flexible (properties can be arbitrary s-expressions)
>> 2) it handles 3rd-party-needs and our own with a single infrastructure
>>
>> I was going to add (3) it requires name-spacing, but “property”
>> tokens will require name spacing anyway (to keep 2 different 3rd
>> parties from colliding).  The only difference is that we get the
>> default name-space to ourselves.
>>
>> In fact, I think the “property” angle is a much better angle to
>> look at it from.  Rather than think of it as a file-format issue,
>> think of it as s-expression meta-data for BOARD_ITEMS.  If we
>> later decide that "(hole-to-hole-clearance 0.25)" should define a
>> distance used by DRC, then great.  Until then, it’s just an opaque
>> s-expression.
>>
>> I also think it demonstrates that the risk of wiping out stuff
>> editing a future board with an older version isn’t really there. 
>> Just like “properties”, we’ll round-trip whatever meta-data was in
>> the board.  Sure, new BOARD_ITEMS that you add won’t have the
>> future data, but why would they?  
>>
>> (And if we remain concerned José’s force-save-as solution is an
>> excellent idea.)
>>
>> The code complexity it adds to the parser is tiny.  It turned out
>> to be far easier to implement than anticipated.
>>
>> Cheers,
>> Jeff.
>>
>>
>>> On 20 Mar 2018, at 14:48, Wayne Stambaugh >> > wrote:
>>>
>>> Jeff,
>>>
>>> I wanted some time to think about this.  It has been discussed
>>> before.
>>> While I'm not completely opposed to it, I still haven't found a more
>>> compelling argument to convince me that it is a better idea than
>>> using
>>> strict parsing.  As a user, it does have a certain appeal.  As a
>>> developer, it opens a Pandora's box of issues that once they are in
>>> play, could be extremely difficult to reverse.  Please see my
>>> responses
>>> below.
>>>
>>> On 3/20/2018 9:40 AM, Jeff Young wrote:
 @Wayne, did you have any thoughts on this iteration?

> On 20 Mar 2018, at 10:22, Jeff Young  
> > wrote:
>
> Oh, and I don’t think this materially alters the support
> equation: we
> already have to deal with hand-edited boards, so what’s in the
> file is
> never guaranteed to be something Kicad put there.
>>>
>>> True, but this is one of the reasons that the board parser is strict.
>>> It is immediately obvious what doesn't belong in the file.  Also,
>>> in the
>>> past users have tricked eeschema and pcbnew into a feature that
>>> didn't
>>> technically exist by taking advantage of the loose file parsing. 
>>> Then
>>> when something changed internally and their clever hacks were
>>> broken, we
>>> ended up with bug reports.  I do not want to repeat this again.
>>>
>
>> On 20 Mar 2018, at 10:19, Jeff Young > 
>> > wrote:
>>
>> Hi Seth,
>>
>> The original version spit out log entries for skipped
>> elements.  This
>> version follows the XML/browser convention of silently ignoring.
>>  Even though this isn’t an XML format, I think we need to
>> recognise
>> that we live in an XML world and XML processing conventions set
>> expectations.
>>>
>>> After reading this, I am even more thankful that we didn't choose XML
>>> for our file format.  Gleefully ignoring file syntax that the parser
>>> doesn't understand IMO is a bad idea.  This flies in the face of the
>>> basic premise that we control the format of our files not someone
>>> else.
>>> As soon as you allow others to dictate your file format, you are
>>> in trouble.
>>>
>>
>> The patch strictly checks everything for round-tripping so
>> that there
>> is no data loss.  The pad stuff is really a separate issue: it was
>> meant to be loose only during development and then tightened
>> up, but
>> the tightening step was forgotten.  Since we don’t store pad
>> stuff we
>> don’t understand, it has to be tightened.  In short: if you
>> can round
>> trip stuff 

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Wayne Stambaugh]

@Jeff, I still have to test the round tripping to make sure there is no
data loss for potential combinations of unknown tokens.  As soon as I
have tested it, I will let you know.

On 3/27/2018 1:34 PM, Jeff Young wrote:
> 5.0, 6.0 or abandon?
> 
>> On 20 Mar 2018, at 16:47, Jeff Young > > wrote:
>>
>> Hi Wayne,
>>
>> This solution /*is*/ pretty much “property” tokens except that:
>>
>> 1) it’s more flexible (properties can be arbitrary s-expressions)
>> 2) it handles 3rd-party-needs and our own with a single infrastructure
>>
>> I was going to add (3) it requires name-spacing, but “property” tokens
>> will require name spacing anyway (to keep 2 different 3rd parties from
>> colliding).  The only difference is that we get the default name-space
>> to ourselves.
>>
>> In fact, I think the “property” angle is a much better angle to look
>> at it from.  Rather than think of it as a file-format issue, think of
>> it as s-expression meta-data for BOARD_ITEMS.  If we later decide that
>> "(hole-to-hole-clearance 0.25)" should define a distance used by DRC,
>> then great.  Until then, it’s just an opaque s-expression.
>>
>> I also think it demonstrates that the risk of wiping out stuff editing
>> a future board with an older version isn’t really there.  Just like
>> “properties”, we’ll round-trip whatever meta-data was in the board.
>>  Sure, new BOARD_ITEMS that you add won’t have the future data, but
>> why would they?  
>>
>> (And if we remain concerned José’s force-save-as solution is an
>> excellent idea.)
>>
>> The code complexity it adds to the parser is tiny.  It turned out to
>> be far easier to implement than anticipated.
>>
>> Cheers,
>> Jeff.
>>
>>
>>> On 20 Mar 2018, at 14:48, Wayne Stambaugh >> > wrote:
>>>
>>> Jeff,
>>>
>>> I wanted some time to think about this.  It has been discussed before.
>>> While I'm not completely opposed to it, I still haven't found a more
>>> compelling argument to convince me that it is a better idea than using
>>> strict parsing.  As a user, it does have a certain appeal.  As a
>>> developer, it opens a Pandora's box of issues that once they are in
>>> play, could be extremely difficult to reverse.  Please see my responses
>>> below.
>>>
>>> On 3/20/2018 9:40 AM, Jeff Young wrote:
 @Wayne, did you have any thoughts on this iteration?

> On 20 Mar 2018, at 10:22, Jeff Young  
> > wrote:
>
> Oh, and I don’t think this materially alters the support equation: we
> already have to deal with hand-edited boards, so what’s in the file is
> never guaranteed to be something Kicad put there.
>>>
>>> True, but this is one of the reasons that the board parser is strict.
>>> It is immediately obvious what doesn't belong in the file.  Also, in the
>>> past users have tricked eeschema and pcbnew into a feature that didn't
>>> technically exist by taking advantage of the loose file parsing.  Then
>>> when something changed internally and their clever hacks were broken, we
>>> ended up with bug reports.  I do not want to repeat this again.
>>>
>
>> On 20 Mar 2018, at 10:19, Jeff Young > 
>> > wrote:
>>
>> Hi Seth,
>>
>> The original version spit out log entries for skipped elements.  This
>> version follows the XML/browser convention of silently ignoring.
>>  Even though this isn’t an XML format, I think we need to recognise
>> that we live in an XML world and XML processing conventions set
>> expectations.
>>>
>>> After reading this, I am even more thankful that we didn't choose XML
>>> for our file format.  Gleefully ignoring file syntax that the parser
>>> doesn't understand IMO is a bad idea.  This flies in the face of the
>>> basic premise that we control the format of our files not someone else.
>>> As soon as you allow others to dictate your file format, you are in
>>> trouble.
>>>
>>
>> The patch strictly checks everything for round-tripping so that there
>> is no data loss.  The pad stuff is really a separate issue: it was
>> meant to be loose only during development and then tightened up, but
>> the tightening step was forgotten.  Since we don’t store pad stuff we
>> don’t understand, it has to be tightened.  In short: if you can round
>> trip stuff you don’t understand then do so; otherwise throw.
>>>
>>> The round tripping is fine and makes sense but it also adds to the code
>>> complexity of the parser.  The pad issue is different.  I don't know
>>> when that was slipped into the parser but it should not be there.  If
>>> there is an unexpected token, that should flag a file load error.
>>>
>>
>> Certainly one use case is opening boards from future versions.  If
>> you edit them, then you’re at your own peril.  This behaviour is
>> common enough that I believe it is well understood (although we
>> should 

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Seth Hillbrand]

Hi Jeff-

Can we split this patch?  I think the addition of strict checking for the
pads should be in 5.

-S

2018-03-27 10:34 GMT-07:00 Jeff Young :

> 5.0, 6.0 or abandon?
>
>
> On 20 Mar 2018, at 16:47, Jeff Young  wrote:
>
> Hi Wayne,
>
> This solution *is* pretty much “property” tokens except that:
>
> 1) it’s more flexible (properties can be arbitrary s-expressions)
> 2) it handles 3rd-party-needs and our own with a single infrastructure
>
> I was going to add (3) it requires name-spacing, but “property” tokens
> will require name spacing anyway (to keep 2 different 3rd parties from
> colliding).  The only difference is that we get the default name-space to
> ourselves.
>
> In fact, I think the “property” angle is a much better angle to look at it
> from.  Rather than think of it as a file-format issue, think of it as
> s-expression meta-data for BOARD_ITEMS.  If we later decide that
> "(hole-to-hole-clearance 0.25)" should define a distance used by DRC, then
> great.  Until then, it’s just an opaque s-expression.
>
> I also think it demonstrates that the risk of wiping out stuff editing a
> future board with an older version isn’t really there.  Just like
> “properties”, we’ll round-trip whatever meta-data was in the board.  Sure,
> new BOARD_ITEMS that you add won’t have the future data, but why would
> they?
>
> (And if we remain concerned José’s force-save-as solution is an excellent
> idea.)
>
> The code complexity it adds to the parser is tiny.  It turned out to be
> far easier to implement than anticipated.
>
> Cheers,
> Jeff.
>
>
> On 20 Mar 2018, at 14:48, Wayne Stambaugh  wrote:
>
> Jeff,
>
> I wanted some time to think about this.  It has been discussed before.
> While I'm not completely opposed to it, I still haven't found a more
> compelling argument to convince me that it is a better idea than using
> strict parsing.  As a user, it does have a certain appeal.  As a
> developer, it opens a Pandora's box of issues that once they are in
> play, could be extremely difficult to reverse.  Please see my responses
> below.
>
> On 3/20/2018 9:40 AM, Jeff Young wrote:
>
> @Wayne, did you have any thoughts on this iteration?
>
> On 20 Mar 2018, at 10:22, Jeff Young  > wrote:
>
> Oh, and I don’t think this materially alters the support equation: we
> already have to deal with hand-edited boards, so what’s in the file is
> never guaranteed to be something Kicad put there.
>
>
> True, but this is one of the reasons that the board parser is strict.
> It is immediately obvious what doesn't belong in the file.  Also, in the
> past users have tricked eeschema and pcbnew into a feature that didn't
> technically exist by taking advantage of the loose file parsing.  Then
> when something changed internally and their clever hacks were broken, we
> ended up with bug reports.  I do not want to repeat this again.
>
>
> On 20 Mar 2018, at 10:19, Jeff Young  > wrote:
>
> Hi Seth,
>
> The original version spit out log entries for skipped elements.  This
> version follows the XML/browser convention of silently ignoring.
>  Even though this isn’t an XML format, I think we need to recognise
> that we live in an XML world and XML processing conventions set
> expectations.
>
>
> After reading this, I am even more thankful that we didn't choose XML
> for our file format.  Gleefully ignoring file syntax that the parser
> doesn't understand IMO is a bad idea.  This flies in the face of the
> basic premise that we control the format of our files not someone else.
> As soon as you allow others to dictate your file format, you are in
> trouble.
>
>
> The patch strictly checks everything for round-tripping so that there
> is no data loss.  The pad stuff is really a separate issue: it was
> meant to be loose only during development and then tightened up, but
> the tightening step was forgotten.  Since we don’t store pad stuff we
> don’t understand, it has to be tightened.  In short: if you can round
> trip stuff you don’t understand then do so; otherwise throw.
>
>
> The round tripping is fine and makes sense but it also adds to the code
> complexity of the parser.  The pad issue is different.  I don't know
> when that was slipped into the parser but it should not be there.  If
> there is an unexpected token, that should flag a file load error.
>
>
> Certainly one use case is opening boards from future versions.  If
> you edit them, then you’re at your own peril.  This behaviour is
> common enough that I believe it is well understood (although we
> should obviously mention it in a version-check dialog).
>
>
> Perhaps, but I just see this ending badly.  Maybe I'm being paranoid
> here but it's so easy to imagine a scenario where someone did a lot of
> editing in a newer version of kicad then makes the fatal mistake of
> opening the board in an older version of kicad and wipes out a lot of
> work.  Technically not kicad's fault but I'm not so sure the user will
> see it that way.
>

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Jeff Young]

5.0, 6.0 or abandon?

> On 20 Mar 2018, at 16:47, Jeff Young  wrote:
> 
> Hi Wayne,
> 
> This solution is pretty much “property” tokens except that:
> 
> 1) it’s more flexible (properties can be arbitrary s-expressions)
> 2) it handles 3rd-party-needs and our own with a single infrastructure
> 
> I was going to add (3) it requires name-spacing, but “property” tokens will 
> require name spacing anyway (to keep 2 different 3rd parties from colliding). 
>  The only difference is that we get the default name-space to ourselves.
> 
> In fact, I think the “property” angle is a much better angle to look at it 
> from.  Rather than think of it as a file-format issue, think of it as 
> s-expression meta-data for BOARD_ITEMS.  If we later decide that 
> "(hole-to-hole-clearance 0.25)" should define a distance used by DRC, then 
> great.  Until then, it’s just an opaque s-expression.
> 
> I also think it demonstrates that the risk of wiping out stuff editing a 
> future board with an older version isn’t really there.  Just like 
> “properties”, we’ll round-trip whatever meta-data was in the board.  Sure, 
> new BOARD_ITEMS that you add won’t have the future data, but why would they?  
> 
> (And if we remain concerned José’s force-save-as solution is an excellent 
> idea.)
> 
> The code complexity it adds to the parser is tiny.  It turned out to be far 
> easier to implement than anticipated.
> 
> Cheers,
> Jeff.
> 
> 
>> On 20 Mar 2018, at 14:48, Wayne Stambaugh > > wrote:
>> 
>> Jeff,
>> 
>> I wanted some time to think about this.  It has been discussed before.
>> While I'm not completely opposed to it, I still haven't found a more
>> compelling argument to convince me that it is a better idea than using
>> strict parsing.  As a user, it does have a certain appeal.  As a
>> developer, it opens a Pandora's box of issues that once they are in
>> play, could be extremely difficult to reverse.  Please see my responses
>> below.
>> 
>> On 3/20/2018 9:40 AM, Jeff Young wrote:
>>> @Wayne, did you have any thoughts on this iteration?
>>> 
 On 20 Mar 2018, at 10:22, Jeff Young >>> 
 >> wrote:
 
 Oh, and I don’t think this materially alters the support equation: we
 already have to deal with hand-edited boards, so what’s in the file is
 never guaranteed to be something Kicad put there.
>> 
>> True, but this is one of the reasons that the board parser is strict.
>> It is immediately obvious what doesn't belong in the file.  Also, in the
>> past users have tricked eeschema and pcbnew into a feature that didn't
>> technically exist by taking advantage of the loose file parsing.  Then
>> when something changed internally and their clever hacks were broken, we
>> ended up with bug reports.  I do not want to repeat this again.
>> 
 
> On 20 Mar 2018, at 10:19, Jeff Young  
> >> wrote:
> 
> Hi Seth,
> 
> The original version spit out log entries for skipped elements.  This
> version follows the XML/browser convention of silently ignoring.
>  Even though this isn’t an XML format, I think we need to recognise
> that we live in an XML world and XML processing conventions set
> expectations.
>> 
>> After reading this, I am even more thankful that we didn't choose XML
>> for our file format.  Gleefully ignoring file syntax that the parser
>> doesn't understand IMO is a bad idea.  This flies in the face of the
>> basic premise that we control the format of our files not someone else.
>> As soon as you allow others to dictate your file format, you are in trouble.
>> 
> 
> The patch strictly checks everything for round-tripping so that there
> is no data loss.  The pad stuff is really a separate issue: it was
> meant to be loose only during development and then tightened up, but
> the tightening step was forgotten.  Since we don’t store pad stuff we
> don’t understand, it has to be tightened.  In short: if you can round
> trip stuff you don’t understand then do so; otherwise throw.
>> 
>> The round tripping is fine and makes sense but it also adds to the code
>> complexity of the parser.  The pad issue is different.  I don't know
>> when that was slipped into the parser but it should not be there.  If
>> there is an unexpected token, that should flag a file load error.
>> 
> 
> Certainly one use case is opening boards from future versions.  If
> you edit them, then you’re at your own peril.  This behaviour is
> common enough that I believe it is well understood (although we
> should obviously mention it in a version-check dialog).
>> 
>> Perhaps, but I just see this ending badly.  Maybe I'm being paranoid
>> here but it's so easy to imagine a scenario where someone did a lot of
>> editing in a newer version of kicad then makes the fatal mistake of
>> opening

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Jeff Young]

Hi Wayne,

This solution is pretty much “property” tokens except that:

1) it’s more flexible (properties can be arbitrary s-expressions)
2) it handles 3rd-party-needs and our own with a single infrastructure

I was going to add (3) it requires name-spacing, but “property” tokens will 
require name spacing anyway (to keep 2 different 3rd parties from colliding).  
The only difference is that we get the default name-space to ourselves.

In fact, I think the “property” angle is a much better angle to look at it 
from.  Rather than think of it as a file-format issue, think of it as 
s-expression meta-data for BOARD_ITEMS.  If we later decide that 
"(hole-to-hole-clearance 0.25)" should define a distance used by DRC, then 
great.  Until then, it’s just an opaque s-expression.

I also think it demonstrates that the risk of wiping out stuff editing a future 
board with an older version isn’t really there.  Just like “properties”, we’ll 
round-trip whatever meta-data was in the board.  Sure, new BOARD_ITEMS that you 
add won’t have the future data, but why would they?  

(And if we remain concerned José’s force-save-as solution is an excellent idea.)

The code complexity it adds to the parser is tiny.  It turned out to be far 
easier to implement than anticipated.

Cheers,
Jeff.


> On 20 Mar 2018, at 14:48, Wayne Stambaugh  wrote:
> 
> Jeff,
> 
> I wanted some time to think about this.  It has been discussed before.
> While I'm not completely opposed to it, I still haven't found a more
> compelling argument to convince me that it is a better idea than using
> strict parsing.  As a user, it does have a certain appeal.  As a
> developer, it opens a Pandora's box of issues that once they are in
> play, could be extremely difficult to reverse.  Please see my responses
> below.
> 
> On 3/20/2018 9:40 AM, Jeff Young wrote:
>> @Wayne, did you have any thoughts on this iteration?
>> 
>>> On 20 Mar 2018, at 10:22, Jeff Young >> > wrote:
>>> 
>>> Oh, and I don’t think this materially alters the support equation: we
>>> already have to deal with hand-edited boards, so what’s in the file is
>>> never guaranteed to be something Kicad put there.
> 
> True, but this is one of the reasons that the board parser is strict.
> It is immediately obvious what doesn't belong in the file.  Also, in the
> past users have tricked eeschema and pcbnew into a feature that didn't
> technically exist by taking advantage of the loose file parsing.  Then
> when something changed internally and their clever hacks were broken, we
> ended up with bug reports.  I do not want to repeat this again.
> 
>>> 
 On 20 Mar 2018, at 10:19, Jeff Young >>> > wrote:
 
 Hi Seth,
 
 The original version spit out log entries for skipped elements.  This
 version follows the XML/browser convention of silently ignoring.
  Even though this isn’t an XML format, I think we need to recognise
 that we live in an XML world and XML processing conventions set
 expectations.
> 
> After reading this, I am even more thankful that we didn't choose XML
> for our file format.  Gleefully ignoring file syntax that the parser
> doesn't understand IMO is a bad idea.  This flies in the face of the
> basic premise that we control the format of our files not someone else.
> As soon as you allow others to dictate your file format, you are in trouble.
> 
 
 The patch strictly checks everything for round-tripping so that there
 is no data loss.  The pad stuff is really a separate issue: it was
 meant to be loose only during development and then tightened up, but
 the tightening step was forgotten.  Since we don’t store pad stuff we
 don’t understand, it has to be tightened.  In short: if you can round
 trip stuff you don’t understand then do so; otherwise throw.
> 
> The round tripping is fine and makes sense but it also adds to the code
> complexity of the parser.  The pad issue is different.  I don't know
> when that was slipped into the parser but it should not be there.  If
> there is an unexpected token, that should flag a file load error.
> 
 
 Certainly one use case is opening boards from future versions.  If
 you edit them, then you’re at your own peril.  This behaviour is
 common enough that I believe it is well understood (although we
 should obviously mention it in a version-check dialog).
> 
> Perhaps, but I just see this ending badly.  Maybe I'm being paranoid
> here but it's so easy to imagine a scenario where someone did a lot of
> editing in a newer version of kicad then makes the fatal mistake of
> opening the board in an older version of kicad and wipes out a lot of
> work.  Technically not kicad's fault but I'm not so sure the user will
> see it that way.
> 
 
 Another use case is 3rd-party tools (which might even be written as
 Python plugins) that want to carry their own stuff around in the
 board.  These might even be proces

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[José Ignacio]

The user would have to click through a big fat warning that the file is
from the future. If you wanna be doubly sure, open things in read only mode
if there is a version mismatch, so the user at worst can save the file with
another name, allowing recovery without having to drop into a text editor.


On Tue, Mar 20, 2018 at 9:48 AM, Wayne Stambaugh 
wrote:

> Jeff,
>
> I wanted some time to think about this.  It has been discussed before.
> While I'm not completely opposed to it, I still haven't found a more
> compelling argument to convince me that it is a better idea than using
> strict parsing.  As a user, it does have a certain appeal.  As a
> developer, it opens a Pandora's box of issues that once they are in
> play, could be extremely difficult to reverse.  Please see my responses
> below.
>
> On 3/20/2018 9:40 AM, Jeff Young wrote:
> > @Wayne, did you have any thoughts on this iteration?
> >
> >> On 20 Mar 2018, at 10:22, Jeff Young  >> > wrote:
> >>
> >> Oh, and I don’t think this materially alters the support equation: we
> >> already have to deal with hand-edited boards, so what’s in the file is
> >> never guaranteed to be something Kicad put there.
>
> True, but this is one of the reasons that the board parser is strict.
> It is immediately obvious what doesn't belong in the file.  Also, in the
> past users have tricked eeschema and pcbnew into a feature that didn't
> technically exist by taking advantage of the loose file parsing.  Then
> when something changed internally and their clever hacks were broken, we
> ended up with bug reports.  I do not want to repeat this again.
>
> >>
> >>> On 20 Mar 2018, at 10:19, Jeff Young  >>> > wrote:
> >>>
> >>> Hi Seth,
> >>>
> >>> The original version spit out log entries for skipped elements.  This
> >>> version follows the XML/browser convention of silently ignoring.
> >>>  Even though this isn’t an XML format, I think we need to recognise
> >>> that we live in an XML world and XML processing conventions set
> >>> expectations.
>
> After reading this, I am even more thankful that we didn't choose XML
> for our file format.  Gleefully ignoring file syntax that the parser
> doesn't understand IMO is a bad idea.  This flies in the face of the
> basic premise that we control the format of our files not someone else.
> As soon as you allow others to dictate your file format, you are in
> trouble.
>
> >>>
> >>> The patch strictly checks everything for round-tripping so that there
> >>> is no data loss.  The pad stuff is really a separate issue: it was
> >>> meant to be loose only during development and then tightened up, but
> >>> the tightening step was forgotten.  Since we don’t store pad stuff we
> >>> don’t understand, it has to be tightened.  In short: if you can round
> >>> trip stuff you don’t understand then do so; otherwise throw.
>
> The round tripping is fine and makes sense but it also adds to the code
> complexity of the parser.  The pad issue is different.  I don't know
> when that was slipped into the parser but it should not be there.  If
> there is an unexpected token, that should flag a file load error.
>
> >>>
> >>> Certainly one use case is opening boards from future versions.  If
> >>> you edit them, then you’re at your own peril.  This behaviour is
> >>> common enough that I believe it is well understood (although we
> >>> should obviously mention it in a version-check dialog).
>
> Perhaps, but I just see this ending badly.  Maybe I'm being paranoid
> here but it's so easy to imagine a scenario where someone did a lot of
> editing in a newer version of kicad then makes the fatal mistake of
> opening the board in an older version of kicad and wipes out a lot of
> work.  Technically not kicad's fault but I'm not so sure the user will
> see it that way.
>
> >>>
> >>> Another use case is 3rd-party tools (which might even be written as
> >>> Python plugins) that want to carry their own stuff around in the
> >>> board.  These might even be processing/manufacturing instructions
> >>> that don’t have any visual expression in Kicad anyway.
>
> This is one of the primary reasons that I do not like ignoring unknown
> file formatting.  It creates the potential for name space pollution that
> could cause issues down the road.  The eventual goal is to implement the
> "property" token to define key/value pairs for third party applications
> to add user specific information to any board object without polluting
> the controlled part of the file format.  The beauty of this is that we
> do not have to coordinate with 3rd party developers to ensure we are not
> clashing with anything the are working on.  They are free to add
> whatever properties they would like while we still maintain strict file
> parsing.  This was always part of the grand plan for the board file
> format that kind of got lost in the noise and me becoming project leader.
>
> Cheers,
>
> Wayne
>
> >>>
> >>> Cheers,
> >>> Jeff.
> >>>
> >>>
> >>

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Wayne Stambaugh]

Jeff,

I wanted some time to think about this.  It has been discussed before.
While I'm not completely opposed to it, I still haven't found a more
compelling argument to convince me that it is a better idea than using
strict parsing.  As a user, it does have a certain appeal.  As a
developer, it opens a Pandora's box of issues that once they are in
play, could be extremely difficult to reverse.  Please see my responses
below.

On 3/20/2018 9:40 AM, Jeff Young wrote:
> @Wayne, did you have any thoughts on this iteration?
> 
>> On 20 Mar 2018, at 10:22, Jeff Young > > wrote:
>>
>> Oh, and I don’t think this materially alters the support equation: we
>> already have to deal with hand-edited boards, so what’s in the file is
>> never guaranteed to be something Kicad put there.

True, but this is one of the reasons that the board parser is strict.
It is immediately obvious what doesn't belong in the file.  Also, in the
past users have tricked eeschema and pcbnew into a feature that didn't
technically exist by taking advantage of the loose file parsing.  Then
when something changed internally and their clever hacks were broken, we
ended up with bug reports.  I do not want to repeat this again.

>>
>>> On 20 Mar 2018, at 10:19, Jeff Young >> > wrote:
>>>
>>> Hi Seth,
>>>
>>> The original version spit out log entries for skipped elements.  This
>>> version follows the XML/browser convention of silently ignoring.
>>>  Even though this isn’t an XML format, I think we need to recognise
>>> that we live in an XML world and XML processing conventions set
>>> expectations.

After reading this, I am even more thankful that we didn't choose XML
for our file format.  Gleefully ignoring file syntax that the parser
doesn't understand IMO is a bad idea.  This flies in the face of the
basic premise that we control the format of our files not someone else.
As soon as you allow others to dictate your file format, you are in trouble.

>>>
>>> The patch strictly checks everything for round-tripping so that there
>>> is no data loss.  The pad stuff is really a separate issue: it was
>>> meant to be loose only during development and then tightened up, but
>>> the tightening step was forgotten.  Since we don’t store pad stuff we
>>> don’t understand, it has to be tightened.  In short: if you can round
>>> trip stuff you don’t understand then do so; otherwise throw.

The round tripping is fine and makes sense but it also adds to the code
complexity of the parser.  The pad issue is different.  I don't know
when that was slipped into the parser but it should not be there.  If
there is an unexpected token, that should flag a file load error.

>>>
>>> Certainly one use case is opening boards from future versions.  If
>>> you edit them, then you’re at your own peril.  This behaviour is
>>> common enough that I believe it is well understood (although we
>>> should obviously mention it in a version-check dialog).

Perhaps, but I just see this ending badly.  Maybe I'm being paranoid
here but it's so easy to imagine a scenario where someone did a lot of
editing in a newer version of kicad then makes the fatal mistake of
opening the board in an older version of kicad and wipes out a lot of
work.  Technically not kicad's fault but I'm not so sure the user will
see it that way.

>>>
>>> Another use case is 3rd-party tools (which might even be written as
>>> Python plugins) that want to carry their own stuff around in the
>>> board.  These might even be processing/manufacturing instructions
>>> that don’t have any visual expression in Kicad anyway.

This is one of the primary reasons that I do not like ignoring unknown
file formatting.  It creates the potential for name space pollution that
could cause issues down the road.  The eventual goal is to implement the
"property" token to define key/value pairs for third party applications
to add user specific information to any board object without polluting
the controlled part of the file format.  The beauty of this is that we
do not have to coordinate with 3rd party developers to ensure we are not
clashing with anything the are working on.  They are free to add
whatever properties they would like while we still maintain strict file
parsing.  This was always part of the grand plan for the board file
format that kind of got lost in the noise and me becoming project leader.

Cheers,

Wayne

>>>
>>> Cheers,
>>> Jeff.
>>>
>>>
 On 19 Mar 2018, at 22:51, Seth Hillbrand >>> > wrote:

 Hi Jeff-

 A few questions on how you are implementing this:

 1) How does the user know what was skipped?  I can imagine team
 members with different versions getting into difficulty, especially
 if the features being skipped change the board layout.

 2) You are removing strict checking for most of the board but you
 are adding strict checking for pad options. What's the difference?

 3) If we k

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Jeff Young]

@Wayne, did you have any thoughts on this iteration?

> On 20 Mar 2018, at 10:22, Jeff Young  wrote:
> 
> Oh, and I don’t think this materially alters the support equation: we already 
> have to deal with hand-edited boards, so what’s in the file is never 
> guaranteed to be something Kicad put there.
> 
>> On 20 Mar 2018, at 10:19, Jeff Young > > wrote:
>> 
>> Hi Seth,
>> 
>> The original version spit out log entries for skipped elements.  This 
>> version follows the XML/browser convention of silently ignoring.  Even 
>> though this isn’t an XML format, I think we need to recognise that we live 
>> in an XML world and XML processing conventions set expectations.
>> 
>> The patch strictly checks everything for round-tripping so that there is no 
>> data loss.  The pad stuff is really a separate issue: it was meant to be 
>> loose only during development and then tightened up, but the tightening step 
>> was forgotten.  Since we don’t store pad stuff we don’t understand, it has 
>> to be tightened.  In short: if you can round trip stuff you don’t understand 
>> then do so; otherwise throw.
>> 
>> Certainly one use case is opening boards from future versions.  If you edit 
>> them, then you’re at your own peril.  This behaviour is common enough that I 
>> believe it is well understood (although we should obviously mention it in a 
>> version-check dialog).
>> 
>> Another use case is 3rd-party tools (which might even be written as Python 
>> plugins) that want to carry their own stuff around in the board.  These 
>> might even be processing/manufacturing instructions that don’t have any 
>> visual expression in Kicad anyway.
>> 
>> Cheers,
>> Jeff.
>> 
>> 
>>> On 19 Mar 2018, at 22:51, Seth Hillbrand >> > wrote:
>>> 
>>> Hi Jeff-
>>> 
>>> A few questions on how you are implementing this:
>>> 
>>> 1) How does the user know what was skipped?  I can imagine team members 
>>> with different versions getting into difficulty, especially if the features 
>>> being skipped change the board layout.
>>> 
>>> 2) You are removing strict checking for most of the board but you are 
>>> adding strict checking for pad options. What's the difference?
>>> 
>>> 3) If we keep these options around but don't allow editing/removing, don't 
>>> we run into a risk of generating a really wonk-y board that only looks 
>>> wonky in a newer version of KiCad but looks fine in an older version?  For 
>>> example, imagine we implement rounded corners as a parameter and then a 
>>> user with an older version of KiCad edits and saves the board.  The rounded 
>>> corner is only visible in KiCad 6 but the user in KiCad 5 can edit the 
>>> board to look right to her only to have her colleague use KiCad 6 and see 
>>> the track violating DRC.  I think that might be counter-intuitive for users 
>>> but maybe there's a way around it.
>>> 
>>> In general, if I understand the use-case, this is to allow users to open 
>>> newer boards in older KiCad versions?  Is there another use case?
>>> 
>>> I think I can see clear to this for options like the 3d-model offset where 
>>> it could be either inches or mm but there isn't a difference in the actual 
>>> layout.  Allowing general unrecognized options would seem to open up a worm 
>>> can in terms of support as in "Please post the KiCad version and the file 
>>> version in order to figure out the problem." 
>>> 
>>> -S
>>> 
>>> 
>>> 
>>> 2018-03-18 9:46 GMT-07:00 Jeff Young >> >:
>>> OK, for your guys’ (re)viewing pleasure, a patch which losslessly 
>>> round-trips stuff it doesn’t understand.
>>> 
>>> 
>>> 
>>> 
 On 16 Mar 2018, at 19:15, hauptmech >>> > wrote:
 
 While i would still like to see this (my) shim go in, I agree with wayne. 
 Until/unless a less brittle approach is used for the parser, it's better 
 to signal a problem painfully and maintain the integrity of the file.
 
 I have to admit that when i first heard the s-expressions idea I assumed 
 that the intention was to use a lisp like virtual machine for loading and 
 maintaining design data. I've used vm's for data storage before (lua and 
 python), and it's great. Parsing is free and you can jam in functions and 
 other data when needed. 
 
 
 
 On 17 Mar 2018 07:17, "Jeff Young" >>> > wrote:
 Hi Wayne,
 
 Patch reverted.
 
 However, I think your concern is misplaced.  If we want to prevent 
 dataloss (ie: from reading a 6.0 file into 5.0), then we should warn the 
 user based on the version string (and give them a chance to say “I still 
 want to open”).
 
 But either way, actually failing to read the file leaves the user in a 
 pickle (especially when it’s easy enough for them to try out a nightly 
 that may very well be ahead of their stable).
 
 (And for that reason I think it’s important to

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Jeff Young]

Oh, and I don’t think this materially alters the support equation: we already 
have to deal with hand-edited boards, so what’s in the file is never guaranteed 
to be something Kicad put there.

> On 20 Mar 2018, at 10:19, Jeff Young  wrote:
> 
> Hi Seth,
> 
> The original version spit out log entries for skipped elements.  This version 
> follows the XML/browser convention of silently ignoring.  Even though this 
> isn’t an XML format, I think we need to recognise that we live in an XML 
> world and XML processing conventions set expectations.
> 
> The patch strictly checks everything for round-tripping so that there is no 
> data loss.  The pad stuff is really a separate issue: it was meant to be 
> loose only during development and then tightened up, but the tightening step 
> was forgotten.  Since we don’t store pad stuff we don’t understand, it has to 
> be tightened.  In short: if you can round trip stuff you don’t understand 
> then do so; otherwise throw.
> 
> Certainly one use case is opening boards from future versions.  If you edit 
> them, then you’re at your own peril.  This behaviour is common enough that I 
> believe it is well understood (although we should obviously mention it in a 
> version-check dialog).
> 
> Another use case is 3rd-party tools (which might even be written as Python 
> plugins) that want to carry their own stuff around in the board.  These might 
> even be processing/manufacturing instructions that don’t have any visual 
> expression in Kicad anyway.
> 
> Cheers,
> Jeff.
> 
> 
>> On 19 Mar 2018, at 22:51, Seth Hillbrand > > wrote:
>> 
>> Hi Jeff-
>> 
>> A few questions on how you are implementing this:
>> 
>> 1) How does the user know what was skipped?  I can imagine team members with 
>> different versions getting into difficulty, especially if the features being 
>> skipped change the board layout.
>> 
>> 2) You are removing strict checking for most of the board but you are adding 
>> strict checking for pad options. What's the difference?
>> 
>> 3) If we keep these options around but don't allow editing/removing, don't 
>> we run into a risk of generating a really wonk-y board that only looks wonky 
>> in a newer version of KiCad but looks fine in an older version?  For 
>> example, imagine we implement rounded corners as a parameter and then a user 
>> with an older version of KiCad edits and saves the board.  The rounded 
>> corner is only visible in KiCad 6 but the user in KiCad 5 can edit the board 
>> to look right to her only to have her colleague use KiCad 6 and see the 
>> track violating DRC.  I think that might be counter-intuitive for users but 
>> maybe there's a way around it.
>> 
>> In general, if I understand the use-case, this is to allow users to open 
>> newer boards in older KiCad versions?  Is there another use case?
>> 
>> I think I can see clear to this for options like the 3d-model offset where 
>> it could be either inches or mm but there isn't a difference in the actual 
>> layout.  Allowing general unrecognized options would seem to open up a worm 
>> can in terms of support as in "Please post the KiCad version and the file 
>> version in order to figure out the problem." 
>> 
>> -S
>> 
>> 
>> 
>> 2018-03-18 9:46 GMT-07:00 Jeff Young > >:
>> OK, for your guys’ (re)viewing pleasure, a patch which losslessly 
>> round-trips stuff it doesn’t understand.
>> 
>> 
>> 
>> 
>>> On 16 Mar 2018, at 19:15, hauptmech >> > wrote:
>>> 
>>> While i would still like to see this (my) shim go in, I agree with wayne. 
>>> Until/unless a less brittle approach is used for the parser, it's better to 
>>> signal a problem painfully and maintain the integrity of the file.
>>> 
>>> I have to admit that when i first heard the s-expressions idea I assumed 
>>> that the intention was to use a lisp like virtual machine for loading and 
>>> maintaining design data. I've used vm's for data storage before (lua and 
>>> python), and it's great. Parsing is free and you can jam in functions and 
>>> other data when needed. 
>>> 
>>> 
>>> 
>>> On 17 Mar 2018 07:17, "Jeff Young" mailto:j...@rokeby.ie>> 
>>> wrote:
>>> Hi Wayne,
>>> 
>>> Patch reverted.
>>> 
>>> However, I think your concern is misplaced.  If we want to prevent dataloss 
>>> (ie: from reading a 6.0 file into 5.0), then we should warn the user based 
>>> on the version string (and give them a chance to say “I still want to 
>>> open”).
>>> 
>>> But either way, actually failing to read the file leaves the user in a 
>>> pickle (especially when it’s easy enough for them to try out a nightly that 
>>> may very well be ahead of their stable).
>>> 
>>> (And for that reason I think it’s important to put it into 5.0, as 
>>> otherwise it won’t help until we start making 7.0 file format changes.)
>>> 
>>> Cheers,
>>> Jeff.
>>> 
>>> > On 16 Mar 2018, at 18:00, Wayne Stambaugh >> > > wrote:
>>> >
>>> > Jeff,
>>> 

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Jeff Young]

Hi Seth,

The original version spit out log entries for skipped elements.  This version 
follows the XML/browser convention of silently ignoring.  Even though this 
isn’t an XML format, I think we need to recognise that we live in an XML world 
and XML processing conventions set expectations.

The patch strictly checks everything for round-tripping so that there is no 
data loss.  The pad stuff is really a separate issue: it was meant to be loose 
only during development and then tightened up, but the tightening step was 
forgotten.  Since we don’t store pad stuff we don’t understand, it has to be 
tightened.  In short: if you can round trip stuff you don’t understand then do 
so; otherwise throw.

Certainly one use case is opening boards from future versions.  If you edit 
them, then you’re at your own peril.  This behaviour is common enough that I 
believe it is well understood (although we should obviously mention it in a 
version-check dialog).

Another use case is 3rd-party tools (which might even be written as Python 
plugins) that want to carry their own stuff around in the board.  These might 
even be processing/manufacturing instructions that don’t have any visual 
expression in Kicad anyway.

Cheers,
Jeff.


> On 19 Mar 2018, at 22:51, Seth Hillbrand  wrote:
> 
> Hi Jeff-
> 
> A few questions on how you are implementing this:
> 
> 1) How does the user know what was skipped?  I can imagine team members with 
> different versions getting into difficulty, especially if the features being 
> skipped change the board layout.
> 
> 2) You are removing strict checking for most of the board but you are adding 
> strict checking for pad options. What's the difference?
> 
> 3) If we keep these options around but don't allow editing/removing, don't we 
> run into a risk of generating a really wonk-y board that only looks wonky in 
> a newer version of KiCad but looks fine in an older version?  For example, 
> imagine we implement rounded corners as a parameter and then a user with an 
> older version of KiCad edits and saves the board.  The rounded corner is only 
> visible in KiCad 6 but the user in KiCad 5 can edit the board to look right 
> to her only to have her colleague use KiCad 6 and see the track violating 
> DRC.  I think that might be counter-intuitive for users but maybe there's a 
> way around it.
> 
> In general, if I understand the use-case, this is to allow users to open 
> newer boards in older KiCad versions?  Is there another use case?
> 
> I think I can see clear to this for options like the 3d-model offset where it 
> could be either inches or mm but there isn't a difference in the actual 
> layout.  Allowing general unrecognized options would seem to open up a worm 
> can in terms of support as in "Please post the KiCad version and the file 
> version in order to figure out the problem." 
> 
> -S
> 
> 
> 
> 2018-03-18 9:46 GMT-07:00 Jeff Young mailto:j...@rokeby.ie>>:
> OK, for your guys’ (re)viewing pleasure, a patch which losslessly round-trips 
> stuff it doesn’t understand.
> 
> 
> 
> 
>> On 16 Mar 2018, at 19:15, hauptmech > > wrote:
>> 
>> While i would still like to see this (my) shim go in, I agree with wayne. 
>> Until/unless a less brittle approach is used for the parser, it's better to 
>> signal a problem painfully and maintain the integrity of the file.
>> 
>> I have to admit that when i first heard the s-expressions idea I assumed 
>> that the intention was to use a lisp like virtual machine for loading and 
>> maintaining design data. I've used vm's for data storage before (lua and 
>> python), and it's great. Parsing is free and you can jam in functions and 
>> other data when needed. 
>> 
>> 
>> 
>> On 17 Mar 2018 07:17, "Jeff Young" mailto:j...@rokeby.ie>> 
>> wrote:
>> Hi Wayne,
>> 
>> Patch reverted.
>> 
>> However, I think your concern is misplaced.  If we want to prevent dataloss 
>> (ie: from reading a 6.0 file into 5.0), then we should warn the user based 
>> on the version string (and give them a chance to say “I still want to open”).
>> 
>> But either way, actually failing to read the file leaves the user in a 
>> pickle (especially when it’s easy enough for them to try out a nightly that 
>> may very well be ahead of their stable).
>> 
>> (And for that reason I think it’s important to put it into 5.0, as otherwise 
>> it won’t help until we start making 7.0 file format changes.)
>> 
>> Cheers,
>> Jeff.
>> 
>> > On 16 Mar 2018, at 18:00, Wayne Stambaugh > > > wrote:
>> >
>> > Jeff,
>> >
>> > Please revert this patch.  Any changes to the board file parser and/or
>> > formatter need to be discussed before the changes are committed.  It has
>> > been the intention that the board parser be pendantic by design to
>> > prevent data loss by ignoring unknown formatting.  Allowing anything
>> > outside of the expected formatting in the board file is not something
>> > that I want to introduce without some discuss

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Seth Hillbrand]

Hi Jeff-

A few questions on how you are implementing this:

1) How does the user know what was skipped?  I can imagine team members
with different versions getting into difficulty, especially if the features
being skipped change the board layout.

2) You are removing strict checking for most of the board but you are
adding strict checking for pad options. What's the difference?

3) If we keep these options around but don't allow editing/removing, don't
we run into a risk of generating a really wonk-y board that only looks
wonky in a newer version of KiCad but looks fine in an older version?  For
example, imagine we implement rounded corners as a parameter and then a
user with an older version of KiCad edits and saves the board.  The rounded
corner is only visible in KiCad 6 but the user in KiCad 5 can edit the
board to look right to her only to have her colleague use KiCad 6 and see
the track violating DRC.  I think that might be counter-intuitive for users
but maybe there's a way around it.

In general, if I understand the use-case, this is to allow users to open
newer boards in older KiCad versions?  Is there another use case?

I think I can see clear to this for options like the 3d-model offset where
it could be either inches or mm but there isn't a difference in the actual
layout.  Allowing general unrecognized options would seem to open up a worm
can in terms of support as in "Please post the KiCad version and the file
version in order to figure out the problem."

-S



2018-03-18 9:46 GMT-07:00 Jeff Young :

> OK, for your guys’ (re)viewing pleasure, a patch which losslessly
> round-trips stuff it doesn’t understand.
>
>
>
>
> On 16 Mar 2018, at 19:15, hauptmech  wrote:
>
> While i would still like to see this (my) shim go in, I agree with wayne.
> Until/unless a less brittle approach is used for the parser, it's better to
> signal a problem painfully and maintain the integrity of the file.
>
> I have to admit that when i first heard the s-expressions idea I assumed
> that the intention was to use a lisp like virtual machine for loading and
> maintaining design data. I've used vm's for data storage before (lua and
> python), and it's great. Parsing is free and you can jam in functions and
> other data when needed.
>
>
>
> On 17 Mar 2018 07:17, "Jeff Young"  wrote:
>
>> Hi Wayne,
>>
>> Patch reverted.
>>
>> However, I think your concern is misplaced.  If we want to prevent
>> dataloss (ie: from reading a 6.0 file into 5.0), then we should warn the
>> user based on the version string (and give them a chance to say “I still
>> want to open”).
>>
>> But either way, actually failing to read the file leaves the user in a
>> pickle (especially when it’s easy enough for them to try out a nightly that
>> may very well be ahead of their stable).
>>
>> (And for that reason I think it’s important to put it into 5.0, as
>> otherwise it won’t help until we start making 7.0 file format changes.)
>>
>> Cheers,
>> Jeff.
>>
>> > On 16 Mar 2018, at 18:00, Wayne Stambaugh  wrote:
>> >
>> > Jeff,
>> >
>> > Please revert this patch.  Any changes to the board file parser and/or
>> > formatter need to be discussed before the changes are committed.  It has
>> > been the intention that the board parser be pendantic by design to
>> > prevent data loss by ignoring unknown formatting.  Allowing anything
>> > outside of the expected formatting in the board file is not something
>> > that I want to introduce without some discussion.  Even should we decide
>> > to accept this patch, I would prefer we put it off until v6.
>> >
>> > That being said, the patch fails to build on windows with following
>> error:
>> >
>> > C:/msys64/home/wstambaugh/src/kicad-trunk/pcbnew/pcb_parser.cpp: In
>> > member function 'void PCB_PARSER::parseUnknown()':
>> > C:/msys64/home/wstambaugh/src/kicad-trunk/pcbnew/pcb_parser.
>> cpp:1269:12:
>> > error: request for member 'LogText' in '__acrt_iob_func(2)', which is of
>> > pointer type  FILE* {aka _iobuf*}' (maybe you meant to use '->' ?)
>> > stderr.LogText( msg );
>> >^~~
>> >
>> > Cheers,
>> >
>> > Wayne
>> >
>> > On 3/16/2018 1:08 PM, Jeff Young wrote:
>> >> Perhaps somewhat germane to this discussion I have removed the
>> strict-format nags from the PCB parser.  It now logs warnings to stderr but
>> otherwise ignores structures it doesn’t understand.
>> >>
>> >> I’m not sure that helps hauptmech much as if the file is subsequently
>> written the unknown markup will be lost, but I thought I’d mention it.
>> >>
>> >> Cheers,
>> >> Jeff.
>> >>
>> >>> On 7 Mar 2018, at 20:12, hauptmech  wrote:
>> >>>
>> >>> Hi Thomasz,
>> >>>
>> >>> I hope I'm able to address you concerns. I'm stuck using kicad stable
>> in many situations. I brought clearances up for discussion last July but no
>> one moved on it, nor are they required to. Clearance management is a major
>> pain point for me so I wrote a fix. This patch will let us (me and the
>> people I collaborate with) work using version 5, but

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Jeff Young]

OK, for your guys’ (re)viewing pleasure, a patch which losslessly round-trips stuff it doesn’t understand.

0001-Lossless-round-tripping-of-unknown-constructs.patch
Description: Binary data
On 16 Mar 2018, at 19:15, hauptmech  wrote:While i would still like to see this (my) shim go in, I agree with wayne. Until/unless a less brittle approach is used for the parser, it's better to signal a problem painfully and maintain the integrity of the file.I have to admit that when i first heard the s-expressions idea I assumed that the intention was to use a lisp like virtual machine for loading and maintaining design data. I've used vm's for data storage before (lua and python), and it's great. Parsing is free and you can jam in functions and other data when needed. On 17 Mar 2018 07:17, "Jeff Young"  wrote:Hi Wayne,

Patch reverted.

However, I think your concern is misplaced.  If we want to prevent dataloss (ie: from reading a 6.0 file into 5.0), then we should warn the user based on the version string (and give them a chance to say “I still want to open”).

But either way, actually failing to read the file leaves the user in a pickle (especially when it’s easy enough for them to try out a nightly that may very well be ahead of their stable).

(And for that reason I think it’s important to put it into 5.0, as otherwise it won’t help until we start making 7.0 file format changes.)

Cheers,
Jeff.

> On 16 Mar 2018, at 18:00, Wayne Stambaugh  wrote:
>
> Jeff,
>
> Please revert this patch.  Any changes to the board file parser and/or
> formatter need to be discussed before the changes are committed.  It has
> been the intention that the board parser be pendantic by design to
> prevent data loss by ignoring unknown formatting.  Allowing anything
> outside of the expected formatting in the board file is not something
> that I want to introduce without some discussion.  Even should we decide
> to accept this patch, I would prefer we put it off until v6.
>
> That being said, the patch fails to build on windows with following error:
>
> C:/msys64/home/wstambaugh/src/kicad-trunk/pcbnew/pcb_parser.cpp: In
> member function 'void PCB_PARSER::parseUnknown()':
> C:/msys64/home/wstambaugh/src/kicad-trunk/pcbnew/pcb_parser.cpp:1269:12:
> error: request for member 'LogText' in '__acrt_iob_func(2)', which is of
> pointer type  FILE* {aka _iobuf*}' (maybe you meant to use '->' ?)
>     stderr.LogText( msg );
>            ^~~
>
> Cheers,
>
> Wayne
>
> On 3/16/2018 1:08 PM, Jeff Young wrote:
>> Perhaps somewhat germane to this discussion I have removed the strict-format nags from the PCB parser.  It now logs warnings to stderr but otherwise ignores structures it doesn’t understand.
>>
>> I’m not sure that helps hauptmech much as if the file is subsequently written the unknown markup will be lost, but I thought I’d mention it.
>>
>> Cheers,
>> Jeff.
>>
>>> On 7 Mar 2018, at 20:12, hauptmech  wrote:
>>>
>>> Hi Thomasz,
>>>
>>> I hope I'm able to address you concerns. I'm stuck using kicad stable in many situations. I brought clearances up for discussion last July but no one moved on it, nor are they required to. Clearance management is a major pain point for me so I wrote a fix. This patch will let us (me and the people I collaborate with) work using version 5, but open and close files written with a version patched with clearance handling code.
>>>
>>> I believe that code exactly like this will go into version 6. Getting it in earlier makes a huge difference to me, so I'm submitting it.
>>>
>>> On 07/03/18 23:30, Tomasz Wlostowski wrote:
 Hi hauptmech,

 I'm sorry but IMHO we can't accept your patch:
 - it changes the file format while we are already in feature freeze.
 This is a way too big change to accept for the V5.
>>>
>>> This patch simply adds tokens to the file format. No clearance data is saved for files that use the netclass only. Files without clearance tokens continue to remain without them.
>>>
>>> Yes it is a backward compatible file format change, but it does no harm to V5 files already in the wild.
>>>
 - we are planning to overhaul the clearance/design rules system in V6.
 Storing the clearance *DIRECTLY* for each track segment/via will
 conflict with any more sophisticated design rule management system.

>>> I'm glad you are planning this. I am sure that regardless of the sophistication of the rule system, you will store clearance directly for exactly the same reason that track width is stored directly now. There are always exceptions to the rules.
>>>
>>> If kicad choose a direction that does not store clearances per item, then it is easy to rip these few lines back out.
>>>
>>> Did this answer your existing concerns about this patch? Are there any other concerns you have about this patch?
>>>
>>>
>>>
>>> ___
>>> Mailing list: https://launchpad.net/~kicad-

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[hauptmech]

While i would still like to see this (my) shim go in, I agree with wayne.
Until/unless a less brittle approach is used for the parser, it's better to
signal a problem painfully and maintain the integrity of the file.

I have to admit that when i first heard the s-expressions idea I assumed
that the intention was to use a lisp like virtual machine for loading and
maintaining design data. I've used vm's for data storage before (lua and
python), and it's great. Parsing is free and you can jam in functions and
other data when needed.



On 17 Mar 2018 07:17, "Jeff Young"  wrote:

> Hi Wayne,
>
> Patch reverted.
>
> However, I think your concern is misplaced.  If we want to prevent
> dataloss (ie: from reading a 6.0 file into 5.0), then we should warn the
> user based on the version string (and give them a chance to say “I still
> want to open”).
>
> But either way, actually failing to read the file leaves the user in a
> pickle (especially when it’s easy enough for them to try out a nightly that
> may very well be ahead of their stable).
>
> (And for that reason I think it’s important to put it into 5.0, as
> otherwise it won’t help until we start making 7.0 file format changes.)
>
> Cheers,
> Jeff.
>
> > On 16 Mar 2018, at 18:00, Wayne Stambaugh  wrote:
> >
> > Jeff,
> >
> > Please revert this patch.  Any changes to the board file parser and/or
> > formatter need to be discussed before the changes are committed.  It has
> > been the intention that the board parser be pendantic by design to
> > prevent data loss by ignoring unknown formatting.  Allowing anything
> > outside of the expected formatting in the board file is not something
> > that I want to introduce without some discussion.  Even should we decide
> > to accept this patch, I would prefer we put it off until v6.
> >
> > That being said, the patch fails to build on windows with following
> error:
> >
> > C:/msys64/home/wstambaugh/src/kicad-trunk/pcbnew/pcb_parser.cpp: In
> > member function 'void PCB_PARSER::parseUnknown()':
> > C:/msys64/home/wstambaugh/src/kicad-trunk/pcbnew/pcb_parser.cpp:1269:12:
> > error: request for member 'LogText' in '__acrt_iob_func(2)', which is of
> > pointer type  FILE* {aka _iobuf*}' (maybe you meant to use '->' ?)
> > stderr.LogText( msg );
> >^~~
> >
> > Cheers,
> >
> > Wayne
> >
> > On 3/16/2018 1:08 PM, Jeff Young wrote:
> >> Perhaps somewhat germane to this discussion I have removed the
> strict-format nags from the PCB parser.  It now logs warnings to stderr but
> otherwise ignores structures it doesn’t understand.
> >>
> >> I’m not sure that helps hauptmech much as if the file is subsequently
> written the unknown markup will be lost, but I thought I’d mention it.
> >>
> >> Cheers,
> >> Jeff.
> >>
> >>> On 7 Mar 2018, at 20:12, hauptmech  wrote:
> >>>
> >>> Hi Thomasz,
> >>>
> >>> I hope I'm able to address you concerns. I'm stuck using kicad stable
> in many situations. I brought clearances up for discussion last July but no
> one moved on it, nor are they required to. Clearance management is a major
> pain point for me so I wrote a fix. This patch will let us (me and the
> people I collaborate with) work using version 5, but open and close files
> written with a version patched with clearance handling code.
> >>>
> >>> I believe that code exactly like this will go into version 6. Getting
> it in earlier makes a huge difference to me, so I'm submitting it.
> >>>
> >>> On 07/03/18 23:30, Tomasz Wlostowski wrote:
>  Hi hauptmech,
> 
>  I'm sorry but IMHO we can't accept your patch:
>  - it changes the file format while we are already in feature freeze.
>  This is a way too big change to accept for the V5.
> >>>
> >>> This patch simply adds tokens to the file format. No clearance data is
> saved for files that use the netclass only. Files without clearance tokens
> continue to remain without them.
> >>>
> >>> Yes it is a backward compatible file format change, but it does no
> harm to V5 files already in the wild.
> >>>
>  - we are planning to overhaul the clearance/design rules system in V6.
>  Storing the clearance *DIRECTLY* for each track segment/via will
>  conflict with any more sophisticated design rule management system.
> 
> >>> I'm glad you are planning this. I am sure that regardless of the
> sophistication of the rule system, you will store clearance directly for
> exactly the same reason that track width is stored directly now. There are
> always exceptions to the rules.
> >>>
> >>> If kicad choose a direction that does not store clearances per item,
> then it is easy to rip these few lines back out.
> >>>
> >>> Did this answer your existing concerns about this patch? Are there any
> other concerns you have about this patch?
> >>>
> >>>
> >>>
> >>> ___
> >>> Mailing list: https://launchpad.net/~kicad-developers
> >>> Post to : kicad-developers@lists.launchpad.net
> >>> Unsubscribe : https://launchpa

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Jeff Young]

Hi Wayne,

Patch reverted.

However, I think your concern is misplaced.  If we want to prevent dataloss 
(ie: from reading a 6.0 file into 5.0), then we should warn the user based on 
the version string (and give them a chance to say “I still want to open”).

But either way, actually failing to read the file leaves the user in a pickle 
(especially when it’s easy enough for them to try out a nightly that may very 
well be ahead of their stable).

(And for that reason I think it’s important to put it into 5.0, as otherwise it 
won’t help until we start making 7.0 file format changes.)

Cheers,
Jeff.

> On 16 Mar 2018, at 18:00, Wayne Stambaugh  wrote:
> 
> Jeff,
> 
> Please revert this patch.  Any changes to the board file parser and/or
> formatter need to be discussed before the changes are committed.  It has
> been the intention that the board parser be pendantic by design to
> prevent data loss by ignoring unknown formatting.  Allowing anything
> outside of the expected formatting in the board file is not something
> that I want to introduce without some discussion.  Even should we decide
> to accept this patch, I would prefer we put it off until v6.
> 
> That being said, the patch fails to build on windows with following error:
> 
> C:/msys64/home/wstambaugh/src/kicad-trunk/pcbnew/pcb_parser.cpp: In
> member function 'void PCB_PARSER::parseUnknown()':
> C:/msys64/home/wstambaugh/src/kicad-trunk/pcbnew/pcb_parser.cpp:1269:12:
> error: request for member 'LogText' in '__acrt_iob_func(2)', which is of
> pointer type  FILE* {aka _iobuf*}' (maybe you meant to use '->' ?)
> stderr.LogText( msg );
>^~~
> 
> Cheers,
> 
> Wayne
> 
> On 3/16/2018 1:08 PM, Jeff Young wrote:
>> Perhaps somewhat germane to this discussion I have removed the strict-format 
>> nags from the PCB parser.  It now logs warnings to stderr but otherwise 
>> ignores structures it doesn’t understand.
>> 
>> I’m not sure that helps hauptmech much as if the file is subsequently 
>> written the unknown markup will be lost, but I thought I’d mention it.
>> 
>> Cheers,
>> Jeff.
>> 
>>> On 7 Mar 2018, at 20:12, hauptmech  wrote:
>>> 
>>> Hi Thomasz,
>>> 
>>> I hope I'm able to address you concerns. I'm stuck using kicad stable in 
>>> many situations. I brought clearances up for discussion last July but no 
>>> one moved on it, nor are they required to. Clearance management is a major 
>>> pain point for me so I wrote a fix. This patch will let us (me and the 
>>> people I collaborate with) work using version 5, but open and close files 
>>> written with a version patched with clearance handling code.
>>> 
>>> I believe that code exactly like this will go into version 6. Getting it in 
>>> earlier makes a huge difference to me, so I'm submitting it.
>>> 
>>> On 07/03/18 23:30, Tomasz Wlostowski wrote:
 Hi hauptmech,
 
 I'm sorry but IMHO we can't accept your patch:
 - it changes the file format while we are already in feature freeze.
 This is a way too big change to accept for the V5.
>>> 
>>> This patch simply adds tokens to the file format. No clearance data is 
>>> saved for files that use the netclass only. Files without clearance tokens 
>>> continue to remain without them.
>>> 
>>> Yes it is a backward compatible file format change, but it does no harm to 
>>> V5 files already in the wild.
>>> 
 - we are planning to overhaul the clearance/design rules system in V6.
 Storing the clearance *DIRECTLY* for each track segment/via will
 conflict with any more sophisticated design rule management system.
 
>>> I'm glad you are planning this. I am sure that regardless of the 
>>> sophistication of the rule system, you will store clearance directly for 
>>> exactly the same reason that track width is stored directly now. There are 
>>> always exceptions to the rules.
>>> 
>>> If kicad choose a direction that does not store clearances per item, then 
>>> it is easy to rip these few lines back out.
>>> 
>>> Did this answer your existing concerns about this patch? Are there any 
>>> other concerns you have about this patch?
>>> 
>>> 
>>> 
>>> ___
>>> Mailing list: https://launchpad.net/~kicad-developers
>>> Post to : kicad-developers@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>> More help   : https://help.launchpad.net/ListHelp
>> 
>> 
>> ___
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to : kicad-developers@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
>> 
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Wayne Stambaugh]

Jeff,

Please revert this patch.  Any changes to the board file parser and/or
formatter need to be discussed before the changes are committed.  It has
been the intention that the board parser be pendantic by design to
prevent data loss by ignoring unknown formatting.  Allowing anything
outside of the expected formatting in the board file is not something
that I want to introduce without some discussion.  Even should we decide
to accept this patch, I would prefer we put it off until v6.

That being said, the patch fails to build on windows with following error:

C:/msys64/home/wstambaugh/src/kicad-trunk/pcbnew/pcb_parser.cpp: In
member function 'void PCB_PARSER::parseUnknown()':
C:/msys64/home/wstambaugh/src/kicad-trunk/pcbnew/pcb_parser.cpp:1269:12:
error: request for member 'LogText' in '__acrt_iob_func(2)', which is of
pointer type  FILE* {aka _iobuf*}' (maybe you meant to use '->' ?)
 stderr.LogText( msg );
^~~

Cheers,

Wayne

On 3/16/2018 1:08 PM, Jeff Young wrote:
> Perhaps somewhat germane to this discussion I have removed the strict-format 
> nags from the PCB parser.  It now logs warnings to stderr but otherwise 
> ignores structures it doesn’t understand.
> 
> I’m not sure that helps hauptmech much as if the file is subsequently written 
> the unknown markup will be lost, but I thought I’d mention it.
> 
> Cheers,
> Jeff.
> 
>> On 7 Mar 2018, at 20:12, hauptmech  wrote:
>>
>> Hi Thomasz,
>>
>> I hope I'm able to address you concerns. I'm stuck using kicad stable in 
>> many situations. I brought clearances up for discussion last July but no one 
>> moved on it, nor are they required to. Clearance management is a major pain 
>> point for me so I wrote a fix. This patch will let us (me and the people I 
>> collaborate with) work using version 5, but open and close files written 
>> with a version patched with clearance handling code.
>>
>> I believe that code exactly like this will go into version 6. Getting it in 
>> earlier makes a huge difference to me, so I'm submitting it.
>>
>> On 07/03/18 23:30, Tomasz Wlostowski wrote:
>>> Hi hauptmech,
>>>
>>> I'm sorry but IMHO we can't accept your patch:
>>> - it changes the file format while we are already in feature freeze.
>>> This is a way too big change to accept for the V5.
>>
>> This patch simply adds tokens to the file format. No clearance data is saved 
>> for files that use the netclass only. Files without clearance tokens 
>> continue to remain without them.
>>
>> Yes it is a backward compatible file format change, but it does no harm to 
>> V5 files already in the wild.
>>
>>> - we are planning to overhaul the clearance/design rules system in V6.
>>> Storing the clearance *DIRECTLY* for each track segment/via will
>>> conflict with any more sophisticated design rule management system.
>>>
>> I'm glad you are planning this. I am sure that regardless of the 
>> sophistication of the rule system, you will store clearance directly for 
>> exactly the same reason that track width is stored directly now. There are 
>> always exceptions to the rules.
>>
>> If kicad choose a direction that does not store clearances per item, then it 
>> is easy to rip these few lines back out.
>>
>> Did this answer your existing concerns about this patch? Are there any other 
>> concerns you have about this patch?
>>
>>
>>
>> ___
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to : kicad-developers@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
> 
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Jeff Young]

Perhaps somewhat germane to this discussion I have removed the strict-format 
nags from the PCB parser.  It now logs warnings to stderr but otherwise ignores 
structures it doesn’t understand.

I’m not sure that helps hauptmech much as if the file is subsequently written 
the unknown markup will be lost, but I thought I’d mention it.

Cheers,
Jeff.

> On 7 Mar 2018, at 20:12, hauptmech  wrote:
> 
> Hi Thomasz,
> 
> I hope I'm able to address you concerns. I'm stuck using kicad stable in many 
> situations. I brought clearances up for discussion last July but no one moved 
> on it, nor are they required to. Clearance management is a major pain point 
> for me so I wrote a fix. This patch will let us (me and the people I 
> collaborate with) work using version 5, but open and close files written with 
> a version patched with clearance handling code.
> 
> I believe that code exactly like this will go into version 6. Getting it in 
> earlier makes a huge difference to me, so I'm submitting it.
> 
> On 07/03/18 23:30, Tomasz Wlostowski wrote:
>> Hi hauptmech,
>> 
>> I'm sorry but IMHO we can't accept your patch:
>> - it changes the file format while we are already in feature freeze.
>> This is a way too big change to accept for the V5.
> 
> This patch simply adds tokens to the file format. No clearance data is saved 
> for files that use the netclass only. Files without clearance tokens continue 
> to remain without them.
> 
> Yes it is a backward compatible file format change, but it does no harm to V5 
> files already in the wild.
> 
>> - we are planning to overhaul the clearance/design rules system in V6.
>> Storing the clearance *DIRECTLY* for each track segment/via will
>> conflict with any more sophisticated design rule management system.
>> 
> I'm glad you are planning this. I am sure that regardless of the 
> sophistication of the rule system, you will store clearance directly for 
> exactly the same reason that track width is stored directly now. There are 
> always exceptions to the rules.
> 
> If kicad choose a direction that does not store clearances per item, then it 
> is easy to rip these few lines back out.
> 
> Did this answer your existing concerns about this patch? Are there any other 
> concerns you have about this patch?
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[hauptmech]

Hi Thomasz,

I hope I'm able to address you concerns. I'm stuck using kicad stable in 
many situations. I brought clearances up for discussion last July but no 
one moved on it, nor are they required to. Clearance management is a 
major pain point for me so I wrote a fix. This patch will let us (me and 
the people I collaborate with) work using version 5, but open and close 
files written with a version patched with clearance handling code.


I believe that code exactly like this will go into version 6. Getting it 
in earlier makes a huge difference to me, so I'm submitting it.


On 07/03/18 23:30, Tomasz Wlostowski wrote:

Hi hauptmech,

I'm sorry but IMHO we can't accept your patch:
- it changes the file format while we are already in feature freeze.
This is a way too big change to accept for the V5.


This patch simply adds tokens to the file format. No clearance data is 
saved for files that use the netclass only. Files without clearance 
tokens continue to remain without them.


Yes it is a backward compatible file format change, but it does no harm 
to V5 files already in the wild.



- we are planning to overhaul the clearance/design rules system in V6.
Storing the clearance *DIRECTLY* for each track segment/via will
conflict with any more sophisticated design rule management system.

I'm glad you are planning this. I am sure that regardless of the 
sophistication of the rule system, you will store clearance directly for 
exactly the same reason that track width is stored directly now. There 
are always exceptions to the rules.


If kicad choose a direction that does not store clearances per item, 
then it is easy to rip these few lines back out.


Did this answer your existing concerns about this patch? Are there any 
other concerns you have about this patch?




___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Clemens Koller]

Hi!

On 2018-03-07 13:44, Tomasz Wlostowski wrote:
> I'm not totally opposed to hauptmech's change, I just think it's too
> early to merge it. We should discuss such ideas before writing code. My
> goal for V6 DRC is to be able to define clearance rules for at least:
> - per-net & per-netclass
> - per-layer
> - user-defined board areas.

Add: - for components
Higher priority rules override lower priority ones. Priorities are predefined 
and/or can be changed manually.
That's not extremely complex but it still needs a lot of work and proper 
planning beforehand.

Regards,

Clemens

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[jp charras]

Le 07/03/2018 à 14:02, Wayne Stambaugh a écrit :
> On 3/7/2018 7:44 AM, Tomasz Wlostowski wrote:
>> On 07/03/18 12:10, Russell Oliver wrote:
>>> Forgive my ignorance but why would storing the clearance for each track
>>> segment (if required to by design intent) conflict with a sophisticated
>>> design rule management system? 
>>
>> For example: what would happen to the clearance of the segment/via if it
>> has been modified by the P&S shove? What clearance should be taken if
>> new segments are generated by the P&S optimizer from multiple original
>> segments with different manually-set clearances?
>>
>> I'm not totally opposed to hauptmech's change, I just think it's too
>> early to merge it. We should discuss such ideas before writing code. My
>> goal for V6 DRC is to be able to define clearance rules for at least:
>> - per-net & per-netclass
>> - per-layer
>> - user-defined board areas.
>>
>> The latter two features IMHO will be more powerful than manually
>> overriding each segments' or via's clearance.
>>
>> Tom
>>
> 
> If there are other requirements, we should clearly define them before
> writing any new clearance code.  It will save us a lot of grief in the
> long run.
> 
> Wayne

Sure!
At least one other requirement is known: having a clearance value for external 
layers, and an other
value for internal layers.
There are also a few known other requirements (related to edge cuts and margin 
layer).


-- 
Jean-Pierre CHARRAS

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Tomasz Wlostowski]

On 07/03/18 14:02, Wayne Stambaugh wrote:
> It will save us a lot of grief in the
> long run.

I can't agree more!

Tom

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Wayne Stambaugh]

On 3/7/2018 7:44 AM, Tomasz Wlostowski wrote:
> On 07/03/18 12:10, Russell Oliver wrote:
>> Forgive my ignorance but why would storing the clearance for each track
>> segment (if required to by design intent) conflict with a sophisticated
>> design rule management system? 
> 
> For example: what would happen to the clearance of the segment/via if it
> has been modified by the P&S shove? What clearance should be taken if
> new segments are generated by the P&S optimizer from multiple original
> segments with different manually-set clearances?
> 
> I'm not totally opposed to hauptmech's change, I just think it's too
> early to merge it. We should discuss such ideas before writing code. My
> goal for V6 DRC is to be able to define clearance rules for at least:
> - per-net & per-netclass
> - per-layer
> - user-defined board areas.
> 
> The latter two features IMHO will be more powerful than manually
> overriding each segments' or via's clearance.
> 
> Tom
> 

If there are other requirements, we should clearly define them before
writing any new clearance code.  It will save us a lot of grief in the
long run.

Wayne

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Tomasz Wlostowski]

On 07/03/18 12:10, Russell Oliver wrote:
> Forgive my ignorance but why would storing the clearance for each track
> segment (if required to by design intent) conflict with a sophisticated
> design rule management system? 

For example: what would happen to the clearance of the segment/via if it
has been modified by the P&S shove? What clearance should be taken if
new segments are generated by the P&S optimizer from multiple original
segments with different manually-set clearances?

I'm not totally opposed to hauptmech's change, I just think it's too
early to merge it. We should discuss such ideas before writing code. My
goal for V6 DRC is to be able to define clearance rules for at least:
- per-net & per-netclass
- per-layer
- user-defined board areas.

The latter two features IMHO will be more powerful than manually
overriding each segments' or via's clearance.

Tom


___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Russell Oliver]

Forgive my ignorance but why would storing the clearance for each track
segment (if required to by design intent) conflict with a sophisticated
design rule management system?
As a general approach shouldn't KiCad allow freedom of design when the
intention of the designer is clear?

Russell




On Wed, Mar 7, 2018 at 9:30 PM Tomasz Wlostowski 
wrote:

> On 07/03/18 11:02, hauptmech wrote:
> > I have a patch for treating track clearance the same as track width. The
> > user can already specify a track width that is an exception to the
> > netclass width, and now do the same for clearance. (
> > https://youtu.be/05vfAvYwDio )
>
> Hi hauptmech,
>
> I'm sorry but IMHO we can't accept your patch:
> - it changes the file format while we are already in feature freeze.
> This is a way too big change to accept for the V5.
> - we are planning to overhaul the clearance/design rules system in V6.
> Storing the clearance *DIRECTLY* for each track segment/via will
> conflict with any more sophisticated design rule management system.
>
> Tom
>
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [PATCH] - File format shim for clearance data

[Tomasz Wlostowski]

On 07/03/18 11:02, hauptmech wrote:
> I have a patch for treating track clearance the same as track width. The
> user can already specify a track width that is an exception to the
> netclass width, and now do the same for clearance. (
> https://youtu.be/05vfAvYwDio )

Hi hauptmech,

I'm sorry but IMHO we can't accept your patch:
- it changes the file format while we are already in feature freeze.
This is a way too big change to accept for the V5.
- we are planning to overhaul the clearance/design rules system in V6.
Storing the clearance *DIRECTLY* for each track segment/via will
conflict with any more sophisticated design rule management system.

Tom

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

[Kicad-developers] [PATCH] - File format shim for clearance data

[hauptmech]

I have a patch for treating track clearance the same as track width. The 
user can already specify a track width that is an exception to the 
netclass width, and now do the same for clearance. ( 
https://youtu.be/05vfAvYwDio )


This is not that patch.

This is a small shim for the file format that will allow reading and 
writing clearance data from the kicad_pcb file without causing a file 
parse error. This will allow us to transparently load and save files 
with clearance data using version 5 stable.


There are 2 patch files. The first deals with the clearance data. It 
adds a (clearance ) field to vias and segments that is only populated if 
the item has a non-netclass clearance. The second patch preserves user 
clearances. It adds a (user_clearance ) field to setup to compliment the 
(user_width ) already there. As stated in the patch title, the 1st patch 
is enough to make a significant difference for my team.


Issues:

    TRACK gains an int sized member. There are hints in the source 
comments that this was a planned improvement all along. Meanwhile some 
may be concerned about impact on memory usage. If anyone is, I'm happy 
to do a memory impact analysis.


    I reduced the patch to the smallest one that would make a 
meaningful impact on the work we do here. Easy to review and most likely 
to be acceptable given the point in the release cycle we are. The wider 
impact of this patch is therefore limited and as a result it might be 
just our team that benefits.


If it's not clear, some of our work computers are locked to the stable 
kicad version, while others are not. We gain benefit from being able to 
do flexible clearance layout on the non-locked computers and still edit 
those files on a stable version of kicad. If you would prefer to look at 
the minimum patch which allows flexible clearance layout, I am happy to 
submit that.



-hauptmech


>From 08165106624d2940ae4507b8981298d58f4ae1a2 Mon Sep 17 00:00:00 2001
From: hauptmech 
Date: Wed, 7 Mar 2018 13:12:20 +1300
Subject: [PATCH 1/2] Minimum viable shim for kicad file format to handle track
 clearances
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="2.16.2"

This is a multi-part message in MIME format.
--2.16.2
Content-Type: text/plain; charset=UTF-8; format=fixed
Content-Transfer-Encoding: 8bit

---
 common/pcb.keywords |  1 +
 pcbnew/class_track.cpp  |  1 +
 pcbnew/class_track.h| 22 ++
 pcbnew/kicad_plugin.cpp |  6 ++
 pcbnew/pcb_parser.cpp   | 14 ++
 5 files changed, 44 insertions(+)


--2.16.2
Content-Type: text/x-patch; name="0001-Minimum-viable-shim-for-kicad-file-format-to-handle-.patch"
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment; filename="0001-Minimum-viable-shim-for-kicad-file-format-to-handle-.patch"

diff --git a/common/pcb.keywords b/common/pcb.keywords
index 1998f9ab5..c36971085 100644
--- a/common/pcb.keywords
+++ b/common/pcb.keywords
@@ -187,6 +187,7 @@ thru_hole_only
 tstamp
 unlocked
 user
+user_clearance
 user_trace_width
 user_via
 uvia_dia
diff --git a/pcbnew/class_track.cpp b/pcbnew/class_track.cpp
index 1c5f69be7..9fd08dae0 100644
--- a/pcbnew/class_track.cpp
+++ b/pcbnew/class_track.cpp
@@ -99,6 +99,7 @@ TRACK::TRACK( BOARD_ITEM* aParent, KICAD_T idtype ) :
 m_Width = Millimeter2iu( 0.2 );
 start   = end = NULL;
 m_Param = 0;
+m_Clearance = NETCLASS_HAS_CLEARANCE; // Netlist drives clearance by default
 }
 
 
diff --git a/pcbnew/class_track.h b/pcbnew/class_track.h
index 1da81e6ff..0de9014b4 100644
--- a/pcbnew/class_track.h
+++ b/pcbnew/class_track.h
@@ -60,6 +60,8 @@ enum VIATYPE_T
 
 #define MIN_VIA_DRAW_SIZE  4   /// Minimum size in pixel for full drawing
 
+#define NETCLASS_HAS_CLEARANCE-1
+
 /**
  * Function GetTrack
  * is a helper function to locate a trace segment having an end point at \a aPosition
@@ -280,6 +282,25 @@ public:
 return wxT( "TRACK" );
 }
 
+/**
+ * Function HasClearance
+ * Return false if clearance is provided by the netclass, otherwise true.
+ */
+bool HasClearance() const
+{
+return m_Clearance != NETCLASS_HAS_CLEARANCE;
+}
+
+void SetClearance( int aClearance )
+{
+m_Clearance = aClearance;
+}
+
+int GetInternalClearance() const
+{
+return m_Clearance;
+}
+
 /**
  * Function GetClearance
  * returns the clearance in internal units.  If \a aItem is not NULL then the
@@ -333,6 +354,7 @@ protected:
 void DrawShortNetname( EDA_DRAW_PANEL* panel, wxDC* aDC, GR_DRAWMODE aDrawMode,
 COLOR4D aBgColor );
 
+int m_Clearance;///< Clearance between this and other elements
 int m_Width;///< Thickness of track, or via diameter
 wxPoint m_Start;///< Line start point
 wxPoint m_End;  ///< Line end point
diff --git a/pcbnew/kicad_plugin.cpp b/pcbnew/kicad_plug