Re: [Kicad-developers] [RFC] Symbol library file format

[mitjan696-ubu...@yahoo.co.uk]

 Hi!
This would also enable fully automated PDN analysis (e.g. a voltage regulator 
could have property "V_OUT" with value "5.0" on the output pin. An IC connected 
to this net would have the property "I_IN" with value "0.2" and property 
"V_IN_MIN" with value "4.5". When PCB would be done then outside tool could 
parse pcb and sch files and get all required info to do the PDN - at least the 
DC part of it).

On Friday, 4 January 2019, 19:56:10 CET, Wayne Stambaugh 
 wrote:  
 
 I'm willing to add properties which are already defined to pins.  There
has been talk about adding about adding electrical constraints for
advance ERC testing to symbols but that will not be part of the v6
implementation of the new file format.  It's going to be a lot of work
to implement the features we've added to v6 so this would be outside the
scope of the first version of the new file format.  I'm certainly open
to discussing this as part of a later version of kicad.

On 1/2/19 3:52 PM, mark.vandoesb...@hetnet.nl wrote:
> One thing I would like to have is user defined pin properties, just like
> the part properties. Currently I use the BOM generator to generate a
> software header file from the netlist. I have to use an additional file
> to store the information I need. It would be nice if this information
> can be stored in the symbol. This could be any kind of information, for
> example bank number or clock region for an FPGA. Or driver strenght/speed
> setting for a microcontroller.
> 
> regards,
> 
> Mark.
> 
> Wayne Stambaugh  wrote:
> 
>     I have updated and published the symbol file format[1] for comment.
>     Hopefully there isn't too much to change.  The only thing to really
>     finalize is the internal units.  The initial concept was unitless but
>     the more I think about it and discuss with other developers, it makes
>     more sense to use units for the following reasons:
> 
>     1. It's easier to visualize in your head how the symbols on a given page
>     size will layout.
> 
>     2. Converting from other file formats (Eagle, Altium, etc) will be
>     easier since most if not all of them have a defined unit.
> 
>     I'm thinking 10u (or possibly 100u) will make a good internal units
>     value.  Once we nail down the units, I will update the file format
>     document accordingly.
> 
>     Please keep in mind that this is the symbol library file format document
>     so things like constraints belong in the schematic file format.  I will
>     be posting the schematic file format as soon as I finish updating it.
> 
>     Cheers,
> 
>     Wayne
> 
>     [1]:
>     
> https://docs.google.com/document/d/1lyL_8FWZRouMkwqLiIt84rd2Htg4v1vz8_2MzRKHRkc/edit
> 
> 
>     ___
>     Mailing list: https://launchpad.net/~kicad-developers
>     Post to    : kicad-developers@lists.launchpad.net
>     Unsubscribe : https://launchpad.net/~kicad-developers
>     More help  : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to    : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help  : https://help.launchpad.net/ListHelp
  ___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Rene Pöschl]

On 04/01/19 15:10, Wayne Stambaugh wrote:


Is there a way to mark a unit as "power unit" (I think you mentioned
sometime back that this might be required to properly make simulation
for multi unit symbols possible while still allowing the power unit to
be separate. A power unit would also be a "must be placed unit" but i
feel separating these two might add more flexibility.)


See definition above.  I'm not sure having a "required" keyword is
useful.  Do you have an example in mind.



There are many parts that not only have power connections but also some
control connections that are necessary for the function of the part.
A lot of these have the control pins shared between many units so it
makes sense to move them to a specialized unit. That unit would then be
required but does not fulfill the "this is a power unit" thing. Hence my
suggestion for a more general "this unit must be placed" vs. "This unit
is optional" field. (The "This is a power unit" thing is not really
necessary for ERC. At least not if there is a "this is a required unit"
flag. The only reason i suggested that is because you mentioned that it
might be required for simulation purposes.)


___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Andrew Lutsenko]

With the amount of data that KiCad handles performance of particular parser
has minuscule impact. File IO delays would thwart any processing time.
What Simon talked about is as he said a separate discussion and is
orthogonal to the decision of how to define data model and what file format
to use. Also I don't agree that there is a dichotomy of "save full memory
state" or "save only bare protocol data". There is lots of room for
compromise on what things on top of bare data you want to save to disk to
speed up restoring full memory state on load. Spatial indexes, if you
choose to save them, would have to be described in terms of another
structure in the IDL, it's not generator's responsibility to understand
what the index represents. But anyway - that's a separate discussion.

I understand that you are cautious to bring in new dependency that most
devs are not familiar with. Is there anything I can do to break the ice?
I was planning to translate the current schematic file format proposal into
proto definitions to show how easy they are to use. I could also try to add
a cmake target that would compile those proto definitions in ready to use
.h and .cpp files. Will you reconsider if I show these examples?

Do you have other concerns about requirements that are potentially not met
aside from performance? I can try to answer any questions you have.
If perf is the only one I could also possibly write some sort of benchmark
comparing parsing performance of KiCad's current s-expr parser and
protobufs. But that is not a small commitment so I don't want to do it if I
can't change your mind.

Regards,
Andrew


On Sat, Jan 5, 2019 at 5:17 AM Wayne Stambaugh  wrote:

> On 1/3/19 11:24 PM, Simon Richter wrote:
> > Hi,
> >
> > On 03.01.19 19:06, José Ignacio wrote:
> >
> >> I
> >> think useful comments to the proposed format should see beyond the
> >> actual low level representation of the data and talk about the overall
> >> model being used to store it.
> >
> > tl;dr: That's a separate discussion.
> >
> > There are two schools of thought here, one that treats saved data as a
> > protocol between two black box instances, and one that treats it as a
> > serialization of the internal state.
> >
> > Both have advantages and disadvantages. The "protocol" approach allows
> > changing internals more easily, and gives better compatibility between
> > versions as changes to the file format have to be made deliberately,
> > while the "serialization" approach gives us load/save basically for
> > free, so we need a lot less code.
> >
> > The "serialization" model also requires us to generate the internal data
> > structures from a more constrained language like IDL, as the marshaller
> > needs to know when to follow pointers, and what members of an array are
> > actually valid.
> >
> > I'm not sure there are generators that include support for spatial
> > indexes, though, which is pretty much a requirement for fast rendering,
> > so this is pretty much impossible at the moment, which places us in
> > "hand written load/save code" territory anyway.
> >
> > We have also ignored diff/merge capability so far, which I believe is a
> > good thing because it cannot really be done on a textual level (schemas
> > are two-dimensional, PCBs are three-dimensional, so there is no normal
> > form with a consistent ordering of elements that will make the files
> > diffable).
> >
> > I fully expect both the internal model and the file format to change
> > significantly in the coming years as new features are added. The main
> > requirement for the file format is that it always needs to be possible
> > to read older files in some way, and to recognize when a file is newer
> > than the current parser understands.
> >
> > It might be a good idea to also have an "extension" mechanism, where we
> > don't increase the version number when adding a new feature, but rather
> > mark files that actually use the new feature, so files written by newer
> > versions that don't use one of the newer functions can be read by older
> > versions. This would also be another point for "hand written" load/save.
> >
> >Simon
> >
>
> I've done some more investigation on this and I'm not convinced that
> this is any better than our current implementation due to some of the
> issues Simon mentioned above.  I've also discovered there is also a
> significant performance hit[1] with the JSON reader.  This is important
> due to the overhead required when loading libraries.  I appreciate the
> information and protobufs are interesting but I am going to stick with
> what we have and know versus what we don't know because I don't think we
> have the extra manpower to experiment only to find out it doesn't suite
> our requirements.
>
> Cheers,
>
> Wayne
>
> [1]:
>
> https://groups.google.com/forum/#!searchin/protobuf/JSON|sort:date/protobuf/23Slq4AX7oE/xuwQX_ZXAAAJ
>
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> 

Re: [Kicad-developers] [RFC] Symbol library file format

[Thomas Pointhuber]

My ERC Proposal was more like a: we should at least think about it a bit
instead of writing the pin type near style and number/name. We can
specify ERC hints in a future-proof way without extending the current
implementation. Like:

(erc_hints (pin_type power_in))

This would allow us to specify new erc hints in the future without
splitting them up at different places in KiCad 7.

I'm in favor of pin properties. In fact I would like to have the ability
to specify properties for all important items. (Pin, Symbol, Net,
Sub-schematic, Label,...). Those can be used by users and 3rd-party
programs, as well as to quickly prototype new stuff like ERC rules.

Regards,
Thomas

Am 04.01.19 um 19:55 schrieb Wayne Stambaugh:
> I'm willing to add properties which are already defined to pins.  There
> has been talk about adding about adding electrical constraints for
> advance ERC testing to symbols but that will not be part of the v6
> implementation of the new file format.  It's going to be a lot of work
> to implement the features we've added to v6 so this would be outside the
> scope of the first version of the new file format.  I'm certainly open
> to discussing this as part of a later version of kicad.
> 
> On 1/2/19 3:52 PM, mark.vandoesb...@hetnet.nl wrote:
>> One thing I would like to have is user defined pin properties, just like
>> the part properties. Currently I use the BOM generator to generate a
>> software header file from the netlist. I have to use an additional file
>> to store the information I need. It would be nice if this information
>> can be stored in the symbol. This could be any kind of information, for
>> example bank number or clock region for an FPGA. Or driver strenght/speed
>> setting for a microcontroller.
>>
>> regards,
>>
>> Mark.
>>
>> Wayne Stambaugh  wrote:
>>
>>  I have updated and published the symbol file format[1] for comment.
>>  Hopefully there isn't too much to change.  The only thing to really
>>  finalize is the internal units.  The initial concept was unitless but
>>  the more I think about it and discuss with other developers, it makes
>>  more sense to use units for the following reasons:
>>
>>  1. It's easier to visualize in your head how the symbols on a given page
>>  size will layout.
>>
>>  2. Converting from other file formats (Eagle, Altium, etc) will be
>>  easier since most if not all of them have a defined unit.
>>
>>  I'm thinking 10u (or possibly 100u) will make a good internal units
>>  value.  Once we nail down the units, I will update the file format
>>  document accordingly.
>>
>>  Please keep in mind that this is the symbol library file format document
>>  so things like constraints belong in the schematic file format.  I will
>>  be posting the schematic file format as soon as I finish updating it.
>>
>>  Cheers,
>>
>>  Wayne
>>
>>  [1]:
>>  
>> https://docs.google.com/document/d/1lyL_8FWZRouMkwqLiIt84rd2Htg4v1vz8_2MzRKHRkc/edit
>>
>>
>>  ___
>>  Mailing list: https://launchpad.net/~kicad-developers
>>  Post to : kicad-developers@lists.launchpad.net
>>  Unsubscribe : https://launchpad.net/~kicad-developers
>>  More help   : https://help.launchpad.net/ListHelp
>>
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Wayne Stambaugh]

Hi Mario,

On 1/4/19 10:53 AM, Mário Luzeiro wrote:
>> Now if you create a unique part number for every
>> resistance value, tolerance, temperature coefficient, manufacturer, etc.
>> that would be tens if not hundreds of thousands of unique part numbers.
> 
> I know my intervention may be a bit offtopic, but it may add some other 
> elements to discussion:
> 
> On such case of the 3D variants, it could be considered a 3D parametric 
> generation but KiCad would need to support it internally somehow.
> Since a 3D file is only need at export time (eg: when user exports the 
> project to STEP) or for 3D Viewer proposes ( generated on the fly ), there 
> was no need to storage any 3D data for share a project as it will always be 
> generated by KiCad.
> 
> Parametric / procedural generation is the easiest part:
> https://github.com/KammutierSpule/kicad3Dmodels
> 
> Mario Luzeiro

This is an interesting idea but I'm not sure I want kicad to be a
parametric 3D model generator.  I think this is best left to tools that
are specialized for this purpose such as FreeCAD.  Although this could
be scripted using python since a lot of this work is already done.  This
is a discussion after v6.

Cheers,

Wayne

> 
> 
> From: Kicad-developers 
>  on behalf of 
> Wayne Stambaugh 
> Sent: 04 January 2019 14:03
> To: kicad-developers@lists.launchpad.net
> Subject: Re: [Kicad-developers] [RFC] Symbol library file format
> 
> On 1/2/2019 2:55 PM, Vesa Solonen wrote:
>> kristoffer Ödmark kirjoitti 2.1.2019 klo 12.32:
>>
>>> I would rather have a new atomic file-format or storage format, so one
>>> could actually send a complete part in an easy way, that would include
>>> 3D-Files, SPICE data, symbol and footprints, and it would also mean that
>>> one could do the mapping to pins in that directory or file instead.
> 
> I can somewhat see the utility in this but it really doesn't make a lot
> of sense to me.  Lets assume for a second that it would be useful to
> create a separate atomic part for every 0603 resistor part number
> currently available.  Given that the 0603 footprint file is 1.7K, the
> step 3D model is 39.6K, the wrl 3D model is 14.4K, and the fully defined
> symbol file would be about 1.5K so that's a total of 57.2K per atomic
> part number.  Now if you create a unique part number for every
> resistance value, tolerance, temperature coefficient, manufacturer, etc.
> that would be tens if not hundreds of thousands of unique part numbers.
>  100K unique atomic part numbers would require 5.72G of storage just for
> the 0603 resistors.  Add all of the other resistors footprints including
> THT and your talking about several 100G.  Add all of the other passive
> components like capacitors, inductors, etc and you are talking about
> several terabytes of storage.  Maybe I'm missing something but even with
> modern storage systems, this doesn't scale well.
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Wayne Stambaugh]

On 1/3/19 11:24 PM, Simon Richter wrote:
> Hi,
> 
> On 03.01.19 19:06, José Ignacio wrote:
> 
>> I
>> think useful comments to the proposed format should see beyond the
>> actual low level representation of the data and talk about the overall
>> model being used to store it.
> 
> tl;dr: That's a separate discussion.
> 
> There are two schools of thought here, one that treats saved data as a
> protocol between two black box instances, and one that treats it as a
> serialization of the internal state.
> 
> Both have advantages and disadvantages. The "protocol" approach allows
> changing internals more easily, and gives better compatibility between
> versions as changes to the file format have to be made deliberately,
> while the "serialization" approach gives us load/save basically for
> free, so we need a lot less code.
> 
> The "serialization" model also requires us to generate the internal data
> structures from a more constrained language like IDL, as the marshaller
> needs to know when to follow pointers, and what members of an array are
> actually valid.
> 
> I'm not sure there are generators that include support for spatial
> indexes, though, which is pretty much a requirement for fast rendering,
> so this is pretty much impossible at the moment, which places us in
> "hand written load/save code" territory anyway.
> 
> We have also ignored diff/merge capability so far, which I believe is a
> good thing because it cannot really be done on a textual level (schemas
> are two-dimensional, PCBs are three-dimensional, so there is no normal
> form with a consistent ordering of elements that will make the files
> diffable).
> 
> I fully expect both the internal model and the file format to change
> significantly in the coming years as new features are added. The main
> requirement for the file format is that it always needs to be possible
> to read older files in some way, and to recognize when a file is newer
> than the current parser understands.
> 
> It might be a good idea to also have an "extension" mechanism, where we
> don't increase the version number when adding a new feature, but rather
> mark files that actually use the new feature, so files written by newer
> versions that don't use one of the newer functions can be read by older
> versions. This would also be another point for "hand written" load/save.
> 
>Simon
> 

I've done some more investigation on this and I'm not convinced that
this is any better than our current implementation due to some of the
issues Simon mentioned above.  I've also discovered there is also a
significant performance hit[1] with the JSON reader.  This is important
due to the overhead required when loading libraries.  I appreciate the
information and protobufs are interesting but I am going to stick with
what we have and know versus what we don't know because I don't think we
have the extra manpower to experiment only to find out it doesn't suite
our requirements.

Cheers,

Wayne

[1]:
https://groups.google.com/forum/#!searchin/protobuf/JSON|sort:date/protobuf/23Slq4AX7oE/xuwQX_ZXAAAJ



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Rene Pöschl]

Something else that i remembered now.

Would it be a good idea to prepare for alternative function handling for 
pins?
Meaning an alternative pin name connected to an alternative type (or a 
number of these)


That would allow better symbols for micro controllers.

I do not expect this to be implemented for v6 but maybe this is 
something that should be integrated in the file format spec such that a 
future implementation is comparably easy.


On 01/01/19 20:59, Wayne Stambaugh wrote:

I have updated and published the symbol file format[1] for comment.
Hopefully there isn't too much to change.  The only thing to really
finalize is the internal units.  The initial concept was unitless but
the more I think about it and discuss with other developers, it makes
more sense to use units for the following reasons:

1. It's easier to visualize in your head how the symbols on a given page
size will layout.

2. Converting from other file formats (Eagle, Altium, etc) will be
easier since most if not all of them have a defined unit.

I'm thinking 10u (or possibly 100u) will make a good internal units
value.  Once we nail down the units, I will update the file format
document accordingly.

Please keep in mind that this is the symbol library file format document
so things like constraints belong in the schematic file format.  I will
be posting the schematic file format as soon as I finish updating it.

Cheers,

Wayne

[1]:
https://docs.google.com/document/d/1lyL_8FWZRouMkwqLiIt84rd2Htg4v1vz8_2MzRKHRkc/edit


___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp




___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Rene Pöschl]

On 04/01/19 15:10, Wayne Stambaugh wrote:

Is there a way to mark a unit as "power unit" (I think you mentioned
sometime back that this might be required to properly make simulation
for multi unit symbols possible while still allowing the power unit to
be separate. A power unit would also be a "must be placed unit" but i
feel separating these two might add more flexibility.)

See definition above.  I'm not sure having a "required" keyword is
useful.  Do you have an example in mind.



There are many parts that not only have power connections but also some 
control connections that are necessary for the function of the part. A 
lot of these have the control pins shared between many units so it makes 
sense to move them to a specialized unit. That unit would then be 
required but does not fulfill the "this is a power unit" thing. Hence my 
suggestion for a more general "this unit must be placed" vs. "This unit 
is optional" field. (The "This is a power unit" thing is not really 
necessary for ERC. At least not if there is a "this is a required unit" 
flag. The only reason i suggested that is because you mentioned that it 
might be required for simulation purposes.)


___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Wayne Stambaugh]

Simon,

On 1/2/19 8:39 PM, Simon Richter wrote:
> Hi Wayne,
> 
> On 02.01.19 23:20, Wayne Stambaugh wrote:
> 
>>> This would be an artificial unit for the file format, not necessarily
>>> the true internal unit (which I want to unify across the entire program
>>> as soon as feasible, but that should be independent from the file formats).
> 
>> I'm not sure exactly what you mean by a true internal unit.
> 
> The unit we're currently using internally (1nm for pcbnew, 1mil for
> eeschema, etc). This one shouldn't leak into the file format IMO.

Like the board file format, I plan to use millmeters as the file unit
which will be precision clamped to the internal unit of the
symbol/schematic file formats be it 10nm or 100nm.

> 
>>> The other thing I'd like to see are standard properties for tagging
>>> library conformance, so components that have been verified by library
>>> maintainers would contain the name, version and variant of the library
>>> conventions they have been tested against, and these tags would be
>>> removed when editing.
> 
> [...]
> 
>> I'm not sure it's a good idea to embed this information in symbol
>> libraries because how would it be defined for a user (non-KiCad)
>> library.  It could give users a false impression that a library is an
>> official library if it's abused.
> 
> That's why it would be nice to have name/version/variant. Official libs
> would use "KLC" as the name, while users can define their own in-house
> rules if they need to.

I'm not sure this will have the desired effect although I'm not opposed
to it.  We have already seen cases where the header in the board file
has been abused by rogue scripts and hand editing.

> 
>Simon
> 
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Wayne Stambaugh]

I'm willing to add properties which are already defined to pins.  There
has been talk about adding about adding electrical constraints for
advance ERC testing to symbols but that will not be part of the v6
implementation of the new file format.  It's going to be a lot of work
to implement the features we've added to v6 so this would be outside the
scope of the first version of the new file format.  I'm certainly open
to discussing this as part of a later version of kicad.

On 1/2/19 3:52 PM, mark.vandoesb...@hetnet.nl wrote:
> One thing I would like to have is user defined pin properties, just like
> the part properties. Currently I use the BOM generator to generate a
> software header file from the netlist. I have to use an additional file
> to store the information I need. It would be nice if this information
> can be stored in the symbol. This could be any kind of information, for
> example bank number or clock region for an FPGA. Or driver strenght/speed
> setting for a microcontroller.
> 
> regards,
> 
> Mark.
> 
> Wayne Stambaugh  wrote:
> 
>   I have updated and published the symbol file format[1] for comment.
>   Hopefully there isn't too much to change.  The only thing to really
>   finalize is the internal units.  The initial concept was unitless but
>   the more I think about it and discuss with other developers, it makes
>   more sense to use units for the following reasons:
> 
>   1. It's easier to visualize in your head how the symbols on a given page
>   size will layout.
> 
>   2. Converting from other file formats (Eagle, Altium, etc) will be
>   easier since most if not all of them have a defined unit.
> 
>   I'm thinking 10u (or possibly 100u) will make a good internal units
>   value.  Once we nail down the units, I will update the file format
>   document accordingly.
> 
>   Please keep in mind that this is the symbol library file format document
>   so things like constraints belong in the schematic file format.  I will
>   be posting the schematic file format as soon as I finish updating it.
> 
>   Cheers,
> 
>   Wayne
> 
>   [1]:
>   
> https://docs.google.com/document/d/1lyL_8FWZRouMkwqLiIt84rd2Htg4v1vz8_2MzRKHRkc/edit
> 
> 
>   ___
>   Mailing list: https://launchpad.net/~kicad-developers
>   Post to : kicad-developers@lists.launchpad.net
>   Unsubscribe : https://launchpad.net/~kicad-developers
>   More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Wayne Stambaugh]

I'm willing to add properties which are already defined to pins.  There
has been talk about adding about adding electrical constraints for
advance ERC testing to symbols but that will not be part of the v6
implementation of the new file format.  It's going to be a lot of work
to implement the features we've added to v6 so this would be outside the
scope of the first version of the new file format.  I'm certainly open
to discussing this as part of a later version of kicad.

On 1/2/19 3:52 PM, mark.vandoesb...@hetnet.nl wrote:
> One thing I would like to have is user defined pin properties, just like
> the part properties. Currently I use the BOM generator to generate a
> software header file from the netlist. I have to use an additional file
> to store the information I need. It would be nice if this information
> can be stored in the symbol. This could be any kind of information, for
> example bank number or clock region for an FPGA. Or driver strenght/speed
> setting for a microcontroller.
> 
> regards,
> 
> Mark.
> 
> Wayne Stambaugh  wrote:
> 
>   I have updated and published the symbol file format[1] for comment.
>   Hopefully there isn't too much to change.  The only thing to really
>   finalize is the internal units.  The initial concept was unitless but
>   the more I think about it and discuss with other developers, it makes
>   more sense to use units for the following reasons:
> 
>   1. It's easier to visualize in your head how the symbols on a given page
>   size will layout.
> 
>   2. Converting from other file formats (Eagle, Altium, etc) will be
>   easier since most if not all of them have a defined unit.
> 
>   I'm thinking 10u (or possibly 100u) will make a good internal units
>   value.  Once we nail down the units, I will update the file format
>   document accordingly.
> 
>   Please keep in mind that this is the symbol library file format document
>   so things like constraints belong in the schematic file format.  I will
>   be posting the schematic file format as soon as I finish updating it.
> 
>   Cheers,
> 
>   Wayne
> 
>   [1]:
>   
> https://docs.google.com/document/d/1lyL_8FWZRouMkwqLiIt84rd2Htg4v1vz8_2MzRKHRkc/edit
> 
> 
>   ___
>   Mailing list: https://launchpad.net/~kicad-developers
>   Post to : kicad-developers@lists.launchpad.net
>   Unsubscribe : https://launchpad.net/~kicad-developers
>   More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Mário Luzeiro]

> Now if you create a unique part number for every
> resistance value, tolerance, temperature coefficient, manufacturer, etc.
> that would be tens if not hundreds of thousands of unique part numbers.

I know my intervention may be a bit offtopic, but it may add some other 
elements to discussion:

On such case of the 3D variants, it could be considered a 3D parametric 
generation but KiCad would need to support it internally somehow.
Since a 3D file is only need at export time (eg: when user exports the project 
to STEP) or for 3D Viewer proposes ( generated on the fly ), there was no need 
to storage any 3D data for share a project as it will always be generated by 
KiCad.

Parametric / procedural generation is the easiest part:
https://github.com/KammutierSpule/kicad3Dmodels

Mario Luzeiro


From: Kicad-developers 
 on behalf of 
Wayne Stambaugh 
Sent: 04 January 2019 14:03
To: kicad-developers@lists.launchpad.net
Subject: Re: [Kicad-developers] [RFC] Symbol library file format

On 1/2/2019 2:55 PM, Vesa Solonen wrote:
> kristoffer Ödmark kirjoitti 2.1.2019 klo 12.32:
>
>> I would rather have a new atomic file-format or storage format, so one
>> could actually send a complete part in an easy way, that would include
>> 3D-Files, SPICE data, symbol and footprints, and it would also mean that
>> one could do the mapping to pins in that directory or file instead.

I can somewhat see the utility in this but it really doesn't make a lot
of sense to me.  Lets assume for a second that it would be useful to
create a separate atomic part for every 0603 resistor part number
currently available.  Given that the 0603 footprint file is 1.7K, the
step 3D model is 39.6K, the wrl 3D model is 14.4K, and the fully defined
symbol file would be about 1.5K so that's a total of 57.2K per atomic
part number.  Now if you create a unique part number for every
resistance value, tolerance, temperature coefficient, manufacturer, etc.
that would be tens if not hundreds of thousands of unique part numbers.
 100K unique atomic part numbers would require 5.72G of storage just for
the 0603 resistors.  Add all of the other resistors footprints including
THT and your talking about several 100G.  Add all of the other passive
components like capacitors, inductors, etc and you are talking about
several terabytes of storage.  Maybe I'm missing something but even with
modern storage systems, this doesn't scale well.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Kristoffer]

Yes you are right Wayne,

Its just that i believe an IDF like protobuf would ease the development 
of such a feature/file-format, the tools around such a feature and allow 
for a better overall iteration of such a feature/fileformat.


That is why i even mentioned it here, because since we are changing the 
symbol library format, I think that specifying the file in protobuf, 
exporting to a readable on-disk-format supported by protobuf to be a

good choice.

- Kristoffer

On 2019-01-04 16:26, Wayne Stambaugh wrote:

Hi Kristoffer,

On 1/4/2019 10:24 AM, Kristoffer wrote:

On 2019-01-04 15:03, Wayne Stambaugh wrote:


I can somewhat see the utility in this but it really doesn't make a lot
of sense to me.  Lets assume for a second that it would be useful to
create a separate atomic part for every 0603 resistor part number
currently available.  Given that the 0603 footprint file is 1.7K, the
step 3D model is 39.6K, the wrl 3D model is 14.4K, and the fully defined
symbol file would be about 1.5K so that's a total of 57.2K per atomic
part number.  Now if you create a unique part number for every
resistance value, tolerance, temperature coefficient, manufacturer, etc.
that would be tens if not hundreds of thousands of unique part numbers.
   100K unique atomic part numbers would require 5.72G of storage just for
the 0603 resistors.  Add all of the other resistors footprints including
THT and your talking about several 100G.  Add all of the other passive
components like capacitors, inductors, etc and you are talking about
several terabytes of storage.  Maybe I'm missing something but even with
modern storage systems, this doesn't scale well.


I agree totally, But today basically we treat the symbol as the ultimate
storage unit for these things, so if one were so inclined, it would mean
a lot of symbols instead.

Basically I just want the symbol to be the graphics to eeschema, and a
higher level part unit keeping the relation of eeschema-graphics,
pcbnew-graphics, metadata and documents information.

I am not advocating the need to generate parts for every resistor out
there. Just that maybe we should allow the possibility for me to instead
of sending a pdf, a footprint, a symbol, a 3d file, store them in
different specific places to match what is written in the symbol file.

Just allow a part to define all of these in one file, or with relative
paths with regards to the part file, for easier transfer to partners or
similar.



I'm not opposed to this idea but that is outside the scope of symbol
library file format.  This would be a different file format and discussion.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp



___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Wayne Stambaugh]

Hi Kristoffer,

On 1/4/2019 10:24 AM, Kristoffer wrote:
> On 2019-01-04 15:03, Wayne Stambaugh wrote:
> 
>> I can somewhat see the utility in this but it really doesn't make a lot
>> of sense to me.  Lets assume for a second that it would be useful to
>> create a separate atomic part for every 0603 resistor part number
>> currently available.  Given that the 0603 footprint file is 1.7K, the
>> step 3D model is 39.6K, the wrl 3D model is 14.4K, and the fully defined
>> symbol file would be about 1.5K so that's a total of 57.2K per atomic
>> part number.  Now if you create a unique part number for every
>> resistance value, tolerance, temperature coefficient, manufacturer, etc.
>> that would be tens if not hundreds of thousands of unique part numbers.
>>   100K unique atomic part numbers would require 5.72G of storage just for
>> the 0603 resistors.  Add all of the other resistors footprints including
>> THT and your talking about several 100G.  Add all of the other passive
>> components like capacitors, inductors, etc and you are talking about
>> several terabytes of storage.  Maybe I'm missing something but even with
>> modern storage systems, this doesn't scale well.
> 
> I agree totally, But today basically we treat the symbol as the ultimate
> storage unit for these things, so if one were so inclined, it would mean
> a lot of symbols instead.
> 
> Basically I just want the symbol to be the graphics to eeschema, and a
> higher level part unit keeping the relation of eeschema-graphics,
> pcbnew-graphics, metadata and documents information.
> 
> I am not advocating the need to generate parts for every resistor out
> there. Just that maybe we should allow the possibility for me to instead
> of sending a pdf, a footprint, a symbol, a 3d file, store them in
> different specific places to match what is written in the symbol file.
> 
> Just allow a part to define all of these in one file, or with relative
> paths with regards to the part file, for easier transfer to partners or
> similar.
> 

I'm not opposed to this idea but that is outside the scope of symbol
library file format.  This would be a different file format and discussion.

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Kristoffer]

On 2019-01-04 15:03, Wayne Stambaugh wrote:


I can somewhat see the utility in this but it really doesn't make a lot
of sense to me.  Lets assume for a second that it would be useful to
create a separate atomic part for every 0603 resistor part number
currently available.  Given that the 0603 footprint file is 1.7K, the
step 3D model is 39.6K, the wrl 3D model is 14.4K, and the fully defined
symbol file would be about 1.5K so that's a total of 57.2K per atomic
part number.  Now if you create a unique part number for every
resistance value, tolerance, temperature coefficient, manufacturer, etc.
that would be tens if not hundreds of thousands of unique part numbers.
  100K unique atomic part numbers would require 5.72G of storage just for
the 0603 resistors.  Add all of the other resistors footprints including
THT and your talking about several 100G.  Add all of the other passive
components like capacitors, inductors, etc and you are talking about
several terabytes of storage.  Maybe I'm missing something but even with
modern storage systems, this doesn't scale well.


I agree totally, But today basically we treat the symbol as the ultimate 
storage unit for these things, so if one were so inclined, it would mean 
a lot of symbols instead.


Basically I just want the symbol to be the graphics to eeschema, and a 
higher level part unit keeping the relation of eeschema-graphics, 
pcbnew-graphics, metadata and documents information.


I am not advocating the need to generate parts for every resistor out 
there. Just that maybe we should allow the possibility for me to instead 
of sending a pdf, a footprint, a symbol, a 3d file, store them in 
different specific places to match what is written in the symbol file.


Just allow a part to define all of these in one file, or with relative 
paths with regards to the part file, for easier transfer to partners or 
similar.



___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Wayne Stambaugh]

On 1/2/2019 8:24 PM, Rene Pöschl wrote:
> I have a few questions regarding multi part (or multi unit) symbols. (I
> do not really see a clear indication for these features in your document.)
> 
> Is there a definition for having only some units exchangeable?

This is provided by the extends keyword.  In the provided 7400 example,
all four gates are extended from the same base part so they would be
interchangeable.

> Is there a way to define a unit as "must be placed" or "optional"?

I added a power keyword to denote power symbols so any power symbol
defined as a sub-unit of a symbol would be required.

> Is there a way to mark a unit as "power unit" (I think you mentioned
> sometime back that this might be required to properly make simulation
> for multi unit symbols possible while still allowing the power unit to
> be separate. A power unit would also be a "must be placed unit" but i
> feel separating these two might add more flexibility.)

See definition above.  I'm not sure having a "required" keyword is
useful.  Do you have an example in mind.

> 
> On 01/01/19 20:59, Wayne Stambaugh wrote:
>> I have updated and published the symbol file format[1] for comment.
>> Hopefully there isn't too much to change.  The only thing to really
>> finalize is the internal units.  The initial concept was unitless but
>> the more I think about it and discuss with other developers, it makes
>> more sense to use units for the following reasons:
>>
>> 1. It's easier to visualize in your head how the symbols on a given page
>> size will layout.
>>
>> 2. Converting from other file formats (Eagle, Altium, etc) will be
>> easier since most if not all of them have a defined unit.
>>
>> I'm thinking 10u (or possibly 100u) will make a good internal units
>> value.  Once we nail down the units, I will update the file format
>> document accordingly.
>>
>> Please keep in mind that this is the symbol library file format document
>> so things like constraints belong in the schematic file format.  I will
>> be posting the schematic file format as soon as I finish updating it.
>>
>> Cheers,
>>
>> Wayne
>>
>> [1]:
>> https://docs.google.com/document/d/1lyL_8FWZRouMkwqLiIt84rd2Htg4v1vz8_2MzRKHRkc/edit
>>
>>
>>
>> ___
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to : kicad-developers@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Wayne Stambaugh]

On 1/2/2019 2:55 PM, Vesa Solonen wrote:
> kristoffer Ödmark kirjoitti 2.1.2019 klo 12.32:
> 
>> I would rather have a new atomic file-format or storage format, so one
>> could actually send a complete part in an easy way, that would include
>> 3D-Files, SPICE data, symbol and footprints, and it would also mean that
>> one could do the mapping to pins in that directory or file instead.

I can somewhat see the utility in this but it really doesn't make a lot
of sense to me.  Lets assume for a second that it would be useful to
create a separate atomic part for every 0603 resistor part number
currently available.  Given that the 0603 footprint file is 1.7K, the
step 3D model is 39.6K, the wrl 3D model is 14.4K, and the fully defined
symbol file would be about 1.5K so that's a total of 57.2K per atomic
part number.  Now if you create a unique part number for every
resistance value, tolerance, temperature coefficient, manufacturer, etc.
that would be tens if not hundreds of thousands of unique part numbers.
 100K unique atomic part numbers would require 5.72G of storage just for
the 0603 resistors.  Add all of the other resistors footprints including
THT and your talking about several 100G.  Add all of the other passive
components like capacitors, inductors, etc and you are talking about
several terabytes of storage.  Maybe I'm missing something but even with
modern storage systems, this doesn't scale well.

>>
>> And since the discussion about adding a manufacturer part number to
>> libraries went nowhere, I cannot see how adding a SPICE model would fit
>> here, and if SPICE models are added. Well then I suggest we add 3D-Files
>> and Footprints as well, no use in half-assing it.
> 
> Agreed. Considering how much has changed (also outside KiCad) since the
> new symbol format was drafted it may be worth a second look at the issues.
> 
> I would suggest having a second look with Horizon EDA if there is any
> common ground as there are certainly only things to be gained from some
> cooperation:
> 
> https://github.com/carrotIndustries/horizon/wiki/The-Pool
> 
> -Vesa
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Simon Richter]

Hi,

On 03.01.19 19:06, José Ignacio wrote:

> I
> think useful comments to the proposed format should see beyond the
> actual low level representation of the data and talk about the overall
> model being used to store it.

tl;dr: That's a separate discussion.

There are two schools of thought here, one that treats saved data as a
protocol between two black box instances, and one that treats it as a
serialization of the internal state.

Both have advantages and disadvantages. The "protocol" approach allows
changing internals more easily, and gives better compatibility between
versions as changes to the file format have to be made deliberately,
while the "serialization" approach gives us load/save basically for
free, so we need a lot less code.

The "serialization" model also requires us to generate the internal data
structures from a more constrained language like IDL, as the marshaller
needs to know when to follow pointers, and what members of an array are
actually valid.

I'm not sure there are generators that include support for spatial
indexes, though, which is pretty much a requirement for fast rendering,
so this is pretty much impossible at the moment, which places us in
"hand written load/save code" territory anyway.

We have also ignored diff/merge capability so far, which I believe is a
good thing because it cannot really be done on a textual level (schemas
are two-dimensional, PCBs are three-dimensional, so there is no normal
form with a consistent ordering of elements that will make the files
diffable).

I fully expect both the internal model and the file format to change
significantly in the coming years as new features are added. The main
requirement for the file format is that it always needs to be possible
to read older files in some way, and to recognize when a file is newer
than the current parser understands.

It might be a good idea to also have an "extension" mechanism, where we
don't increase the version number when adding a new feature, but rather
mark files that actually use the new feature, so files written by newer
versions that don't use one of the newer functions can be read by older
versions. This would also be another point for "hand written" load/save.

   Simon



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Andrew Lutsenko]

> The important thing is what is in the file. If nothing else, S-exp is a
concise way to express this concept during development. Exact format
representation in disk is, right now, bikeshedding.
The important thing is a clean data model and formal IDL like proto helps
with that immensely. If you read through Wayne's google doc comments you
will see how tightly coupled is the discussion of the data model to file
format, which should not be the case. That does not lend itself well to
extensibility and evolution of the data model which is natural part of any
actively developed software.
Thomas pointed out excellent example of issues with current approach in the
other reply. That is very similar to my experience of digging in pcbnew
internals but it's applicable to eeschema as well.

Imagine if pcbnew plugin API was proto based. No more swig quirks for devs
to take care of, no more plugin breakages on every tiny change in Cpp
source, data exchange between main app and plugin is straightforward, they
agree on data model because it has an actual formal definition.

> On the subject of parsers, as opposed to formats, I strongly suggest not
to use the pcbnew one, as it is highly bound up in the data rather than the
syntax. The beauty of sexp is the abstract nature, which lends itself will
to extensible formats. How it is implemented in pcbnew is quite contrary to
that aim and naturally leads to fragility of the parser (for example
unexpected, but syntactically valid, fields can be lethal).

Or you could just use a library that takes care of parsing for you, does it
in all major languages for free, represents formal definition of your data
model and will not break when you add a field.

And to Jose's point
> the S expression parser is already implemented and it works fine, it is
trivial to convert s-expressions to any other data representation you like,
be it, xml, json or whatever comes up next week in NPM
Yeah it works fine but only in cpp and if you want to use it outside of
KiCad codebase you will spend hours reading code to determine actual
format. It is trivial to convert, as long as you put in the aforementioned
hours AND nothing ever changes in the format. Oh and protos have been there
before NPM was a thing and will likely outlive it too (at least one of
thrift or proto definitely will).

> I think useful comments to the proposed format should see beyond the
actual low level representation of the data and talk about the overall
model being used to store it.
I agree completely. Protobufs help with decoupling that.

Regards,
Andrew

On Thu, Jan 3, 2019 at 11:02 AM John Beard  wrote:

> I agree. The important thing is what is in the file. If nothing else,
> S-exp is a concise way to express this concept during development. Exact
> format representation in disk is, right now, bikeshedding.
>
> When we get to that stage, all that is required is that the format is VCS
> friendly and human readable. I suggest that we work out what will be
> represented in the v6 files, then work with a preliminary format at first
> while we implement the kicad data structure handling.
>
> Sexp is probably the simpler path as we have both the pseudo-sexp parser
> used in pcbnew, as well as a "real" one for STEP export (I can't check
> details now, I'm not at my computer).
>
> On the subject of parsers, as opposed to formats, I strongly suggest not
> to use the pcbnew one, as it is highly bound up in the data rather than the
> syntax. The beauty of sexp is the abstract nature, which lends itself will
> to extensible formats. How it is implemented in pcbnew is quite contrary to
> that aim and naturally leads to fragility of the parser (for example
> unexpected, but syntactically valid, fields can be lethal).
>
> Whatever the format, we should strive to separate the handling of the
> syntax of the file and the meaning of the data therein. By doing this, the
> format layer can be swapped and tweaked without reference to the data
> model, and the data model can be changed without touching a byte of the
> format parser/writers.
>
> Cheers,
>
> John
>
>
>
>
> On 3 January 2019 18:06:45 GMT, "José Ignacio" 
> wrote:
>>
>> I think all this babble about data representations to be pointless and
>> counterproductive. the S expression parser is already implemented and it
>> works fine, it is trivial to convert s-expressions to any other data
>> representation you like, be it, xml, json or whatever comes up next week in
>> NPM. The issue with the file format is really to come up with a good data
>> model to represent the objects in kicad, and neither protobufs nor any of
>> the other guys really does anything for us in that area, if anything that
>> is the input that needs to be given to whatever parser generator, or
>> manually generated parser process we choose to utilize. I think useful
>> comments to the proposed format should see beyond the actual low level
>> representation of the data and talk about the overall model being used to
>> 

Re: [Kicad-developers] [RFC] Symbol library file format

[John Beard]

I agree. The important thing is what is in the file. If nothing else, S-exp is 
a concise way to express this concept during development. Exact format 
representation in disk is, right now, bikeshedding.

When we get to that stage, all that is required is that the format is VCS 
friendly and human readable. I suggest that we work out what will be 
represented in the v6 files, then work with a preliminary format at first while 
we implement the kicad data structure handling. 

Sexp is probably the simpler path as we have both the pseudo-sexp parser used 
in pcbnew, as well as a "real" one for STEP export (I can't check details now, 
I'm not at my computer).

On the subject of parsers, as opposed to formats, I strongly suggest not to use 
the pcbnew one, as it is highly bound up in the data rather than the syntax. 
The beauty of sexp is the abstract nature, which lends itself will to 
extensible formats. How it is implemented in pcbnew is quite contrary to that 
aim and naturally leads to fragility of the parser (for example unexpected, but 
syntactically valid, fields can be lethal).

Whatever the format, we should strive to separate the handling of the syntax of 
the file and the meaning of the data therein. By doing this, the format layer 
can be swapped and tweaked without reference to the data model, and the data 
model can be changed without touching a byte of the format parser/writers.

Cheers, 

John




On 3 January 2019 18:06:45 GMT, "José Ignacio"  wrote:
>I think all this babble about data representations to be pointless and
>counterproductive. the S expression parser is already implemented and
>it
>works fine, it is trivial to convert s-expressions to any other data
>representation you like, be it, xml, json or whatever comes up next
>week in
>NPM. The issue with the file format is really to come up with a good
>data
>model to represent the objects in kicad, and neither protobufs nor any
>of
>the other guys really does anything for us in that area, if anything
>that
>is the input that needs to be given to whatever parser generator, or
>manually generated parser process we choose to utilize. I think useful
>comments to the proposed format should see beyond the actual low level
>representation of the data and talk about the overall model being used
>to
>store it.
>
>On Thu, Jan 3, 2019 at 3:37 AM Andrew Lutsenko 
>wrote:
>
>> Hi Martijn,
>> My guess is that most of the complexity is in switching to new data
>model
>> in eeschema, which is going to happen anyway. Since with protobufs
>you
>> don't concern yourself with parsing data the only tricky thing would
>be to
>> seamlessly integrate proto codegen into build process on all
>platforms.
>> Since eeschema data model is going to change without backward
>> compatibility, file compatibility is out of the window too so this is
>good
>> opportunity to change underlying format to something that is better
>in the
>> long run. If we decide to go this route than ideally pcbnew would
>switch to
>> proto at some point too, not necessarily in v6.
>>
>> > Would it be feasible to support both in V6 and let the future tell
>us
>> which one would prevail?
>> What do you mean by prevailing? It's clear at this point that
>> s-expressions are not a winner of global trends while I can guarantee
>that
>> sizable portion of the data in the internet (except probably porn,
>lol) is
>> exchanged in protobufs and thrift since all tech giants use them.
>> If you mean in terms of what KiCad users prefer I don't think vast
>> majority would care either way. Both formats are easy to edit
>directly and
>> protos are much better for devs because codegen takes all the routine
>out
>> of data manipulation.
>> While having both formats coexist is theoretically possible I think
>it
>> will only split the user base since people will pick one and stick to
>it. I
>> think this is the case where devs should be empowered to make the
>choice,
>> same as when switch to s-expressions happened.
>> Of course new version still needs to be able to read old files or at
>least
>> some conversion utility needs to be provided to migrate projects to
>v6 so
>> s-expressions code will not be going away for some time.
>>
>> Andrew
>>
>> On Wed, Jan 2, 2019 at 11:53 PM Martijn Kuipers
>
>> wrote:
>>
>>>
>>>
>>> On 3 Jan 2019, at 04:17, Andrew Lutsenko 
>wrote:
>>>
>>> Wayne,
>>>
>>> > There are some interesting and practical concepts with protobuf
>but it's
>>> functionally a binary storage method which I am opposed to.
>>>
>>> That is a somewhat common misconception because protobufs are
>frequently
>>> used for efficient storage/transfer in binary format. But it's not
>tied to
>>> that format at all, at it's core protobufs are just a way of
>defining well
>>> structured data, nothing more. It comes with bells and whistles like
>>> ability to serialize it in various ways, binary being one of them.
>>>
>>> > Encoding and decoding to a text format would be an acceptable
>solution.
>>> Perfect, here is 

Re: [Kicad-developers] [RFC] Symbol library file format

[Thomas Pointhuber]

Hi,

 

I didn't unterstand the reason for the proposal at the start due to new schematic format, new dependencies,...

 

But thinking about it a bit more it really makes sense to me now. In the past, I wrote importer as well as exporter for the footprint file format, and using protobuf would likely simplify such work quite a bit (no requirement to write sexpr parser/writer, a FORMAL specification of the file format). Have you ever seen the import/export implementation in KiCad itself? I would call it a mess. Locale dependent code which requires workarounds, exporter written with string format which can have issues like missing space between brackets, importers, while written as propper Parser, it should actually be implemented in two stages (stage 1: parse sexpr, stage 2: parse data). Furthermore, you need to have those two completly separated pieces of code to actually produce/understand the same data.

 

In the end, it's the decision of the Developer implementing that stuff. But thinking about the current state, I would say even with sexpr format, the code needs to be refactored quite a bit to be less error-prone and easier to extend. Switching to protobuf should simplify import/export, couples the two parts tightly together, removes the burden to write the actual input/output stuff and simplifies the usage of the file format for 3rd party software (including propper validation). And, in comparsion to the current google-doc, it would be an actual FORMAL specification. I did not even found a normal specification for the .kicad_mod format for example. The best specification is the one where code is generated from. In this case you know there are no implementation differences and it is always up-to-date.

 

Regards,

Thomas

 

Gesendet: Donnerstag, 03. Januar 2019 um 19:06 Uhr
Von: "José Ignacio" 
An: anlutse...@gmail.com
Cc: "KiCad Developers" 
Betreff: Re: [Kicad-developers] [RFC] Symbol library file format


I think all this babble about data representations to be pointless and counterproductive. the S _expression_ parser is already implemented and it works fine, it is trivial to convert s-expressions to any other data representation you like, be it, xml, json or whatever comes up next week in NPM. The issue with the file format is really to come up with a good data model to represent the objects in kicad, and neither protobufs nor any of the other guys really does anything for us in that area, if anything that is the input that needs to be given to whatever parser generator, or manually generated parser process we choose to utilize. I think useful comments to the proposed format should see beyond the actual low level representation of the data and talk about the overall model being used to store it.
 


On Thu, Jan 3, 2019 at 3:37 AM Andrew Lutsenko <anlutse...@gmail.com> wrote:



Hi Martijn,
My guess is that most of the complexity is in switching to new data model in eeschema, which is going to happen anyway. Since with protobufs you don't concern yourself with parsing data the only tricky thing would be to seamlessly integrate proto codegen into build process on all platforms. 

Since eeschema data model is going to change without backward compatibility, file compatibility is out of the window too so this is good opportunity to change underlying format to something that is better in the long run. If we decide to go this route than ideally pcbnew would switch to proto at some point too, not necessarily in v6.

 

> Would it be feasible to support both in V6 and let the future tell us which one would prevail?  

What do you mean by prevailing? It's clear at this point that s-expressions are not a winner of global trends while I can guarantee that sizable portion of the data in the internet (except probably porn, lol) is exchanged in protobufs and thrift since all tech giants use them.

If you mean in terms of what KiCad users prefer I don't think vast majority would care either way. Both formats are easy to edit directly and protos are much better for devs because codegen takes all the routine out of data manipulation.

While having both formats coexist is theoretically possible I think it will only split the user base since people will pick one and stick to it. I think this is the case where devs should be empowered to make the choice, same as when switch to s-expressions happened.

Of course new version still needs to be able to read old files or at least some conversion utility needs to be provided to migrate projects to v6 so s-expressions code will not be going away for some time.

 

Andrew

 


On Wed, Jan 2, 2019 at 11:53 PM Martijn Kuipers <martijn.kuip...@gmail.com> wrote:


 
 

On 3 Jan 2019, at 04:17, Andrew Lutsenko <anlutse...@gmail.com> wrote:
 




Wayne,
 

> There are some interesting and practical concepts with protobuf but it's
functionally a binary storage method which I am opposed to. 

 

That is a somewhat common misconc

Re: [Kicad-developers] [RFC] Symbol library file format

[José Ignacio]

I think all this babble about data representations to be pointless and
counterproductive. the S expression parser is already implemented and it
works fine, it is trivial to convert s-expressions to any other data
representation you like, be it, xml, json or whatever comes up next week in
NPM. The issue with the file format is really to come up with a good data
model to represent the objects in kicad, and neither protobufs nor any of
the other guys really does anything for us in that area, if anything that
is the input that needs to be given to whatever parser generator, or
manually generated parser process we choose to utilize. I think useful
comments to the proposed format should see beyond the actual low level
representation of the data and talk about the overall model being used to
store it.

On Thu, Jan 3, 2019 at 3:37 AM Andrew Lutsenko  wrote:

> Hi Martijn,
> My guess is that most of the complexity is in switching to new data model
> in eeschema, which is going to happen anyway. Since with protobufs you
> don't concern yourself with parsing data the only tricky thing would be to
> seamlessly integrate proto codegen into build process on all platforms.
> Since eeschema data model is going to change without backward
> compatibility, file compatibility is out of the window too so this is good
> opportunity to change underlying format to something that is better in the
> long run. If we decide to go this route than ideally pcbnew would switch to
> proto at some point too, not necessarily in v6.
>
> > Would it be feasible to support both in V6 and let the future tell us
> which one would prevail?
> What do you mean by prevailing? It's clear at this point that
> s-expressions are not a winner of global trends while I can guarantee that
> sizable portion of the data in the internet (except probably porn, lol) is
> exchanged in protobufs and thrift since all tech giants use them.
> If you mean in terms of what KiCad users prefer I don't think vast
> majority would care either way. Both formats are easy to edit directly and
> protos are much better for devs because codegen takes all the routine out
> of data manipulation.
> While having both formats coexist is theoretically possible I think it
> will only split the user base since people will pick one and stick to it. I
> think this is the case where devs should be empowered to make the choice,
> same as when switch to s-expressions happened.
> Of course new version still needs to be able to read old files or at least
> some conversion utility needs to be provided to migrate projects to v6 so
> s-expressions code will not be going away for some time.
>
> Andrew
>
> On Wed, Jan 2, 2019 at 11:53 PM Martijn Kuipers 
> wrote:
>
>>
>>
>> On 3 Jan 2019, at 04:17, Andrew Lutsenko  wrote:
>>
>> Wayne,
>>
>> > There are some interesting and practical concepts with protobuf but it's
>> functionally a binary storage method which I am opposed to.
>>
>> That is a somewhat common misconception because protobufs are frequently
>> used for efficient storage/transfer in binary format. But it's not tied to
>> that format at all, at it's core protobufs are just a way of defining well
>> structured data, nothing more. It comes with bells and whistles like
>> ability to serialize it in various ways, binary being one of them.
>>
>> > Encoding and decoding to a text format would be an acceptable solution.
>> Perfect, here is example of built in proto text encoder, it resembles
>> JSON in that it uses curly braces to encase submessages but doesn't abuse
>> punctuation marks unnecessarily.
>>
>> user_collection {
>>   description = "my default users"
>>   users {
>> key: "user_1234"
>> value {
>>   handle: "winniepoo"
>>   paid_membership: true
>> }
>>   }
>>   users {
>> key: "user_9b27"
>> value {
>>   handle: "smokeybear"
>> }
>>   }}
>>
>> > There is also the issue of learning curve and another build dependency.
>> Yes, that is inevitable but benefits I outlined in my earlier email are
>> too significant to overlook in my opinion.
>>
>> > S-expr is not at all like XML at least not in terms of readability.
>> It actually is a lot like XML, just less pointy brackets. Same arbitrary
>> distinctions between attributes (which are not even named in S-expr, which
>> adds to confusion when glancing at the file) and subfields.
>>
>> Here is real example:
>> (fp_text value 330K (at 0 -1.75) (layer B.Fab)
>>   (effects (font (size 1 1) (thickness 0.15)) (justify mirror))
>> )
>> or
>> (pad 2 smd rect (at 0.95 0 180) (size 0.7 1.3) (layers B.Cu B.Paste
>> B.Mask)
>>   (net 95 "Net-(R17-Pad2)"))
>>
>> Without reading file format docs and/or source code I have no idea why
>> some data is in subfields and some data is in some special fixed order on
>> the same level as it's container.
>> It's also very easy to confuse "330K" being related to "value" when in
>> fact "value" is field name and "330K" is the value. In proto that would
>> 

Re: [Kicad-developers] [RFC] Symbol library file format

[Andrew Lutsenko]

Hi Martijn,
My guess is that most of the complexity is in switching to new data model
in eeschema, which is going to happen anyway. Since with protobufs you
don't concern yourself with parsing data the only tricky thing would be to
seamlessly integrate proto codegen into build process on all platforms.
Since eeschema data model is going to change without backward
compatibility, file compatibility is out of the window too so this is good
opportunity to change underlying format to something that is better in the
long run. If we decide to go this route than ideally pcbnew would switch to
proto at some point too, not necessarily in v6.

> Would it be feasible to support both in V6 and let the future tell us
which one would prevail?
What do you mean by prevailing? It's clear at this point that s-expressions
are not a winner of global trends while I can guarantee that sizable
portion of the data in the internet (except probably porn, lol) is
exchanged in protobufs and thrift since all tech giants use them.
If you mean in terms of what KiCad users prefer I don't think vast majority
would care either way. Both formats are easy to edit directly and protos
are much better for devs because codegen takes all the routine out of data
manipulation.
While having both formats coexist is theoretically possible I think it will
only split the user base since people will pick one and stick to it. I
think this is the case where devs should be empowered to make the choice,
same as when switch to s-expressions happened.
Of course new version still needs to be able to read old files or at least
some conversion utility needs to be provided to migrate projects to v6 so
s-expressions code will not be going away for some time.

Andrew

On Wed, Jan 2, 2019 at 11:53 PM Martijn Kuipers 
wrote:

>
>
> On 3 Jan 2019, at 04:17, Andrew Lutsenko  wrote:
>
> Wayne,
>
> > There are some interesting and practical concepts with protobuf but it's
> functionally a binary storage method which I am opposed to.
>
> That is a somewhat common misconception because protobufs are frequently
> used for efficient storage/transfer in binary format. But it's not tied to
> that format at all, at it's core protobufs are just a way of defining well
> structured data, nothing more. It comes with bells and whistles like
> ability to serialize it in various ways, binary being one of them.
>
> > Encoding and decoding to a text format would be an acceptable solution.
> Perfect, here is example of built in proto text encoder, it resembles JSON
> in that it uses curly braces to encase submessages but doesn't abuse
> punctuation marks unnecessarily.
>
> user_collection {
>   description = "my default users"
>   users {
> key: "user_1234"
> value {
>   handle: "winniepoo"
>   paid_membership: true
> }
>   }
>   users {
> key: "user_9b27"
> value {
>   handle: "smokeybear"
> }
>   }}
>
> > There is also the issue of learning curve and another build dependency.
> Yes, that is inevitable but benefits I outlined in my earlier email are
> too significant to overlook in my opinion.
>
> > S-expr is not at all like XML at least not in terms of readability.
> It actually is a lot like XML, just less pointy brackets. Same arbitrary
> distinctions between attributes (which are not even named in S-expr, which
> adds to confusion when glancing at the file) and subfields.
>
> Here is real example:
> (fp_text value 330K (at 0 -1.75) (layer B.Fab)
>   (effects (font (size 1 1) (thickness 0.15)) (justify mirror))
> )
> or
> (pad 2 smd rect (at 0.95 0 180) (size 0.7 1.3) (layers B.Cu B.Paste
> B.Mask)
>   (net 95 "Net-(R17-Pad2)"))
>
> Without reading file format docs and/or source code I have no idea why
> some data is in subfields and some data is in some special fixed order on
> the same level as it's container.
> It's also very easy to confuse "330K" being related to "value" when in
> fact "value" is field name and "330K" is the value. In proto that would
> look like this:
>
> fp_text {
>   name: "value"
>   value: "330K"
>   at {
> x: 0
> y: -1.75
>   }
>   layer: "B.Fab"
>   effects {
> ...
>   }
> }
>
> But I do agree on the point that all markup formats have their downsides.
> For example json/yaml or proto text format would be less dense per line,
> however that may be an upside when looked at it from version control and
> diff-ing perspective.
>
> > To me self documenting means that the file format doesn't even require a
> > document to explain it's contents.  It should be self evident from the
> > contents of the file.
> What I meant by self documenting was that the document describing the
> format and document implementing the format (actual source code) was the
> same.
> But even as evident from example above, proto text format yields file
> contents that are pretty well self documented too.
>
> > Changing file formats would be a
> > benefit but why would we need to do that?  If we have a human readable
> > 

Re: [Kicad-developers] [RFC] Symbol library file format

[Martijn Kuipers]

> On 3 Jan 2019, at 04:17, Andrew Lutsenko  wrote:
> 
> Wayne,
> 
> > There are some interesting and practical concepts with protobuf but it's
> functionally a binary storage method which I am opposed to. 
> 
> That is a somewhat common misconception because protobufs are frequently used 
> for efficient storage/transfer in binary format. But it's not tied to that 
> format at all, at it's core protobufs are just a way of defining well 
> structured data, nothing more. It comes with bells and whistles like ability 
> to serialize it in various ways, binary being one of them.
> 
> > Encoding and decoding to a text format would be an acceptable solution. 
> Perfect, here is example of built in proto text encoder, it resembles JSON in 
> that it uses curly braces to encase submessages but doesn't abuse punctuation 
> marks unnecessarily.
> 
> user_collection {
>   description = "my default users"
>   users {
> key: "user_1234"
> value {
>   handle: "winniepoo"
>   paid_membership: true
> }
>   }
>   users {
> key: "user_9b27"
> value {
>   handle: "smokeybear"
> }
>   }
> }
> > There is also the issue of learning curve and another build dependency.
> Yes, that is inevitable but benefits I outlined in my earlier email are too 
> significant to overlook in my opinion.
> 
> > S-expr is not at all like XML at least not in terms of readability.
> It actually is a lot like XML, just less pointy brackets. Same arbitrary 
> distinctions between attributes (which are not even named in S-expr, which 
> adds to confusion when glancing at the file) and subfields.
> 
> Here is real example:
> (fp_text value 330K (at 0 -1.75) (layer B.Fab)
>   (effects (font (size 1 1) (thickness 0.15)) (justify mirror))
> )
> or
> (pad 2 smd rect (at 0.95 0 180) (size 0.7 1.3) (layers B.Cu B.Paste 
> B.Mask)
>   (net 95 "Net-(R17-Pad2)"))
> 
> Without reading file format docs and/or source code I have no idea why some 
> data is in subfields and some data is in some special fixed order on the same 
> level as it's container.
> It's also very easy to confuse "330K" being related to "value" when in fact 
> "value" is field name and "330K" is the value. In proto that would look like 
> this:
> 
> fp_text {
>   name: "value"
>   value: "330K"
>   at {
> x: 0
> y: -1.75
>   }
>   layer: "B.Fab"
>   effects {
> ...
>   }
> }
> 
> But I do agree on the point that all markup formats have their downsides. For 
> example json/yaml or proto text format would be less dense per line, however 
> that may be an upside when looked at it from version control and diff-ing 
> perspective.
> 
> > To me self documenting means that the file format doesn't even require a
> > document to explain it's contents.  It should be self evident from the
> > contents of the file. 
> What I meant by self documenting was that the document describing the format 
> and document implementing the format (actual source code) was the same.
> But even as evident from example above, proto text format yields file 
> contents that are pretty well self documented too.
> 
> > Changing file formats would be a
> > benefit but why would we need to do that?  If we have a human readable
> > file format that can be parsed easily and quickly by a computer, what
> > other criteria do we need in a file format?
> 
> See benefit #3 from my first message. By using something standard we make it 
> so much easier to expand on KiCad ecosystem in other languages. I already 
> made an example as a web viewer, here is another: someone may write a plugin 
> for java autorouter software that will read kicad files directly. 
> S-expressions was likely a good choice when it was made but today it's far 
> from widespread and is pretty much unsupported in most languages so the 
> burden is wholly on the developers. Argument can be made that while 
> S-expressions are both human and computer readable it excels at neither since 
> humans still need supporting documentation or source and computers need 
> custom written libraries.
> 
> Regards,
> Andrew
> 
> and Happy New Year :)
> 
> On Wed, Jan 2, 2019 at 8:01 AM Wayne Stambaugh  > wrote:
> On 1/2/2019 5:24 AM, kristoffer Ödmark wrote:
> > I like the idea of using something as Protobuf and I agree fully with
> > the benefits, especially since one can add/remove fields with minimal
> > impact.
> 
> There are some interesting and practical concepts with protobuf but it's
> functionally a binary storage method which I am opposed to.  Encoding
> and decoding to a text format would be an acceptable solution.  There is
> also the issue of learning curve and another build dependency.
> 
> > 
> > Basically the S-expression system used now is looking very much like a
> > reinvented XML to me anyway, and storing protobuf-defined stuff as XML
> > or similar seems actually nice.
> 
> S-expr is not at all like XML at least not in terms of readability.
> Obviously there are 

Re: [Kicad-developers] [RFC] Symbol library file format

[Andrew Lutsenko]

Wayne,

> There are some interesting and practical concepts with protobuf but it's
functionally a binary storage method which I am opposed to.

That is a somewhat common misconception because protobufs are frequently
used for efficient storage/transfer in binary format. But it's not tied to
that format at all, at it's core protobufs are just a way of defining well
structured data, nothing more. It comes with bells and whistles like
ability to serialize it in various ways, binary being one of them.

> Encoding and decoding to a text format would be an acceptable solution.
Perfect, here is example of built in proto text encoder, it resembles JSON
in that it uses curly braces to encase submessages but doesn't abuse
punctuation marks unnecessarily.

user_collection {
  description = "my default users"
  users {
key: "user_1234"
value {
  handle: "winniepoo"
  paid_membership: true
}
  }
  users {
key: "user_9b27"
value {
  handle: "smokeybear"
}
  }}

> There is also the issue of learning curve and another build dependency.
Yes, that is inevitable but benefits I outlined in my earlier email are too
significant to overlook in my opinion.

> S-expr is not at all like XML at least not in terms of readability.
It actually is a lot like XML, just less pointy brackets. Same arbitrary
distinctions between attributes (which are not even named in S-expr, which
adds to confusion when glancing at the file) and subfields.

Here is real example:
(fp_text value 330K (at 0 -1.75) (layer B.Fab)
  (effects (font (size 1 1) (thickness 0.15)) (justify mirror))
)
or
(pad 2 smd rect (at 0.95 0 180) (size 0.7 1.3) (layers B.Cu B.Paste
B.Mask)
  (net 95 "Net-(R17-Pad2)"))

Without reading file format docs and/or source code I have no idea why some
data is in subfields and some data is in some special fixed order on the
same level as it's container.
It's also very easy to confuse "330K" being related to "value" when in fact
"value" is field name and "330K" is the value. In proto that would look
like this:

fp_text {
  name: "value"
  value: "330K"
  at {
x: 0
y: -1.75
  }
  layer: "B.Fab"
  effects {
...
  }
}

But I do agree on the point that all markup formats have their downsides.
For example json/yaml or proto text format would be less dense per line,
however that may be an upside when looked at it from version control and
diff-ing perspective.

> To me self documenting means that the file format doesn't even require a
> document to explain it's contents.  It should be self evident from the
> contents of the file.
What I meant by self documenting was that the document describing the
format and document implementing the format (actual source code) was the
same.
But even as evident from example above, proto text format yields file
contents that are pretty well self documented too.

> Changing file formats would be a
> benefit but why would we need to do that?  If we have a human readable
> file format that can be parsed easily and quickly by a computer, what
> other criteria do we need in a file format?

See benefit #3 from my first message. By using something standard we make
it so much easier to expand on KiCad ecosystem in other languages. I
already made an example as a web viewer, here is another: someone may write
a plugin for java autorouter software that will read kicad files directly.
S-expressions was likely a good choice when it was made but today it's far
from widespread and is pretty much unsupported in most languages so the
burden is wholly on the developers. Argument can be made that while
S-expressions are both human and computer readable it excels at neither
since humans still need supporting documentation or source and computers
need custom written libraries.

Regards,
Andrew

and Happy New Year :)

On Wed, Jan 2, 2019 at 8:01 AM Wayne Stambaugh  wrote:

> On 1/2/2019 5:24 AM, kristoffer Ödmark wrote:
> > I like the idea of using something as Protobuf and I agree fully with
> > the benefits, especially since one can add/remove fields with minimal
> > impact.
>
> There are some interesting and practical concepts with protobuf but it's
> functionally a binary storage method which I am opposed to.  Encoding
> and decoding to a text format would be an acceptable solution.  There is
> also the issue of learning curve and another build dependency.
>
> >
> > Basically the S-expression system used now is looking very much like a
> > reinvented XML to me anyway, and storing protobuf-defined stuff as XML
> > or similar seems actually nice.
>
> S-expr is not at all like XML at least not in terms of readability.
> Obviously there are an infinite number of ways to store information.  I
> do find it amusing and somewhat telling that there are so many markdown
> formats available these days.  I think the jury has spoken on the
> readability of markup formats.
>
> > There is one catch, and that is that we have to support opening a newer
> > file, in an old 

Re: [Kicad-developers] [RFC] Symbol library file format

[Simon Richter]

Hi Wayne,

On 02.01.19 23:20, Wayne Stambaugh wrote:

>> This would be an artificial unit for the file format, not necessarily
>> the true internal unit (which I want to unify across the entire program
>> as soon as feasible, but that should be independent from the file formats).

> I'm not sure exactly what you mean by a true internal unit.

The unit we're currently using internally (1nm for pcbnew, 1mil for
eeschema, etc). This one shouldn't leak into the file format IMO.

>> The other thing I'd like to see are standard properties for tagging
>> library conformance, so components that have been verified by library
>> maintainers would contain the name, version and variant of the library
>> conventions they have been tested against, and these tags would be
>> removed when editing.

[...]

> I'm not sure it's a good idea to embed this information in symbol
> libraries because how would it be defined for a user (non-KiCad)
> library.  It could give users a false impression that a library is an
> official library if it's abused.

That's why it would be nice to have name/version/variant. Official libs
would use "KLC" as the name, while users can define their own in-house
rules if they need to.

   Simon



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Rene Pöschl]

I have a few questions regarding multi part (or multi unit) symbols. (I 
do not really see a clear indication for these features in your document.)


Is there a definition for having only some units exchangeable?
Is there a way to define a unit as "must be placed" or "optional"?
Is there a way to mark a unit as "power unit" (I think you mentioned 
sometime back that this might be required to properly make simulation 
for multi unit symbols possible while still allowing the power unit to 
be separate. A power unit would also be a "must be placed unit" but i 
feel separating these two might add more flexibility.)


On 01/01/19 20:59, Wayne Stambaugh wrote:

I have updated and published the symbol file format[1] for comment.
Hopefully there isn't too much to change.  The only thing to really
finalize is the internal units.  The initial concept was unitless but
the more I think about it and discuss with other developers, it makes
more sense to use units for the following reasons:

1. It's easier to visualize in your head how the symbols on a given page
size will layout.

2. Converting from other file formats (Eagle, Altium, etc) will be
easier since most if not all of them have a defined unit.

I'm thinking 10u (or possibly 100u) will make a good internal units
value.  Once we nail down the units, I will update the file format
document accordingly.

Please keep in mind that this is the symbol library file format document
so things like constraints belong in the schematic file format.  I will
be posting the schematic file format as soon as I finish updating it.

Cheers,

Wayne

[1]:
https://docs.google.com/document/d/1lyL_8FWZRouMkwqLiIt84rd2Htg4v1vz8_2MzRKHRkc/edit


___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp




___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Wayne Stambaugh]

Hey Simon,

On 1/1/2019 5:03 PM, Simon Richter wrote:
> Hi Wayne,
> 
> On 01.01.19 20:59, Wayne Stambaugh wrote:
> 
>> I have updated and published the symbol file format[1] for comment.
>> Hopefully there isn't too much to change.  The only thing to really
>> finalize is the internal units.
> 
> This would be an artificial unit for the file format, not necessarily
> the true internal unit (which I want to unify across the entire program
> as soon as feasible, but that should be independent from the file formats).

I'm not sure exactly what you mean by a true internal unit.

> 
> If we want to keep using the grid as a poor man's magnetic anchor, it
> would perhaps make sense to just state the grid size in the component,
> and then use integer grid coordinates in the file for anything that is
> electrically connected. Users could then standardize on whatever grid
> size they prefer, get warnings for mixing components with different grid
> sizes, and/or get the option to scale the component so it fits.

Once the new tool frame work is in place in the schematic and symbol
editors, this should be less of an issue.  We should be able to add pin
connection snapping so it doesn't matter if the pin is on grid other
than it may make for some really ugly schematics.  I'm not too worried
about that other than our standard libraries stick to a constant pin
position.

> 
> For graphical parts, we need finer resolution than the grid, but
> inaccuracies no longer matter, so we can just use floating point numbers
> here.
> 
> I have several feature ideas I'd like to add at later points, so it
> would be nice to have a format version number and/or a list of format
> extensions somewhere near the beginning.

I accidentally left that out but I will add a header similar to the
footprint file format which includes the file version time stamp.

> 
> The other thing I'd like to see are standard properties for tagging
> library conformance, so components that have been verified by library
> maintainers would contain the name, version and variant of the library
> conventions they have been tested against, and these tags would be
> removed when editing.
> 
> This will allow for easier tracking of components through format
> updates, for example the new format adds pin swapping hints, which
> presumably the next version of the KLC will require for parts with
> swappable pins, but doesn't now because the feature doesn't exist yet.
> 
> Converted parts from the old library would be marked as conformant with
> an older KLC version, so the differences between the KLC versions can be
> used as a checklist to bring the component up to the current standard
> without requiring a full recheck.

I'm not sure it's a good idea to embed this information in symbol
libraries because how would it be defined for a user (non-KiCad)
library.  It could give users a false impression that a library is an
official library if it's abused.

Cheers,

Wayne

> 
>Simon
> 
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Vesa Solonen]

kristoffer Ödmark kirjoitti 2.1.2019 klo 12.32:

> I would rather have a new atomic file-format or storage format, so one
> could actually send a complete part in an easy way, that would include
> 3D-Files, SPICE data, symbol and footprints, and it would also mean that
> one could do the mapping to pins in that directory or file instead.
> 
> And since the discussion about adding a manufacturer part number to
> libraries went nowhere, I cannot see how adding a SPICE model would fit
> here, and if SPICE models are added. Well then I suggest we add 3D-Files
> and Footprints as well, no use in half-assing it.

Agreed. Considering how much has changed (also outside KiCad) since the
new symbol format was drafted it may be worth a second look at the issues.

I would suggest having a second look with Horizon EDA if there is any
common ground as there are certainly only things to be gained from some
cooperation:

https://github.com/carrotIndustries/horizon/wiki/The-Pool

-Vesa

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Wayne Stambaugh]

On 1/2/2019 5:24 AM, kristoffer Ödmark wrote:
> I like the idea of using something as Protobuf and I agree fully with
> the benefits, especially since one can add/remove fields with minimal
> impact.

There are some interesting and practical concepts with protobuf but it's
functionally a binary storage method which I am opposed to.  Encoding
and decoding to a text format would be an acceptable solution.  There is
also the issue of learning curve and another build dependency.

> 
> Basically the S-expression system used now is looking very much like a
> reinvented XML to me anyway, and storing protobuf-defined stuff as XML
> or similar seems actually nice.

S-expr is not at all like XML at least not in terms of readability.
Obviously there are an infinite number of ways to store information.  I
do find it amusing and somewhat telling that there are so many markdown
formats available these days.  I think the jury has spoken on the
readability of markup formats.

> There is one catch, and that is that we have to support opening a newer
> file, in an old software, and then store it again, without losing data
> that the software is not aware of. Or implement a way of not being able
> to store values in older software, when they open something newer.

This is the reason that we have not implemented this in our own file
formats.  I don't see anyone who would be happy about someone loosing
information by saving a board file with an older version of KiCad.  We
could always warn users when saving with a version of kicad that is
older than the file format but even that may cause unexpected loss of data.

> 
> There is also a middle way here, and that is to actually implement a
> Protobuf to S-Expression decoder/encoder, with the real benefit of
> actually defining fields in a modern well-known way, where the
> specification and implementetation does not have to manually be synced
> in code, comments, and a google doc. I have yet to see anything actually
> stay synchronized in such a manner over time, and many bugs manifest
> themself in these synchronization attempts. Anyway to avoid having to
> change the file-format another time, or add extra files to the side, I
> think that using an IDF is great next-step, mostly since the tooling,
> libraries and workflows for these are better defined.

To me self documenting means that the file format doesn't even require a
document to explain it's contents.  It should be self evident from the
contents of the file.  If it isn't, you've done something wrong.  The
only reason I published the file format is so I can get everyone's input
to make sure we have everything we need for the new features we plan to
implement during v6.  I expect over time that this document will not be
kept up to date even though it probably should be.

Writing an s-expr encoder and decoder is not likely to be a trivial task
so finding someone who has the time to implement it for an IDF is
probably low.

> 
> But to be honest, I have a hard time understanding why we have to stick
> to the KiCad S-Expression, when there are quite readable text-formats
> that are widely supported already.
> 
> I know the requirement for the file format is readability, but I have
> yet to find and editor that actually understands the KiCad S-Expression
> (I have not searched extensively), but JSON,XML,YAML are usually read
> just fine, with syntax highlighting out of box. And an IDF would make
> these discussions quite reduntant, since changing file formats would be
> a minimal change in code, and not as now, where it is actually quite
> time-consuming.

I wouldn't be opposed to JSON although I still think that it is more
verbose than necessary.  XML was rejected by the project along time ago
and I've seen nothing to change my mind about that.  I am not familiar
with YAML.

I doubt using an IDF will make these discussions redundant because there
will always be disagreements about file formatting irregardless of how
the information is defined internally.  Changing file formats would be a
benefit but why would we need to do that?  If we have a human readable
file format that can be parsed easily and quickly by a computer, what
other criteria do we need in a file format?

Cheers,

Wayne

> 
> - Kristoffer
> 
> On 2019-01-02 01:37, Andrew Lutsenko wrote:
>> Hi Wayne,
>>
>> I would like to take this opportunity to do an elevator pitch for idea
>> of using one of IDL languages widely accepted in the industry like
>> Apache Thrift or Google Protobufs to define formats in KiCad.
>> There are few large benefits in favor of using such languages:
>>
>> 1. They are self documenting. No more keeping a google doc in sync
>> with sources.
>> 2. They are easily extensible. Just add a field, old parsers will
>> ignore it, new ones will pick it up. Need to deprecate a field? Just
>> add it's ID to reserved list to never reuse it again.
>> 3. They have code generators for pretty much all commonly used
>> languages. That means anyone can pick 

Re: [Kicad-developers] [RFC] Symbol library file format

[Wayne Stambaugh]

I have no intention of embedding spice models in KiCad symbol files.
The current method of assigning spice information to symbols using
fields works very well so I don't see any reason to change it.

Cheers,

Wayne

On 1/2/2019 5:32 AM, kristoffer Ödmark wrote:
> I agree with Carsten Here,
> 
> The same way the footprints are not part of the symbol, the same way
> SPICE models should be defined separate.
> 
> I would rather have a new atomic file-format or storage format, so one
> could actually send a complete part in an easy way, that would include
> 3D-Files, SPICE data, symbol and footprints, and it would also mean that
> one could do the mapping to pins in that directory or file instead.
> 
> And since the discussion about adding a manufacturer part number to
> libraries went nowhere, I cannot see how adding a SPICE model would fit
> here, and if SPICE models are added. Well then I suggest we add 3D-Files
> and Footprints as well, no use in half-assing it.
> 
> - Kristoffer
> 
> On 2019-01-02 11:27, Thomas Pointhuber wrote:
>> Hi Carsten,
>>
>> Unfortunate, I don't think the reality works like that. While my
>> experience with SPICE is quite sparse (especially inside KiCad), I do
>> NOT think in this case about open and free spice model for IC's. Not
>> everyone uses the KiCad supplied libraries but their own, and simple
>> spice stuff like putting some resistors together for an R-Network can be
>> done in the official as well.
>>
>> Fully separate spice out of the symbol would not work anyway because
>> from what I noticed spice pin does not map to symbol pin in many cases.
>> You need to link that somewhere anyway. I'm heavily in favor of
>> simplifying handling of simulation for users. Integrating this into one
>> file would be less error-prone than separating simulation into multiple
>> additional files. Including external SPICE models would then be an
>> option, but not necessary.
>>
>> A modular approach where SPICE is only one of many possibilities is in
>> my favor, but in a different way than you proposed (integrated in the
>> symbol). There is for example the topic of digital simulation, which
>> would require different types of models. We should at least think about
>> it so it can be integrated later in a nice way.
>>
>> Regards,
>> Thomas
>>
>> Am 02.01.19 um 11:05 schrieb Carsten Schoenert:
>>> Hello Thomas,
>>>
>>> Am 02.01.19 um 10:39 schrieb Thomas Pointhuber:
 * I do not see anything related to SPICE in this document. I would vote
 to add it including the possibility to embedded spice models
 (BASE64-encoded) into the symbol itself.
>>> I disagree on that.
>>> Please do not mix two different main purposes and add more complexity.
>>>
>>> Schematic simulation is a add-on, not a primary key feature of a symbol
>>> format or within Eeeschema.
>>>
>>> Including such stuff into a schematic symbol make it harder to maintain
>>> the symbols in along term as you always need to touch the symbol no
>>> matter what peace need a update. But the (new extended) symbol format
>>> can get of course a new field of course for a referenced file to look
>>> at. Like similar done for associated footprints. Add a new environment
>>> variable like SPICE_MODEL_DIR to add a default folder.
>>>
>>> Currently it's not clear how a open and free spice working model can
>>> look like, I really suggest to not create a own world for spice models.
>>> Make it modular so work needs to be done more than needed.
>>>
>>> I also suggest to get in touch with Holger from the ngspice project
>>> about his thoughts and possible suggestions on this topic.
>>>
>>
>> ___
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to : kicad-developers@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
> 
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[kristoffer Ödmark]

I agree with Carsten Here,

The same way the footprints are not part of the symbol, the same way 
SPICE models should be defined separate.


I would rather have a new atomic file-format or storage format, so one 
could actually send a complete part in an easy way, that would include 
3D-Files, SPICE data, symbol and footprints, and it would also mean that 
one could do the mapping to pins in that directory or file instead.


And since the discussion about adding a manufacturer part number to 
libraries went nowhere, I cannot see how adding a SPICE model would fit 
here, and if SPICE models are added. Well then I suggest we add 3D-Files 
and Footprints as well, no use in half-assing it.


- Kristoffer

On 2019-01-02 11:27, Thomas Pointhuber wrote:

Hi Carsten,

Unfortunate, I don't think the reality works like that. While my
experience with SPICE is quite sparse (especially inside KiCad), I do
NOT think in this case about open and free spice model for IC's. Not
everyone uses the KiCad supplied libraries but their own, and simple
spice stuff like putting some resistors together for an R-Network can be
done in the official as well.

Fully separate spice out of the symbol would not work anyway because
from what I noticed spice pin does not map to symbol pin in many cases.
You need to link that somewhere anyway. I'm heavily in favor of
simplifying handling of simulation for users. Integrating this into one
file would be less error-prone than separating simulation into multiple
additional files. Including external SPICE models would then be an
option, but not necessary.

A modular approach where SPICE is only one of many possibilities is in
my favor, but in a different way than you proposed (integrated in the
symbol). There is for example the topic of digital simulation, which
would require different types of models. We should at least think about
it so it can be integrated later in a nice way.

Regards,
Thomas

Am 02.01.19 um 11:05 schrieb Carsten Schoenert:

Hello Thomas,

Am 02.01.19 um 10:39 schrieb Thomas Pointhuber:

* I do not see anything related to SPICE in this document. I would vote
to add it including the possibility to embedded spice models
(BASE64-encoded) into the symbol itself.

I disagree on that.
Please do not mix two different main purposes and add more complexity.

Schematic simulation is a add-on, not a primary key feature of a symbol
format or within Eeeschema.

Including such stuff into a schematic symbol make it harder to maintain
the symbols in along term as you always need to touch the symbol no
matter what peace need a update. But the (new extended) symbol format
can get of course a new field of course for a referenced file to look
at. Like similar done for associated footprints. Add a new environment
variable like SPICE_MODEL_DIR to add a default folder.

Currently it's not clear how a open and free spice working model can
look like, I really suggest to not create a own world for spice models.
Make it modular so work needs to be done more than needed.

I also suggest to get in touch with Holger from the ngspice project
about his thoughts and possible suggestions on this topic.



___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp



___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Thomas Pointhuber]

Hi Carsten,

Unfortunate, I don't think the reality works like that. While my
experience with SPICE is quite sparse (especially inside KiCad), I do
NOT think in this case about open and free spice model for IC's. Not
everyone uses the KiCad supplied libraries but their own, and simple
spice stuff like putting some resistors together for an R-Network can be
done in the official as well.

Fully separate spice out of the symbol would not work anyway because
from what I noticed spice pin does not map to symbol pin in many cases.
You need to link that somewhere anyway. I'm heavily in favor of
simplifying handling of simulation for users. Integrating this into one
file would be less error-prone than separating simulation into multiple
additional files. Including external SPICE models would then be an
option, but not necessary.

A modular approach where SPICE is only one of many possibilities is in
my favor, but in a different way than you proposed (integrated in the
symbol). There is for example the topic of digital simulation, which
would require different types of models. We should at least think about
it so it can be integrated later in a nice way.

Regards,
Thomas

Am 02.01.19 um 11:05 schrieb Carsten Schoenert:
> Hello Thomas,
> 
> Am 02.01.19 um 10:39 schrieb Thomas Pointhuber:
>> * I do not see anything related to SPICE in this document. I would vote
>> to add it including the possibility to embedded spice models
>> (BASE64-encoded) into the symbol itself.
> 
> I disagree on that.
> Please do not mix two different main purposes and add more complexity.
> 
> Schematic simulation is a add-on, not a primary key feature of a symbol
> format or within Eeeschema.
> 
> Including such stuff into a schematic symbol make it harder to maintain
> the symbols in along term as you always need to touch the symbol no
> matter what peace need a update. But the (new extended) symbol format
> can get of course a new field of course for a referenced file to look
> at. Like similar done for associated footprints. Add a new environment
> variable like SPICE_MODEL_DIR to add a default folder.
> 
> Currently it's not clear how a open and free spice working model can
> look like, I really suggest to not create a own world for spice models.
> Make it modular so work needs to be done more than needed.
> 
> I also suggest to get in touch with Holger from the ngspice project
> about his thoughts and possible suggestions on this topic.
> 



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[kristoffer Ödmark]

I like the idea of using something as Protobuf and I agree fully with 
the benefits, especially since one can add/remove fields with minimal 
impact.


Basically the S-expression system used now is looking very much like a 
reinvented XML to me anyway, and storing protobuf-defined stuff as XML 
or similar seems actually nice.
There is one catch, and that is that we have to support opening a newer 
file, in an old software, and then store it again, without losing data 
that the software is not aware of. Or implement a way of not being able 
to store values in older software, when they open something newer.


There is also a middle way here, and that is to actually implement a 
Protobuf to S-Expression decoder/encoder, with the real benefit of 
actually defining fields in a modern well-known way, where the 
specification and implementetation does not have to manually be synced 
in code, comments, and a google doc. I have yet to see anything actually 
stay synchronized in such a manner over time, and many bugs manifest 
themself in these synchronization attempts. Anyway to avoid having to 
change the file-format another time, or add extra files to the side, I 
think that using an IDF is great next-step, mostly since the tooling, 
libraries and workflows for these are better defined.


But to be honest, I have a hard time understanding why we have to stick 
to the KiCad S-Expression, when there are quite readable text-formats 
that are widely supported already.


I know the requirement for the file format is readability, but I have 
yet to find and editor that actually understands the KiCad S-Expression 
(I have not searched extensively), but JSON,XML,YAML are usually read 
just fine, with syntax highlighting out of box. And an IDF would make 
these discussions quite reduntant, since changing file formats would be 
a minimal change in code, and not as now, where it is actually quite 
time-consuming.


- Kristoffer

On 2019-01-02 01:37, Andrew Lutsenko wrote:

Hi Wayne,

I would like to take this opportunity to do an elevator pitch for idea 
of using one of IDL languages widely accepted in the industry like 
Apache Thrift or Google Protobufs to define formats in KiCad.

There are few large benefits in favor of using such languages:

1. They are self documenting. No more keeping a google doc in sync 
with sources.
2. They are easily extensible. Just add a field, old parsers will 
ignore it, new ones will pick it up. Need to deprecate a field? Just 
add it's ID to reserved list to never reuse it again.
3. They have code generators for pretty much all commonly used 
languages. That means anyone can pick KiCad file and just parse it in 
Java/Go/Haskell or whatever language they fancy without porting over 
s-expressions library or meticulously studying the file format doc. 
This opens lot's of possibilities for third party tools to be added to 
KiCad ecosystem. Writing a web viewer for schematic/pcb would be a 
piece of cake for example.


Other probably less impactful benefits:
4. Easy to serialize/encode in multiple formats. Need to send data 
over network in compact form? No problem, just serialize using compact 
binary protocol. Need to store in text file? just use text encoder.
5. Code generators will reduce amount of boilerplate in KiCad source. 
Only actual application logic needs to be added on top of generated 
data objects.


I have personally worked extensively with both Thrift and Protobufs, I 
think for KICad use case proto is better fit. Thrift has a lot more 
library support for client/server RPC logic and defining RPCs is core 
part of the language but we don't need any of that (at least for now). 
Proto has all of that as extensions but it's core is just definition 
of data types and it has better support for plain text format.

Here are docs for both:
https://developers.google.com/protocol-buffers/
https://thrift.apache.org/tutorial/

Let me know if any of that sounds interesting and if you have any 
questions. I think this is worth investing time into and I'm willing 
to help if needed.


Regards,
Andrew

On Tue, Jan 1, 2019 at 11:59 AM Wayne Stambaugh > wrote:


I have updated and published the symbol file format[1] for comment.
Hopefully there isn't too much to change.  The only thing to really
finalize is the internal units.  The initial concept was unitless but
the more I think about it and discuss with other developers, it makes
more sense to use units for the following reasons:

1. It's easier to visualize in your head how the symbols on a
given page
size will layout.

2. Converting from other file formats (Eagle, Altium, etc) will be
easier since most if not all of them have a defined unit.

I'm thinking 10u (or possibly 100u) will make a good internal units
value.  Once we nail down the units, I will update the file format
document accordingly.

Please keep in mind that this is the symbol library file 

Re: [Kicad-developers] [RFC] Symbol library file format

[Carsten Schoenert]

Hello Thomas,

Am 02.01.19 um 10:39 schrieb Thomas Pointhuber:
> * I do not see anything related to SPICE in this document. I would vote
> to add it including the possibility to embedded spice models
> (BASE64-encoded) into the symbol itself.

I disagree on that.
Please do not mix two different main purposes and add more complexity.

Schematic simulation is a add-on, not a primary key feature of a symbol
format or within Eeeschema.

Including such stuff into a schematic symbol make it harder to maintain
the symbols in along term as you always need to touch the symbol no
matter what peace need a update. But the (new extended) symbol format
can get of course a new field of course for a referenced file to look
at. Like similar done for associated footprints. Add a new environment
variable like SPICE_MODEL_DIR to add a default folder.

Currently it's not clear how a open and free spice working model can
look like, I really suggest to not create a own world for spice models.
Make it modular so work needs to be done more than needed.

I also suggest to get in touch with Holger from the ngspice project
about his thoughts and possible suggestions on this topic.

-- 
Regards
Carsten Schoenert

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Thomas Pointhuber]

Hi,

nice to see that the new file format is shaping up now. Some comments
from my side:

* I do not see anything related to SPICE in this document. I would vote
to add it including the possibility to embedded spice models
(BASE64-encoded) into the symbol itself.

* pin type as it is currently specified should be removed. Instead it
should be replaced with a more powerful electrical constraints
specification entity which includes the pin type, but allows the
addition of various other constrains as well, like:
  - output-voltage-range
  - voltage range for valid high/low signal
  - net/bus-type (e.g. I2C/SDA,...)
  - electrical hypergraph (to detect invalid connections between
galvanic separated net-groups)
  - ...

* specifying diff-pair in symbol itself without relying on a naming
convention?

* How are power symbols defined now?

Regards,
Thomas

Am 01.01.19 um 20:59 schrieb Wayne Stambaugh:
> I have updated and published the symbol file format[1] for comment.
> Hopefully there isn't too much to change.  The only thing to really
> finalize is the internal units.  The initial concept was unitless but
> the more I think about it and discuss with other developers, it makes
> more sense to use units for the following reasons:
> 
> 1. It's easier to visualize in your head how the symbols on a given page
> size will layout.
> 
> 2. Converting from other file formats (Eagle, Altium, etc) will be
> easier since most if not all of them have a defined unit.
> 
> I'm thinking 10u (or possibly 100u) will make a good internal units
> value.  Once we nail down the units, I will update the file format
> document accordingly.
> 
> Please keep in mind that this is the symbol library file format document
> so things like constraints belong in the schematic file format.  I will
> be posting the schematic file format as soon as I finish updating it.
> 
> Cheers,
> 
> Wayne
> 
> [1]:
> https://docs.google.com/document/d/1lyL_8FWZRouMkwqLiIt84rd2Htg4v1vz8_2MzRKHRkc/edit
> 
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Andrew Lutsenko]

Hi Wayne,

I would like to take this opportunity to do an elevator pitch for idea of
using one of IDL languages widely accepted in the industry like Apache
Thrift or Google Protobufs to define formats in KiCad.
There are few large benefits in favor of using such languages:

1. They are self documenting. No more keeping a google doc in sync with
sources.
2. They are easily extensible. Just add a field, old parsers will ignore
it, new ones will pick it up. Need to deprecate a field? Just add it's ID
to reserved list to never reuse it again.
3. They have code generators for pretty much all commonly used languages.
That means anyone can pick KiCad file and just parse it in Java/Go/Haskell
or whatever language they fancy without porting over s-expressions library
or meticulously studying the file format doc. This opens lot's of
possibilities for third party tools to be added to KiCad ecosystem. Writing
a web viewer for schematic/pcb would be a piece of cake for example.

Other probably less impactful benefits:
4. Easy to serialize/encode in multiple formats. Need to send data over
network in compact form? No problem, just serialize using compact binary
protocol. Need to store in text file? just use text encoder.
5. Code generators will reduce amount of boilerplate in KiCad source. Only
actual application logic needs to be added on top of generated data objects.

I have personally worked extensively with both Thrift and Protobufs, I
think for KICad use case proto is better fit. Thrift has a lot more library
support for client/server RPC logic and defining RPCs is core part of the
language but we don't need any of that (at least for now). Proto has all of
that as extensions but it's core is just definition of data types and it
has better support for plain text format.
Here are docs for both:
https://developers.google.com/protocol-buffers/
https://thrift.apache.org/tutorial/

Let me know if any of that sounds interesting and if you have any
questions. I think this is worth investing time into and I'm willing to
help if needed.

Regards,
Andrew

On Tue, Jan 1, 2019 at 11:59 AM Wayne Stambaugh 
wrote:

> I have updated and published the symbol file format[1] for comment.
> Hopefully there isn't too much to change.  The only thing to really
> finalize is the internal units.  The initial concept was unitless but
> the more I think about it and discuss with other developers, it makes
> more sense to use units for the following reasons:
>
> 1. It's easier to visualize in your head how the symbols on a given page
> size will layout.
>
> 2. Converting from other file formats (Eagle, Altium, etc) will be
> easier since most if not all of them have a defined unit.
>
> I'm thinking 10u (or possibly 100u) will make a good internal units
> value.  Once we nail down the units, I will update the file format
> document accordingly.
>
> Please keep in mind that this is the symbol library file format document
> so things like constraints belong in the schematic file format.  I will
> be posting the schematic file format as soon as I finish updating it.
>
> Cheers,
>
> Wayne
>
> [1]:
>
> https://docs.google.com/document/d/1lyL_8FWZRouMkwqLiIt84rd2Htg4v1vz8_2MzRKHRkc/edit
>
>
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] [RFC] Symbol library file format

[Simon Richter]

Hi Wayne,

On 01.01.19 20:59, Wayne Stambaugh wrote:

> I have updated and published the symbol file format[1] for comment.
> Hopefully there isn't too much to change.  The only thing to really
> finalize is the internal units.

This would be an artificial unit for the file format, not necessarily
the true internal unit (which I want to unify across the entire program
as soon as feasible, but that should be independent from the file formats).

If we want to keep using the grid as a poor man's magnetic anchor, it
would perhaps make sense to just state the grid size in the component,
and then use integer grid coordinates in the file for anything that is
electrically connected. Users could then standardize on whatever grid
size they prefer, get warnings for mixing components with different grid
sizes, and/or get the option to scale the component so it fits.

For graphical parts, we need finer resolution than the grid, but
inaccuracies no longer matter, so we can just use floating point numbers
here.

I have several feature ideas I'd like to add at later points, so it
would be nice to have a format version number and/or a list of format
extensions somewhere near the beginning.

The other thing I'd like to see are standard properties for tagging
library conformance, so components that have been verified by library
maintainers would contain the name, version and variant of the library
conventions they have been tested against, and these tags would be
removed when editing.

This will allow for easier tracking of components through format
updates, for example the new format adds pin swapping hints, which
presumably the next version of the KLC will require for parts with
swappable pins, but doesn't now because the feature doesn't exist yet.

Converted parts from the old library would be marked as conformant with
an older KLC version, so the differences between the KLC versions can be
used as a checklist to bring the component up to the current standard
without requiring a full recheck.

   Simon



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

[Kicad-developers] [RFC] Symbol library file format

[Wayne Stambaugh]

I have updated and published the symbol file format[1] for comment.
Hopefully there isn't too much to change.  The only thing to really
finalize is the internal units.  The initial concept was unitless but
the more I think about it and discuss with other developers, it makes
more sense to use units for the following reasons:

1. It's easier to visualize in your head how the symbols on a given page
size will layout.

2. Converting from other file formats (Eagle, Altium, etc) will be
easier since most if not all of them have a defined unit.

I'm thinking 10u (or possibly 100u) will make a good internal units
value.  Once we nail down the units, I will update the file format
document accordingly.

Please keep in mind that this is the symbol library file format document
so things like constraints belong in the schematic file format.  I will
be posting the schematic file format as soon as I finish updating it.

Cheers,

Wayne

[1]:
https://docs.google.com/document/d/1lyL_8FWZRouMkwqLiIt84rd2Htg4v1vz8_2MzRKHRkc/edit


___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp