Calcite is a classic open source application. No ODBC driver but they support queries in pig latin.
On Sat, Aug 29, 2020 at 7:03 AM Wayne Stambaugh <stambau...@gmail.com> wrote: > BINGO! That didn't take long. I knew the non-sql database question > would rear it's ugly head. It just happened quicker than I thought it > would ;) > > On 8/29/20 9:58 AM, Tim Hawkins wrote: > > https://calcite.apache.org/docs/adapter.html > > > > Apache calcite is an SQL query engine for non-sql data sources. > > > > > > > > On Sat, Aug 29, 2020, 21:21 Jeff Young <j...@rokeby.ie > > <mailto:j...@rokeby.ie>> wrote: > > > > Surely there must be an open source impl of an ODBC interface on a > > CSV file? > > > > Although I’m not sure of the desire to avoid MySQL. It’s remarkably > > easy to set up an instance (or auto-deploy one with an app). > > > > Apologies if we’ve already talked about that; I’ll confess to not > > having followed this thread 100%…. > > > >> On 29 Aug 2020, at 14:11, Wayne Stambaugh <stambau...@gmail.com > >> <mailto:stambau...@gmail.com>> wrote: > >> > >> I would most likely reject any solution that was tied to a > particular > >> database. All this would do is open up a Pandora's box of > >> complaints as > >> to why we didn't use database A over database B. ODBC is the most > >> flexible solution that I am aware of and allows users to choose > their > >> preferred database. > >> > >> Cheers, > >> > >> Wayne > >> > >> On 8/29/20 8:29 AM, Jon Evans wrote: > >>> Putting aside the fact that I think this feature isn't really > >>> aimed at > >>> hobbyists, I would not be opposed to people wanting to extend it > >>> beyond > >>> ODBC but that comes with extra complexity that must be handled. > >>> > >>> With ODBC, KiCad just needs to know about the interface to > >>> retrieve the > >>> data. > >>> > >>> With a CSV file, KiCad actually needs to read that file in and > >>> keep it > >>> in memory. Watch for modifications on disk, or else lock it > >>> exclusively. Things like that. > >>> > >>> I'm not sure I really see the advantage of a CSV-backed > >>> "database" over > >>> the existing KiCad library system, if we're talking about a > >>> single user. > >>> > >>> -Jon > >>> > >>> On Sat, Aug 29, 2020 at 8:19 AM Mark Roszko > >>> <mark.ros...@gmail.com <mailto:mark.ros...@gmail.com> > >>> <mailto:mark.ros...@gmail.com>> wrote: > >>> > >>>> Sqlite was a quick&dirty way to test if my ideas would work. > >>> There are ODBC wrappers for SQLite........ > >>> > >>>> I mean libreoffice would do for the management. > >>> Yes, and you know what you would use? Not CSV files. > >>> LibreOffice Base / Microsoft Access. > >>> This is the office suite database that's basically SQLite and > >>> there > >>> are ODBC wrappers as well :D > >>> > >>>> Also, why would a hobbyist fire up a sql database when a CSV file > >>> would be sufficient? I mean libreoffice would do for the > >>> management. > >>> > >>> KiCad's uses aren't limited to hobbyists... > >>> And, you assume there aren't hobbyists like me who will gladly > >>> take > >>> that ODBC link and spin up an frontend in a few hours to the > whole > >>> system :D > >>> > >>> > >>> On Sat, Aug 29, 2020 at 4:22 AM Johann Wilhelm > >>> <johann.wilhelm@wilhelm.consulting > >>> <mailto:johann.wilhelm@wilhelm.consulting>> wrote: > >>> > >>> Hi there! > >>> > >>> Well, then my comment was not completely wrong. > >>> > >>> Sqlite was a quick&dirty way to test if my ideas would > >>> work. For > >>> my future productive system I really want to use a mix of > >>> couchdb and maybe postgres. i.e. a JSON document storage > >>> for the > >>> component information and sql for inventory management. > >>> > >>> So ODBC would not work well for me. > >>> > >>> Also, why would a hobbyist fire up a sql database when a CSV > >>> file would be sufficient? I mean libreoffice would do for > the > >>> management. > >>> > >>> Additionally, I'd suggest looking at the BOM creation. > There, > >>> external programs are called. > >>> > >>> So why not define a dataformat (xml, json, csv,...) and just > >>> call an external app or read from a file/uri? > >>> > >>> Anyways, I would volunteer for implementing some > alternatives > >>> (read from file/uri and output of executable) to the ODBC > >>> interface if someone guides me through the KiCad procedures. > >>> > >>> Regards, > >>> Johann > >>> > >>> > >>> Jon Evans <j...@craftyjon.com > >>> <mailto:j...@craftyjon.com> <mailto:j...@craftyjon.com>> schrieb > >>> am Fr., 28. Aug. 2020, 19:54: > >>> > >>> My idea is to make it possible for KiCad to talk to an > >>> external database. > >>> The database itself (and its schema) will not be > >>> defined as > >>> part of this spec, and will not be part of KiCad. > >>> > >>> The only requirement is that you have some columns in > your > >>> schema that KiCad understands (for example, to point > >>> to the > >>> right symbols and footprints in the KiCad libraries). > >>> The planned interface to connect to the database is > ODBC. > >>> This would in theory allow using sqlite to create a > >>> database > >>> as a file on disk, although I anticipate that most users > >>> will be using something like MariaDB or Postgres. > >>> > >>> There has been some discussion of supporting talking > >>> to web > >>> interfaces via some REST API, or even talking to > arbitrary > >>> interfaces via Python scripting, but that discussion > >>> should > >>> stay separate. > >>> The original thread was about getting component > >>> information > >>> out of a database and I just wanted to let people know > >>> that > >>> I am working on this. > >>> > >>> People are welcome to also discuss the ideas of getting > >>> component information from a web API or from some > >>> Python script. > >>> But, I am not working on that right now, and there > hasn't > >>> even been a conversation started on what that spec would > >>> look like. > >>> > >>> Best, > >>> Jon > >>> > >>> On Fri, Aug 28, 2020 at 1:42 PM Johann Wilhelm > >>> <johann.wilhelm@wilhelm.consulting > >>> <mailto:johann.wilhelm@wilhelm.consulting>> wrote: > >>> > >>> Hi Jon, > >>> > >>> Well, you're idea was to define a database or at > >>> least a > >>> database schema if I understood this correctly: > >>> > >>> What I plan on implementing is not a "full" > >>> database management system, but > >>> > >>> rather an interface to grab info out of a > >>> database, just like you say. > >>> > >>> The only difference is, there are no plans to > >>> store actual symbols or > >>> > >>> footprints in the database. > >>> > >>> The symbols and footprints will still be > >>> stored in "normal" KiCad library > >>> > >>> files; the database will just contain > >>> "pointers" telling KiCad which symbol > >>> > >>> to use (and what library it can be found in). > >>> > >>> > >>> > >>> I was pointing toward a specification of a data > >>> format.So either read the data from a file or > >>> webinterface (that's why I used URI). > >>> My script-set currently gives me that information > via > >>> http://localhost:8000/API/Component/getMatchWisdom > >>> > >>> Others would maybe like to save a file in ~/.KiCad > (so > >>> the URI would be file://~/.KiCad) > >>> > >>> If "database"/"database interface" would include > >>> something which could read from files and/or web > >>> resources, well, never mind my comment :) > >>> > >>> With "global parameter" I mean something which > >>> could be > >>> part of a .pro file. > >>> > >>> Regards, > >>> Johann > >>> > >>> > >>> Am Fr., 28. Aug. 2020 um 19:04 Uhr schrieb Jon Evans > >>> <j...@craftyjon.com > >>> <mailto:j...@craftyjon.com> <mailto:j...@craftyjon.com>>: > >>> > >>> Hi Johann, > >>> > >>> I am not sure exactly what you are arguing > >>> against. > >>> I don't see any difference between what you are > >>> looking for and what is in my spec. > >>> I am not sure what you mean by "global URI > >>> parameter" but the part picker will be able to > >>> filter using any of the external data present. > >>> > >>> Best, > >>> Jon > >>> > >>> On Fri, Aug 28, 2020 at 12:56 PM Johann Wilhelm > >>> <johann.wilhelm@wilhelm.consulting > >>> <mailto:johann.wilhelm@wilhelm.consulting>> wrote: > >>> > >>> Hi everyone! > >>> > >>> I'm new here in the mailing list but I'm > >>> currently building my own electronics > business > >>> around KiCad which I use and love for > >>> years now. > >>> > >>> I spent the last weeks, trying to tie > together > >>> procurement, PCB design, and fabrication in > a > >>> single script-set. > >>> Just to have it mentioned - before getting > >>> self-employed, I worked for 10 years in the > >>> electronics industry in a huge enterprise. > >>> So I had some strict requirements for my > >>> tooling > >>> and I have some very strong opinions on how > my > >>> perfect workflow should look like :) > >>> > >>> I'm very sorry but I need to say: PLEASE, > >>> don't > >>> just throw a part database at the community! > >>> Why? Because everyone has different > >>> approaches how to do procurement and > inventory > >>> management! > >>> It's ok (and actually good!) if you try to > >>> come > >>> up with something but PLEASE, go for a > defined > >>> filter interface! > >>> > >>> What I would suggest implementing - > >>> actually, I > >>> have plans to implement it myself in > >>> mid-term - > >>> is a simple checkbox in the component dialog > >>> named "Apply Parts Filter" and a global URI > >>> parameter that provides the filter (or > better: > >>> the source of "wisdom" used by the filter). > >>> Maybe another parameter defining the > >>> default for > >>> the checkbox would be nice as well... > >>> > >>> I think even a simple CSV with columns for > >>> Symbol, Footprint, and Value could provide > >>> sufficient information for effective > >>> filtering/adaption of the symbol-tree. > >>> > >>> A CSV like this: > >>> > >>> > >>> > "Audio:TLV320AIC23BPW", "TLV320AIC23BPW", > "Package_SO:TSSOP-28_4.4x9.7mm_P0.65mm" > >>> > "Device:R_Small", "1k", > "Resistor_SMD:R_0805_2012Metric" > >>> "Device:R*", "12k4", > >>> "Resistor_SMD:R_0805_2012Metric" > >>> > "Device:R*", "12k4", > "Resistor_SMD:R_1206_3216Metric" > >>> > "Device:R*", "10R", > "Resistor_SMD:R_0805_2012Metric" > >>> > >>> > >>> should result in a component tree like this: > >>> > >>> Audio > >>> > >>> TLV320AIC23BPW <-- this symbol > >>> has a > >>> matching default footprint and value > >>> > >>> Device > >>> > >>> R <-- this symbol is included in > the > >>> CSV data > >>> > >>> 12k4 <-- a value of 12k4 > would > >>> match Device:R and a > corresponding > >>> symbol is added by the filter > >>> > >>> > Resistor_SMD:R_0805_2012Metric > >>> <--- both footprints would > >>> match Device:R 12k4 so the > >>> filter adds both symbols > with > >>> the footprint and values > >>> fields > >>> complemented > >>> > Resistor_SMD:R_0805_2012Metric > >>> <--- .... > >>> > >>> 10R > >>> > >>> > Resistor_SMD:R_0805_2012Metric > >>> > >>> R_Small > >>> > >>> 1k <-- 1k only matches > >>> Device:R_Small > >>> > >>> > Resistor_SMD:R_0805_2012Metric > >>> > >>> 12k4 > >>> > >>> > Resistor_SMD:R_0805_2012Metric > >>> > Resistor_SMD:R_0805_2012Metric > >>> > >>> 10R > >>> > >>> > Resistor_SMD:R_0805_2012Metric > >>> > >>> > >>> So the filter should add all symbols > >>> referenced > >>> in the CSV. > >>> If a symbol has no matching footprint and/or > >>> value => "create" them on fly > >>> > >>> With this, you can use the whole symbol > >>> library > >>> or the existing parts in your inventory. > >>> The best of both worlds! > >>> > >>> Regards, > >>> Johann > >>> > >>> > >>> Am Fr., 28. Aug. 2020 um 16:38 Uhr schrieb > >>> Brian > >>> <lotha...@gmail.com > >>> <mailto:lotha...@gmail.com> <mailto:lotha...@gmail.com>>: > >>> > >>> Just want to add my +1 for the interface > >>> approach. I am glad to hear that is the > >>> intent (I’ve not had time to read the > >>> proposal). With such an interface, as > has > >>> been pointed out already, > >>> data-source-specific implementations > >>> should > >>> be relatively straightforward, and then > I > >>> don’t have to potentially throw away my > >>> years of privately curated data or > figure > >>> out how to cram it into some alternate > >>> schema. All I would need to do is > >>> write the > >>> code to answer the questions presented > by > >>> the interface. > >>> > >>> Hopefully in the next few days I’ll be > >>> able > >>> to read Jon’s draft spec and comment > >>> from a > >>> better-informed position. > >>> > >>> Cheers, > >>> -Brian > >>> > >>>> On Aug 28, 2020, at 9:43 AM, Jon Evans > >>>> <j...@craftyjon.com > >>>> <mailto:j...@craftyjon.com> > >>>> <mailto:j...@craftyjon.com>> wrote: > >>>> > >>>> > >>>> Hi Clemens, > >>>> > >>>> On Fri, Aug 28, 2020 at 9:34 AM Clemens > >>>> Koller <c...@embeon.de > >>>> <mailto:c...@embeon.de> > >>>> <mailto:c...@embeon.de>> wrote: > >>>> > >>>> Hello! > >>>> > >>>> This is related to the previous > >>>> thread: "Automatic assignment of > >>>> footprint with a database" > >>>> > >>>> I would generally prefer assemble > >>>> real > >>>> components on a real PCB right from > >>>> the beginning instead of first > >>>> placing > >>>> generic components and then assign > >>>> footprints + manufacturers + > >>>> types + x > >>>> manually. This seems extra work for > >>>> each component which could > >>>> possibly be > >>>> avoided. > >>>> > >>>> > >>>> Me too! > >>>> > >>>> > >>>> So, regarding on the Kicad > >>>> codebase, I > >>>> would very likely not recomment to > >>>> embed a full component management / > >>>> database system, since this might > >>>> vary > >>>> from site to site and even from > >>>> project/assembly house to > >>>> project/house. But it would be > great > >>>> to be able to have an interface to > >>>> grab a component out of a > >>>> database and > >>>> Kicad grabs all desired / a > selection > >>>> of individual columns out of the > >>>> database as needed. > >>>> This might include the actual > >>>> footprints stored in the database > >>>> as well. > >>>> > >>>> > >>>> What I plan on implementing is not a > >>>> "full" database management system, but > >>>> rather an interface to grab info out > of a > >>>> database, just like you say. > >>>> The only difference is, there are no > >>>> plans > >>>> to store actual symbols or footprints > in > >>>> the database. > >>>> The symbols and footprints will still > be > >>>> stored in "normal" KiCad library files; > >>>> the database will just contain > "pointers" > >>>> telling KiCad which symbol to use (and > >>>> what library it can be found in). > >>>> > >>>> BTW, the example schema in your email > >>>> looks very familiar to me. This is the > >>>> kind of data source that can be used > with > >>>> the feature I am talking about: just > add > >>>> columns to the schema for tracking > which > >>>> KiCad symbol and footprint(s) should be > >>>> used for a part. > >>>> > >>>> Regarding the advanced features you > >>>> mention, some of them sound like they > >>>> would be handled by a PLM tool. > >>>> Some of the PLM tools I have worked > with > >>>> can interface to external databases for > >>>> managing this kind of component library > >>>> for an EDA or mechanical CAD tool. > >>>> > >>>> Best, > >>>> Jon > >>>> > _______________________________________________ > >>>> Mailing list: > >>>> > https://launchpad.net/~kicad-developers > >>>> Post to : > >>>> kicad-developers@lists.launchpad.net > >>>> <mailto:kicad-developers@lists.launchpad.net> > >>>> <mailto: > kicad-developers@lists.launchpad.net> > >>>> Unsubscribe : > >>>> > https://launchpad.net/~kicad-developers > >>>> More help : > >>>> https://help.launchpad.net/ListHelp > >>> > _______________________________________________ > >>> Mailing list: > >>> https://launchpad.net/~kicad-developers > >>> Post to : > >>> kicad-developers@lists.launchpad.net > >>> <mailto:kicad-developers@lists.launchpad.net> > >>> <mailto: > kicad-developers@lists.launchpad.net> > >>> Unsubscribe : > >>> https://launchpad.net/~kicad-developers > >>> More help : > >>> https://help.launchpad.net/ListHelp > >>> > >>> > _______________________________________________ > >>> Mailing list: > >>> https://launchpad.net/~kicad-developers > >>> Post to : > >>> kicad-developers@lists.launchpad.net > >>> <mailto:kicad-developers@lists.launchpad.net> > >>> <mailto: > kicad-developers@lists.launchpad.net> > >>> Unsubscribe : > >>> https://launchpad.net/~kicad-developers > >>> More help > >>> : https://help.launchpad.net/ListHelp > >>> > >>> _______________________________________________ > >>> Mailing list: https://launchpad.net/~kicad-developers > >>> Post to : kicad-developers@lists.launchpad.net > >>> <mailto:kicad-developers@lists.launchpad.net> > >>> <mailto:kicad-developers@lists.launchpad.net> > >>> Unsubscribe : https://launchpad.net/~kicad-developers > >>> More help : https://help.launchpad.net/ListHelp > >>> > >>> > >>> > >>> -- > >>> Mark > >>> > >>> > >>> _______________________________________________ > >>> Mailing list: https://launchpad.net/~kicad-developers > >>> Post to : kicad-developers@lists.launchpad.net > >>> <mailto:kicad-developers@lists.launchpad.net> > >>> Unsubscribe : https://launchpad.net/~kicad-developers > >>> More help : https://help.launchpad.net/ListHelp > >>> > >> > >> _______________________________________________ > >> Mailing list: https://launchpad.net/~kicad-developers > >> Post to : kicad-developers@lists.launchpad.net > >> <mailto:kicad-developers@lists.launchpad.net> > >> Unsubscribe : https://launchpad.net/~kicad-developers > >> More help : https://help.launchpad.net/ListHelp > > > > _______________________________________________ > > Mailing list: https://launchpad.net/~kicad-developers > > Post to : kicad-developers@lists.launchpad.net > > <mailto:kicad-developers@lists.launchpad.net> > > Unsubscribe : https://launchpad.net/~kicad-developers > > More help : https://help.launchpad.net/ListHelp > > > > _______________________________________________ > Mailing list: https://launchpad.net/~kicad-developers > Post to : kicad-developers@lists.launchpad.net > Unsubscribe : https://launchpad.net/~kicad-developers > More help : https://help.launchpad.net/ListHelp >
_______________________________________________ Mailing list: https://launchpad.net/~kicad-developers Post to : kicad-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~kicad-developers More help : https://help.launchpad.net/ListHelp
