For now I've moved everything under a GitHub organization:
https://github.com/rustrpm
The Rust bindings to librpm are now located at:
https://github.com/rustrpm/librpm-rs
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
@dmach would you consider creating a repository under
https://github.com/rpm-software-management/ where I can push my Rust bindings?
Alternatively I could open a PR to
https://github.com/rpm-software-management/rpm containing them
Or if you'd like to wait until they're a bit more mature, I can
@tarcieri If you give @dmach access, he can move it.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
I think it's a bit hard to move things that way (without making me an admin),
but given it's a brand new repo I can either:
1) Just push all the same code to a newly created repo under
`rpm-software-management` or
2) Give someone admin access to `librpm-rs` so they can move it
--
You are
@dmach Can we get @tarcieri the ability to move`librpm-rs` to the
rpm-software-management organization?
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
I've renamed the project to librpm.rs:
https://crates.io/crates/librpm
I've extracted it into its own Git repo here, with everything licensed as
LGPLv2.1+:
https://github.com/tarcieri/librpm-rs
I've also changed the binding to move everything relating to match iterators
and headers into an
> Alright, perhaps I can take a step back from the reference-based stuff and
> take a look at creating "owned" values... although my preferred way to do
> that in Rust would be for Rust to make a copy itself, as opposed to having
> rpm allocate the memory and me having to call rpm again to free
@tarcieri: I am not particularly fond of XML either, but using headerFormat()
queryformat to export data from a header simplifies bindings for what is
usually desired: masses of RO metadata about rpm packages, an entirely well
understood programming technique.
Note that rpm -q --xml is also
@n3npq can't say I'm super thrilled with the prospects of replacing access to
an in-memory data structure with something based on XML
>As for handling tag data, you probably don't want to expose rpmtd to rust
>directly but convert to native representation [...]
> Worse, any data addition to a
@tarcieri , headers exist outside database iterators and such, and you really
dont want to require keeping potentially huge headers (tens of megabytes
possibly) in memory just in order to remember, say, a name of a package. Worse,
any data addition to a header can make the data retrieved with
There's also an alternative way to buffer all rpmlog() output.
If rust has some reasonable XML support, then data can be exported from a
header into whatever structure you wish rather effortlessly.
E.g. try "rpm -q --xml bash". The --xml is just a queryformat disguised as a
CLI option using a
Overriding qva->qva_showpackage with a short routine that does str =
headerFormat() is what I was recalling having implemented.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
Ah my bad: yes, the output is delivered to stout/stderr. And yes rpmcliQuery.
Meanwhile, all the output for each header ends up in a single buffer and a
single fprintf iirc. So it would not be hard to deliver the headerFormat output
for each header through a callback.
Delivery of textual info
@n3npq where is `rpmQuery` defined? (is it `rpmcliQuery`?)
This is all I could find, and it wasn't immediately obvious how to read results
from it:
http://ftp.rpm.org/api/4.12.0.1/group__rpmcli.html#ga2dbfa3886628a79646b2c286ea5c56d9
--
You are receiving this because you are subscribed to
Again, look at headerFormat() bindings which is kind of like asprintf(3) for
headers and you can then parse the string into whatever rust types you wish
within rust.
I previously suggested you might want to bind the high level rpmQuery() same as
the CLI rpm --query --queryformat ... command to
See lib/rpmtag.h: where the tags are defined. The usual type is appended in the
comment.
There is also rpm -vv --querytags which displays the known tags and types (if
RPM4 doesn't display the types, then an RFE is appropriate)
--
You are receiving this because you are subscribed to this
Garrison "... possibly with count 1". Specifically a RPM_CHAR_TYPE returns a
pointer to data and count is the number of chars in that array. A CHAR scalar
is returned as an array with count 1, ditto for INT* types.
--
You are receiving this because you are subscribed to this thread.
Reply to
@n3npq I am not... definitely good to know as the current code is bogus.
What'd be really helpful in regard to testing tag data is a tag type which
corresponds to each of the tag data types, particularly arrays/vectors of the
integer types.
--
You are receiving this because you are subscribed
(After perusing the td.rs code: perhaps I am misreading some assertions)
Hmmm ... you do realize that only BIN/STRING/I18NSTRING types are scalars?
Specifically, that means that CHAR and INT* types return a pointer to an array
with count items.
Of course BIN is an array of octets, and STRING
@pmatilai using "owned" memory instead of borrowed, in Rust terminology, is an
alternative, and one that could potentially eliminate any possible
use-after-free vulnerabilities.
However, building zero copy APIs on top of things like `HEADERGET_MINMEM` is
something of Rust's raison d'etre, and
I'd really recommend avoiding HEADERGET_MINMEM entirely because of all the
hidden implications, just use HEADERGET_ALLOC always so you can trust the data
isn't going to vanish or move underneath you, like n3npq said you need to
manage unallocation anyway for string arrays.
As for handling tag
I've opened a PR which implements a bunch of @n3npq's suggestions:
https://github.com/iqlusion-io/crates/pull/28/files
Namely:
- No more `Database` type. Instead `db.rs` is a handful of static functions for
querying the database
- `TransactionSet` is no-longer part of the public API (although
In practice, there is usually a ref count held on a header by a higher level
container like a ts or a mi.
In C it's usually quick to use valgrind, or compile with -fsanitize=address to
fix occasional mistakes. I don't know enough about rust tools used to check
memory problems at runtime,
There are 3-4 levels of memory usage with header tag data using MINMEM:
1) the rpmtd container with malloc/free, non-refcounted
2) (string arrays only) the array of pointers to strings, free'd on release
3) the data, protected by the header ref count
4) (if used) string pools to memoize/uniqify
@n3npq I'm working on simplifying the API and also switching from a
`StreamingIterator` to a regular `Iterator` by using reference counting on
header data. I had a question about this though:
> The trickiest problem with header tag data and MINMEM is that the header
> reference count sometimes
@n3npq yeah I've been meaning to add some sort of higher level facade around
`Header` which makes it easy to do the obvious things like get the package's
name and description, but I wasn't quite sure what it would look like yet.
@Conan-Kudo thanks!
--
You are receiving this because you are
@tarcieri The rpm.org/rpm4 documentation can be found here:
http://ftp.rpm.org/api/4.14.0/
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
Re hiding header objects
You an also hide headers in rust bindings.
There are 3 places that headers become visible:
1) when read from packages in order to create a transaction
This can be handled by passing *.rpm file names to rpmInstall or package names
to rpmErase
2) Most usage cases for an
Great! With that I can add a wrapper type for accessing it which acquires the
lock, and when dropped calls `rpmtsClean()` as the last thing before it
releases the lock.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
Re a global rpmts
There is rpmtsClean() to make the global rpmts reusable
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
> What I am suggesting (but I know nothing serious about rust) is running an
> array of header references during an iteration using *Link() that have
> *Free() called when the rust Iterator goes out of scope.
Yep, that's exactly what I'd like to do.
--
You are receiving this because you are
Re use of rust's regular Iterator type
Since you are programming with rust+ffi there are some games you may be able to
play with ref counting in C to adjust the lifetime of headers returned from an
Iterator.
What I am suggesting (but I know nothing serious about rust) is running an
array of
There are almost no interesting usage cases for multiple transactions (at least
until one attempts threaded installs, which are currently prevented by an
exclusive fcntl lock in RPM4 anyways).
What that means for bindings is that you can take out a static global
transaction which can be used
> Do an additional ref++ using *Link() if you want to make a header persistent
> for further processing outside of an iteration (but don't forget the ref--
> using *Free() or you will have a huge memory leak ;-)
Are there any additional lifetimes to worry about here, e.g. the lifetime of
the
Re the RPM5 doco
That do yen info hasn't been updated for almost a decade (but not much has
changed since last century).
(aside)
Meanwhile there are some significant differences in RPM5 match iterator S,
including lazy index generation, partial matches on patterns applied to Bree
index keys
Re using rust iterator S w ref counting
The match iterator does a ref++ followed by a ref-- at the end of an iteration
which guarantees that the header and MINMEM tag data exist for each iteration.
Do an additional ref++ using *Link() if you want to make a header persistent
for further
Aha, so if I call this:
http://rpm5.org/docs/api/group__header.html#g4f07e8040ed3195e374a44919ffe97c2
...and `headerFree()` on drop, it sounds like I could switch to a normal Rust
iterator which iterates over immutable references.
--
You are receiving this because you are subscribed to this
Re: ref counting.
Yes, an increment is exactly a *Link() call, and a decrement (and possible
delayed/lazy de-allocation) is a * Free() call.
There is also a *Unlink() method that is strictly a decrement w/o a
deallocation, but that is really only useful when debugging, or when some
snarly ref
Actually, rpmtsInitIterator() is/was an attempt to simplify retrievals by
hiding the DB handle getter within a pass through lookup. Acceptance and
updating code has been glacially slow all century.
Note that rpmdbOpen's calls have had some very peculiar side effects to
accomodate user demands
@n3npq a few questions about something you said earlier:
> RPM object (like ts, h, db) is ref-counted, but the counter increment is not
> atomic.
When are the reference counts incremented/decremented, and can I bump them
externally? If so, I could have Rust increment the reference count when
(random comments on db.rs retrievals)
Note that mire (match iterator regular expressions) can be rather expensive if
patterns are used. rpm itself almost always uses strings through an index for
performance. A proper implementation for mire patterns would need to do a
partial match on the
I would have used `rpm`, unfortunately someone else took it for something
completely unrelated (and won't give it up, I already asked):
https://crates.io/crates/rpm
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
IMO it makes sense to use `rpm` + `rpm-sys` as a crate names.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
> These days, we tend to call it librpm rather than rpmlib
Haha oh really! What's great about that is I started down the road of naming
things `librpm` then changed to `rpmlib` after noticing all the references to
it: https://crates.io/crates/librpm-sys
I can re-release the crate as `librpm`
@tarcieri
A few things:
* These days, we tend to call it `librpm` rather than `rpmlib`, though of
course, the API still calls it `rpmlib` all over the place...
* That old document from Fedora is a terrible reference for how `librpm` can be
used. Some better examples can be found in the code
Thanks for all of the feedback @n3npq! I will go through it and try to fix up
my library accordingly.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
(From a quick perusal of your rust code)
You may need bindings for a FD_t since certain routines in the rpmlib API take
a FD_t.
There is also (in RPM4) a key ring object, basically a container for pubkeys.
However RPM avoided handling private keys last century "munitions" and
externalized
FWIW, there is little reason to expose a DB object per se in bindings. Instead
wrap the mi "match iterator" methods. There's very little that can be done
outside of rpm with a DB object because Berkeley DB is not simple programming,
and all known replacements for Berkeley DB are exactly that, a
RPM object (like ts, h, db) is ref-counted, but the counter increment is not
atomic.
With MINMEM, the tag data should be treated as if it were read-only (but during
header load/unload integer data will be swabbed).
The trickiest problem with header tag data and MINMEM is that the header
* STRING is ASCII, I18N tends to be UTF-8.
* (my guess) count is the same as count in header data (I will verify/correct
if my guess is wrong)
* count is the number of octets for binary data
* the CHAR data type should be treated as a uint8/int8. There are very few
(perhaps only one) tag with
Yep, I'll go through and fix things up to use that instead.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
By the way, RPM stands for RPM Package Manager nowadays.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
52 matches
Mail list logo
