Re: [Pulp-dev] Integer IDs in Pulp 3

2018-07-18 Thread David Davis
The change to use int ids[0] has been merged. When you pull from pulp/pulp, you’ll have to redo your migrations and recreate your database. The first option is recreate your dev environment if you’re running Vagrant. Alternatively, you can remove all the migrations from pulp (rm -rf

Re: [Pulp-dev] Integer IDs in Pulp 3

2018-07-12 Thread Brian Bouterse
I'm +1 on grooming that ticket and sprint nominating it. I commented on question there about how to handle RQ. On Wed, Jul 11, 2018 at 4:53 PM, Dennis Kliban wrote: > Thanks David. I am in favor of this change. > > On Wed, Jul 11, 2018 at 4:39 PM, David Davis > wrote: > >> There is now: >> >>

Re: [Pulp-dev] Integer IDs in Pulp 3

2018-07-11 Thread Dennis Kliban
Thanks David. I am in favor of this change. On Wed, Jul 11, 2018 at 4:39 PM, David Davis wrote: > There is now: > > https://pulp.plan.io/issues/3848 > > David > > > On Wed, Jul 11, 2018 at 4:23 PM Brian Bouterse > wrote: > >> A 30% improvement I think is a good case for integers over uuids.

Re: [Pulp-dev] Integer IDs in Pulp 3

2018-07-11 Thread David Davis
There is now: https://pulp.plan.io/issues/3848 David On Wed, Jul 11, 2018 at 4:23 PM Brian Bouterse wrote: > A 30% improvement I think is a good case for integers over uuids. > > Is there a ticket tracking that change? > > On Wed, Jul 11, 2018 at 3:55 PM, Daniel Alley wrote: > >> w/

Re: [Pulp-dev] Integer IDs in Pulp 3

2018-07-11 Thread Brian Bouterse
A 30% improvement I think is a good case for integers over uuids. Is there a ticket tracking that change? On Wed, Jul 11, 2018 at 3:55 PM, Daniel Alley wrote: > w/ creating 400,000 units, the non-uuid PK is 30% faster at 42.22 seconds > vs. 55.98 seconds. > > w/ searching through the same

Re: [Pulp-dev] Integer IDs in Pulp 3

2018-07-11 Thread David Davis
I actually started working on converting IDs from UUIDs to integer IDs. It was pretty easy with one exception. Jobs in rq/redis are created using task id[0] and this job id needs to be a uuid. I see two possible solutions: 1. We leave task id as a UUID but every other id is an integer 2. We add a

Re: [Pulp-dev] Integer IDs in Pulp 3

2018-07-11 Thread Daniel Alley
w/ creating 400,000 units, the non-uuid PK is 30% faster at 42.22 seconds vs. 55.98 seconds. w/ searching through the same 400,000 units, performance is still about 30% faster. Doing a filter for file content units that have a relative_path__startswith={some random letter} (I put UUIDs in all

Re: [Pulp-dev] Integer IDs in Pulp 3

2018-07-11 Thread Daniel Alley
So, since I've already been working on some Pulp 3 benchmarking I decided to go ahead and benchmark this to get some actual data. Disclaimer: The following data is using bulk_create() with a modified, flat, non-inheriting content model, not the current multi-table inherited content model we're

Re: [Pulp-dev] Integer IDs in Pulp 3

2018-05-24 Thread David Davis
The link you sent about bulk_create seems to indicate only that the primary key attribute on the model doesn’t get set unless you are PostgreSQL. This ticket seems to confirm this: https://code.djangoproject.com/ticket/19527 That seems like a big caveat but not a showstopper that would prevent

Re: [Pulp-dev] Integer IDs in Pulp 3

2018-05-24 Thread Sean Myers
On 05/24/2018 11:22 AM, Austin Macdonald wrote: > I have a few concerns, but they all may be addressable. > > 1. URLs and security. If this integer is in the url, it is easy to guess > other urls. Hopefully, our security model won't depend on obscurity, so > maybe this isn't much of a concern.

Re: [Pulp-dev] Integer IDs in Pulp 3

2018-05-24 Thread Sean Myers
Responses inline. On 05/23/2018 02:26 PM, David Davis wrote: > Before the release of Pulp 3.0 GA, I think it’s worth just checking in to > make sure we want to use UUIDs over integer based IDs. Changing from UUIDs > to ints would be a very easy change at this point (1-2 lines of code) but >

Re: [Pulp-dev] Integer IDs in Pulp 3

2018-05-24 Thread Austin Macdonald
I have a few concerns, but they all may be addressable. 1. URLs and security. If this integer is in the url, it is easy to guess other urls. Hopefully, our security model won't depend on obscurity, so maybe this isn't much of a concern. 2. bulk_create. Apparently, bulk_creates would work, but

[Pulp-dev] Integer IDs in Pulp 3

2018-05-23 Thread David Davis
Before the release of Pulp 3.0 GA, I think it’s worth just checking in to make sure we want to use UUIDs over integer based IDs. Changing from UUIDs to ints would be a very easy change at this point (1-2 lines of code) but after GA ships, it would be hard if not impossible to switch. I think