On Friday, September 2, 2016 at 2:31:29 PM UTC-5, Jeremy Evans wrote:
>
> I'm guessing streaming would preform better, but you would still need to
> use a separate thread or sharding when using streaming.
>
That's great. I'll give streaming a try, but I believe the bottleneck isn't
going to be
On Friday, September 2, 2016 at 11:48:46 AM UTC-7, Trevor Turk wrote:
> On Friday, September 2, 2016 at 1:19:02 PM UTC-5, Jeremy Evans wrote:
>>
>> To work around this issue, you can use sharding (Dataset option like
>> :servers=>{:foo=>{}}), and do dataset.server(:foo).paged_each.
>> Alternati
On Friday, September 2, 2016 at 1:19:02 PM UTC-5, Jeremy Evans wrote:
>
> To work around this issue, you can use sharding (Dataset option like
> :servers=>{:foo=>{}}), and do dataset.server(:foo).paged_each.
> Alternatively, you can issue queries in a separate thread inside the
> Dataset#each b
On Friday, September 2, 2016 at 10:43:50 AM UTC-7, Trevor Turk wrote:
>
> Hello -- just reporting back on my progress so far.
>
> It appears we can use paged_each safely, however I was surprised to find
> that we can't run two simultaneously if they issue updates in the block.
>
In general it's
On Thursday, September 1, 2016 at 12:51:58 PM UTC-5, Jeremy Evans wrote:
>
> On Thursday, September 1, 2016 at 9:42:11 AM UTC-7, Trevor Turk wrote:
>>
>> On Thursday, September 1, 2016 at 11:34:44 AM UTC-5, Trevor Turk wrote:
>>>
>>> In my case, I'm on AWS RDS PostgreSQL version 9.3.10 and I'm usin
On Thursday, September 1, 2016 at 9:42:11 AM UTC-7, Trevor Turk wrote:
>
> On Thursday, September 1, 2016 at 11:34:44 AM UTC-5, Trevor Turk wrote:
>>
>> In my case, I'm on AWS RDS PostgreSQL version 9.3.10 and I'm using the
>> "sequel" and "pg" gems. So, I'd be using cursors with paged_each as it
On Thursday, September 1, 2016 at 11:34:44 AM UTC-5, Trevor Turk wrote:
>
> In my case, I'm on AWS RDS PostgreSQL version 9.3.10 and I'm using the
> "sequel" and "pg" gems. So, I'd be using cursors with paged_each as it
> stands. Do you know if that ought to be safe? I'm considering switching to
On Thursday, September 1, 2016 at 1:20:21 AM UTC-5, Jeremy Evans wrote:
>
> Dataset#paged_each shouldn't lock the entire table, as it just selects
> rows, but the behavior in regards to locking depends on the database. It
> would probably be best for you to try it in a test environment to be sur
On Wednesday, August 31, 2016 at 9:47:52 PM UTC-7, Trevor Turk wrote:
>
> Hello,
>
> I've been reading about paged_each and it seems to work well in my
> testing, but I'm concerned about using it in production because the
> documentation says it uses a transaction internally.
>
> I plan to make
Hello,
I've been reading about paged_each and it seems to work well in my testing,
but I'm concerned about using it in production because the documentation
says it uses a transaction internally.
I plan to make a "backfill" script that iterates over all rows in a
production database and enqueu
10 matches
Mail list logo
