While the Cypress team suggests taking advantage of stubs where you can,
especially during development, we would definitely be able to test real
endpoints [1]. It can be used exactly like how Protractor is used, but with
the many benefits and features it provides [2]. Cypress also offers tools
for
I think I'm torn on this, specifically because it's batch and would
generally be run as-needed. Justin, can you elaborate on your concerns
there? This feels functionally very similar to our flat file loaders, which
all have inputs for config from the CLI only. On the other hand, our flat
file
The profile not being able to read from ZK feels like a fairly substantial,
if subtle, set of potential problems. I'd like to see that in either
before merging or at least pretty soon after merging. Is it a lot of work
to add that functionality based on where things are right now?
On Thu, Sep
I think more often than not, you would want to load your profile definition
from a file. This is why I considered the 'load from Zk' more of a
nice-to-have.
- In use case 1 and 2, this would definitely be the case. The profiles
I am working with are speculative and I am using the batch
> How do we establish "tm" from 1.1 above? Any concerns about overlap or
gaps after the seeding is performed?
Good point. Right now, if the Streaming and Batch Profiler overlap the
last write wins. And presumably the output of the Streaming and Batch
Profiler are the same, so no worries, right?
> It's just cleaner from a usage/management perspective to say "I want to put
a profile in prod, just use streaming profiler and the batch profiler with
the same setup and they're good to go."
Agreed. I can add it. It would be a simple addition.
On Thu, Sep 20, 2018 at 12:49 PM Justin Leet
Assuming you have 9 months of data archived, yes.
On Thu, Sep 20, 2018 at 1:22 PM Michael Miklavcic <
michael.miklav...@gmail.com> wrote:
> So in the case of 3 - if you had 6 months of data that hadn't been profiled
> and another 3 that had been profiled (9 months total data), in its current
>
Ok, makes sense. That's sort of what I was thinking as well, Nick. Pulling
at this thread just a bit more...
1. I have an existing system that's been up a while, and I have added k
profiles - assume these are the first profiles I've created.
1. I would have t0 - tm (where m is the
I think the main difference between this and the flatfile loader is that we
actively maintain our profiles in ZK for streaming. Doing this from files
is likely going to be the main usage, particularly for speculative usage.
For me, the main use case for ZK is definitely use case 3.
I can
So in the case of 3 - if you had 6 months of data that hadn't been profiled
and another 3 that had been profiled (9 months total data), in its current
form the batch job runs over all 9 months?
On Thu, Sep 20, 2018 at 11:13 AM Nick Allen wrote:
> > How do we establish "tm" from 1.1 above? Any
That's good feedback, thanks Shane!
On Thu, Sep 20, 2018 at 6:23 AM Shane Ardell
wrote:
> While the Cypress team suggests taking advantage of stubs where you can,
> especially during development, we would definitely be able to test real
> endpoints [1]. It can be used exactly like how
Here is another limitation that I just thought. It can only read a profile
definition from a file. It probably also makes sense to add an option that
allows it to read the current Profiler configuration from Zookeeper.
> Is it worth setting up a default config that pulls from the main indexing
I think we might want to allow the flexibility to choose the date range
then. I don't yet feel like I have a good enough understanding of all the
ways in which users would want to seed to force them to run the batch job
over all the data. It might also make it easier to deal with remediation,
ie
Yeah, agreed. Per use case 3, when deploying to production there really
wouldn't be a huge overlap like 3 months of already profiled data. Its day
1, the profile was just deployed around the same time as you are running
the Batch Profiler, so the overlap is in minutes, maybe hours. But I can
14 matches
Mail list logo
