Soup gave a talk about something down this alley: https://www.youtube.com/watch?v=aedejUXWrV0
On Thu, Oct 1, 2015 at 2:58 PM, Keith Turner <[email protected]> wrote: > Could possibly use a ThreadLocal containing a SoftReference > > Another place you could possibly put this code instead of in a constraint > is in a minor compaction iterator. Although you will still have the clean > up problem since iterators do not have a close method. There is an open > ticket where Billie suggested that Accumulo can call close on any iterator > that implements Closeable > > On Thu, Oct 1, 2015 at 2:48 PM, Parise, Jonathan < > [email protected]> wrote: > >> I think this is one of those things where there really aren’t great >> solutions. >> >> >> >> A static connection could work, but if multiple Constraint instances can >> exist at the same time, it would probably not work. Since all of them would >> be trying to use the same connection at the same time. >> >> >> >> ThreadLocal could possibly work better. The only question is how long >> lived is the thread that calls the constraints? For example that thread >> could be torn down as soon as the constraint is done. In that case the >> performance would be no better than creating and tearing down everything >> each time check() is called. >> >> >> >> This is why I am trying to understand the Constraint’s lifecycle, so I >> can come up with the least bad way of solving this problem. >> >> >> >> Thanks for the ideas! I am just not sure I know enough about the >> lifecycle of Constraints to understand if these suggestions would be >> helpful. >> >> >> >> Jon Parise >> >> >> >> *From:* John Vines [mailto:[email protected]] >> *Sent:* Thursday, October 01, 2015 2:40 PM >> >> *To:* [email protected] >> *Subject:* Re: Watching for Changes with Write Ahead Log? >> >> >> >> As dirty as it is, that sounds like a case for a static, or maybe thread >> local, object >> >> >> >> On Thu, Oct 1, 2015, 7:19 PM Parise, Jonathan <[email protected]> >> wrote: >> >> I have a few follow up questions in regard to constraints. >> >> >> >> What is the lifecycle of a constraint? What I mean by this is are the >> constraints somehow tied to Accumulo’s lifecycle or are they just >> instantiated each time a mutation occurs and then disposed? >> >> >> >> Also, are there multiple instances of the same constraint class at any >> time or do all mutation on a table go through the exact same constraint? >> >> >> >> My guess is that when a mutation comes in a new constraint is made >> through reflection. Then check() is called, the violation codes are parsed >> and the object is disposed/finalized. >> >> >> >> The reason I ask is that what I want to do is update my ElasticSearch >> index each time I see a mutation on the table. However, I don’t want to >> have to make a connection, send the data and then tear down the connection >> each time. That’s a lot of unnecessary overhead and with all that overhead >> happening on every mutation performance could be badly impacted. >> >> >> >> Is there some way to cache something like a connection and reuse it >> between calls to the Constraint’s check() method? How would such a thing be >> cleaned up if Accumulo is shut down? >> >> >> >> >> >> Thanks again, >> >> >> >> Jon >> >> *From:* Parise, Jonathan [mailto:[email protected] >> <[email protected]>] >> *Sent:* Wednesday, September 30, 2015 9:21 AM >> *To:* [email protected] >> *Subject:* RE: Watching for Changes with Write Ahead Log? >> >> >> >> In this particular case, I need to update some of my application state >> when changes made by another system occur. >> >> >> >> I would need to do a few things to accomplish my goal. >> >> >> >> 1) Be notified or see that a table had changed >> >> 2) Checked that against changes I know my system has made >> >> 3) If my system is not the originator of the change, update >> internal state to reflect the change. >> >> >> >> Examples of state I may need to update include an ElasticSearch index and >> also an in memory cache. >> >> >> >> I’m going to read up on constraints again and see if I can use them for >> this purpose. >> >> >> >> Thanks! >> >> >> >> Jon >> >> >> >> >> >> >> >> *From:* Adam Fuchs [mailto:[email protected] <[email protected]>] >> *Sent:* Tuesday, September 29, 2015 5:46 PM >> *To:* [email protected] >> *Subject:* Re: Watching for Changes with Write Ahead Log? >> >> >> >> Jon, >> >> >> >> You might think about putting a constraint on your table. I think the API >> for constraints is flexible enough for your purpose, but I'm not exactly >> sure how you would want to manage the results / side effects of your >> observations. >> >> >> >> Adam >> >> >> >> >> >> On Tue, Sep 29, 2015 at 5:41 PM, Parise, Jonathan < >> [email protected]> wrote: >> >> Hi, >> >> >> >> I’m working on a system where generally changes to Accumulo will come >> through that system. However, in some cases, another system may change data >> without my system being aware of it. >> >> >> >> What I would like to do is somehow listen for changes to the tables my >> system cares about. I know there is a write ahead log that I could >> potentially listen to for changes, but I don’t know how to use it. I looked >> around for some documentation about it, and I don’t see much. I get the >> impression that it isn’t really intended for this type of use case. >> >> >> >> Does anyone have any suggestions on how to watch a table for changes and >> then determine if those changes were made by a different system. >> >> >> >> Is there some documentation about how to use the write ahead log? >> >> >> >> >> >> Thanks, >> >> >> >> Jon Parise >> >> >> >> >
