Thanks James - The provenance dates I am seeing in the UI are very explicitly NOT in UTC. While I am sure they are stored that way in the repository, the 1.5 version of the UI shows them in my local timezone, currently CDT. I have confirmed that the original provenance events I had captured before things stopped showing up are shown in the correct local timezone - The provenance events I am looking for were emitted from these processors earlier in the day, and these newer attempts are doing the same exact thing which should result in the same kinds of events. They simply are not showing up.
I have cleared the search and looked at the events from the top menu. As I reported earlier, the events after a certain time seem to have stopped showing up altogether, even though data was flowing through the system after that point -Tim > On May 22, 2018, at 12:01 AM, James Wing <[email protected]> wrote: > > Tim, > > It is most likely that provenance data is being collected, and the search UI > has a few quirks that might explain your experience: > > * Provenance dates are in UTC. If you are not careful with your filter, you > can end up fighting over the time zone. > * Processors emit provenance events as flowfiles are modified, and not > necessarily when data is not modified. If you right-click on a processor > that is not emitting events, it may in fact not find provenance data when > filtering by that processor's ID, even though flowfiles obviously traveled > through the processor. > > I recommend trying "Clear search" or searching provenance from the top down > via "Data Provenance" in the top menu. This can help clarify if there is any > data for the time period, and make it easier to experiment with the search > results. The provenance search will show the oldest event date in the top > right, and with no search filter you should see the most recent events. > > Does that help? > > Thanks, > > James > > > On Mon, May 21, 2018 at 5:32 PM, Tim Dean <[email protected] > <mailto:[email protected]>> wrote: > Hello, > > I am having a hard time troubleshooting a NiFi flow to see where things are > failing. I am trying to look at the provenance repository for a variety of > processors, but for some reason nothing more recent seems to be appearing > there. For example: > At approximately 10:30 this morning I started a flow and observed it for a > couple of hours before disabling it to look into a few unexpected results. > By right-clicking individual processors and selecting “View data provenance” > I can see the NiFi Data Provenance view > For each processor I investigate I can see anywhere from 10 to 100 provenance > events that came in during the hours I was running my flow > A few hours later I restart the flow. Data once again flows through and after > a while I stop my flow again > Now I again right-click on the processors and select “View data provenance”. > No new provenance events seem to show up in the NiFi Data Provenance view > > I have checked m search filter to make sure I am not accidentally filtering > out events. I have looked at the external systems that this flow touches and > confirmed that data is/was flowing through these processors. But for some > reason I can see no provenance records in the UI. > > I am using NiFi version 1.5 > > I have not (yet) changed any of the default settings for NiFi and how its > provenance repository is configured > > Any advice on where my provenance events are going or what I might be doing > that causes the provenance system to go silent on me? > > Thanks > > -Tim >
