Re: Sanity check on a use case

[Mark Payne]

Agreed. Is a bit unclear whether you're looking to partition by the date field 
or sort by the date field, or both. If you want to partition, then you'd use 
PartitionRecord. If you want to sort, we do not have a SortRecord processor. 
However, what we do have is QueryRecord, which can be used to easily sort the 
data using the query:

SELECT *
FROM FLOWFILE
ORDER BY date

Assuming that 'date' is the name of the field that you want to order by.

Hope this helps!
-Mark

> On Dec 3, 2019, at 9:12 AM, Joe Witt  wrote:
> 
> I read/replied too fast - if you mean that you want them together but sorted 
> by date then it makes sense we'd offer a SortRecord processor.  If you wanted 
> to simply group them by month then PartitionRecord should do the trick.
> 
> On Tue, Dec 3, 2019 at 8:10 AM Joe Witt  > wrote:
> Sounds like a perfect use of PartitionRecord.  And if you wanted larger 
> bundles of such things you could then follow it with MergeRecord correlated 
> on that same partitioned value.
> 
> Thanks
> 
> On Tue, Dec 3, 2019 at 8:09 AM Mike Thomsen  > wrote:
> We need to be able to split a record set by examining a date field and 
> sorting the messages by month into new record sets. The reason is that 
> they're going to be fed to an Elastic cluster that uses an index template to 
> build new indexes based on a date convention. We have a simple solution for 
> now that matches our volume, but I'd like to know if there is a better way to 
> do this out of the box than an ExecuteScript and if there might be others 
> who'd benefit from a broader solution.
> 
> Thanks,
> 
> Mike

Re: Sanity check on a use case

[Bryan Bende]

Sounds like PartitionRecord by month.

On Tue, Dec 3, 2019 at 9:12 AM Joe Witt  wrote:
>
> I read/replied too fast - if you mean that you want them together but sorted 
> by date then it makes sense we'd offer a SortRecord processor.  If you wanted 
> to simply group them by month then PartitionRecord should do the trick.
>
> On Tue, Dec 3, 2019 at 8:10 AM Joe Witt  wrote:
>>
>> Sounds like a perfect use of PartitionRecord.  And if you wanted larger 
>> bundles of such things you could then follow it with MergeRecord correlated 
>> on that same partitioned value.
>>
>> Thanks
>>
>> On Tue, Dec 3, 2019 at 8:09 AM Mike Thomsen  wrote:
>>>
>>> We need to be able to split a record set by examining a date field and 
>>> sorting the messages by month into new record sets. The reason is that 
>>> they're going to be fed to an Elastic cluster that uses an index template 
>>> to build new indexes based on a date convention. We have a simple solution 
>>> for now that matches our volume, but I'd like to know if there is a better 
>>> way to do this out of the box than an ExecuteScript and if there might be 
>>> others who'd benefit from a broader solution.
>>>
>>> Thanks,
>>>
>>> Mike

Re: Sanity check on a use case

[Joe Witt]

I read/replied too fast - if you mean that you want them together but
sorted by date then it makes sense we'd offer a SortRecord processor.  If
you wanted to simply group them by month then PartitionRecord should do the
trick.

On Tue, Dec 3, 2019 at 8:10 AM Joe Witt  wrote:

> Sounds like a perfect use of PartitionRecord.  And if you wanted larger
> bundles of such things you could then follow it with MergeRecord correlated
> on that same partitioned value.
>
> Thanks
>
> On Tue, Dec 3, 2019 at 8:09 AM Mike Thomsen 
> wrote:
>
>> We need to be able to split a record set by examining a date field and
>> sorting the messages by month into new record sets. The reason is that
>> they're going to be fed to an Elastic cluster that uses an index template
>> to build new indexes based on a date convention. We have a simple solution
>> for now that matches our volume, but I'd like to know if there is a better
>> way to do this out of the box than an ExecuteScript and if there might be
>> others who'd benefit from a broader solution.
>>
>> Thanks,
>>
>> Mike
>>
>

Re: Sanity check on a use case

[Joe Witt]

Sounds like a perfect use of PartitionRecord.  And if you wanted larger
bundles of such things you could then follow it with MergeRecord correlated
on that same partitioned value.

Thanks

On Tue, Dec 3, 2019 at 8:09 AM Mike Thomsen  wrote:

> We need to be able to split a record set by examining a date field and
> sorting the messages by month into new record sets. The reason is that
> they're going to be fed to an Elastic cluster that uses an index template
> to build new indexes based on a date convention. We have a simple solution
> for now that matches our volume, but I'd like to know if there is a better
> way to do this out of the box than an ExecuteScript and if there might be
> others who'd benefit from a broader solution.
>
> Thanks,
>
> Mike
>