> On 5 Jan 2016, at 20:34, Manoj Samel <manojsamelt...@gmail.com> wrote:
>
> For delete case,
>
> Digging further into slider HDFS data in history directory via hdfs dfs
> -cat
> /user/xxx/.slider/cluster/ms999/history/rolehistory-0000015213700161.json;
> it seems history keeps all roles that are active or inactive
>
> {"entry":{"org.apache.slider.server.avro.RoleHistoryHeader":{"version":1,"saved":1452025053537,"savedx":"15213700161","savedate":"5
> Jan 2016 20:17:33 GMT","roles":3}}}
> {"entry":{"org.apache.slider.server.avro.RoleHistoryMapping":{"rolemap":{"slider-appmaster":0,"user1":1,"user2":2}}}}
> {"entry":{"org.apache.slider.server.avro.NodeEntryRecord":{"host":"host1","role":1,"active":false,"last_used":1452025053537}}}
> {"entry":{"org.apache.slider.server.avro.NodeEntryRecord":{"host":"host2","role":2,"active":true,"last_used":1452024753905}}}
> {"entry":{"org.apache.slider.server.avro.RoleHistoryFooter":{"count":2}}}
>
> Noted role active is false, as done by setting yarn.component.instances for
> user1 to 0 ( I am sure flex 0 would do same ) to achive stopping component.
>
> I understand this history will be required if someone did a flex 0 and then
> need to do flex N (N > 0) later.
Its used more in restart than flex in a running app ... in the running
application the RoleHistory class remembers this stuff ... it's also the class
which saves the histories you are seeing
> However, in the use cases where this
> component will never be used again; there should be a option to "delete"
> the component from all configs permanently ...
File a JIRA on this, you've got a use case that's not been covered before —you
want to deal with the risk of massive growth in (non-needed) profiles, from
saved history and running state.
I could imagine the flex command having a bool to say "purge all zero records"
which would then go through the application state structures and do this
