Abdel and Jacques -
It is absolutely a bad profile, however, Jacques, it was not a empty file
profile, instead, it was a profile where the write didn't complete (likely
due to my bad cluster state post).
I was able to reproduce it synthetically like this, say you had a profile
that ended similar to this:
..."endpoint":{"address":"zeta01.mydomain.net","userPort":
20001,"controlPort":20002,"dataPort":20003},"lastUpdate":
1464730828126,"lastProgress":1464730828126}]}],"user":"adm"}
(The ... represents the part I removed for brevity in this email, the rest
of the profile is below)
We can reproduce the UI profiles issue by removing ,"user":"adm"}
so it ends like this:
..."endpoint":{"address":"zeta01.mydomain.net","userPort":
20001,"controlPort":20002,"dataPort":20003},"lastUpdate":
1464730828126,"lastProgress":1464730828126}]}]
Let me know if you know of a JIRA, otherwise I'll open one, this is a pain
for users, because you have to go through an validate the json on every
profile to find the one that is breaking your UI.
John
Full "Working" Profile:
{"id":{"part1":2932401029014293687,"part2":6412631480783062510},"type":1,
"start":1464730828036,"end":1464730828131,"query":"alter session set
`store.parquet.enable_dictionary_encoding` = true","foreman":{"address":"
zeta01.mydomain.net","userPort":20001,"controlPort":20002,"
dataPort":20003},"state":2,"totalFragments":1,"finishedFragments":0,"
fragmentProfile":[{"majorFragmentId":0,"minorFragmentProfile":[{"
state":3,"minorFragmentId":0,"operatorProfile":[{"
inputProfile":[{"records":1,"batches":1,"schemas":1}],"
operatorId":0,"operatorType":26,"setupNanos":0,"processNanos":1020862,"
peakLocalMemoryAllocated":4530432,"waitNanos":0},{"
inputProfile":[{"records":1,"batches":1,"schemas":1}],"
operatorId":0,"operatorType":13,"setupNanos":0,"processNanos":154675,"
peakLocalMemoryAllocated":4530176,"metric":[{"metricId":
0,"longValue":59}],"waitNanos":1936593}],"startTime":
1464730828114,"endTime":1464730828123,"memoryUsed":0,"
maxMemoryUsed":9060608,"endpoint":{"address":"zeta01.mydomain.net
","userPort":20001,"controlPort":20002,"dataPort":20003},"lastUpdate":
1464730828126,"lastProgress":1464730828126}]}],"user":"adm"}
Full "Broken" Profile:
{"id":{"part1":2932401029014293687,"part2":6412631480783062510},"type":1,
"start":1464730828036,"end":1464730828131,"query":"alter session set
`store.parquet.enable_dictionary_encoding` = true","foreman":{"address":"
zeta01.mydomain.net","userPort":20001,"controlPort":20002,"
dataPort":20003},"state":2,"totalFragments":1,"finishedFragments":0,"
fragmentProfile":[{"majorFragmentId":0,"minorFragmentProfile":[{"
state":3,"minorFragmentId":0,"operatorProfile":[{"
inputProfile":[{"records":1,"batches":1,"schemas":1}],"
operatorId":0,"operatorType":26,"setupNanos":0,"processNanos":1020862,"
peakLocalMemoryAllocated":4530432,"waitNanos":0},{"
inputProfile":[{"records":1,"batches":1,"schemas":1}],"
operatorId":0,"operatorType":13,"setupNanos":0,"processNanos":154675,"
peakLocalMemoryAllocated":4530176,"metric":[{"metricId":
0,"longValue":59}],"waitNanos":1936593}],"startTime":
1464730828114,"endTime":1464730828123,"memoryUsed":0,"
maxMemoryUsed":9060608,"endpoint":{"address":"zeta01.mydomain.net
","userPort":20001,"controlPort":20002,"dataPort":20003},"lastUpdate":
1464730828126,"lastProgress":1464730828126}]}]
On Wed, Jun 1, 2016 at 6:22 AM, John Omernik <j...@omernik.com> wrote:
> I will walk through and look for the corrupt profile. This could be the
> result of my "cluster in a bad state problem" (actually, it IS the result
> of that, I have Ben doing testing to recreate the issue). Is there a JIRA
> for this ? I agree the UI should handle this.
>
>
> On Tuesday, May 31, 2016, Jacques Nadeau <jacq...@dremio.com> wrote:
>
>> Odds are one of your profiles is corrupt. Last I checked, there was a bug
>> that a corrupt profile would cause no profiles to show up in list. Check
>> if
>> any of your profiles are zero bytes and delete them.
>>
>> The UI should really handle this.
>>
>> --
>> Jacques Nadeau
>> CTO and Co-Founder, Dremio
>>
>> On Tue, May 31, 2016 at 12:49 PM, John Omernik <j...@omernik.com> wrote:
>>
>> > I am scratching my head at this one... I made some minor changes to my
>> > drill-env.sh to enable gclogging, and was using the profiles in the
>> webui
>> > just fine. Due to some previously mentioned issues, I've had to restart
>> > drill bits due to GC issues etc.
>> >
>> > Now, while my profiles directory still exists, and my
>> drill-override.conf
>> > has not been changed, no profiles now show up in the webui, even after
>> > drillbit restarts, and running more queries... The profiles are still
>> being
>> > created (I can see them being added to the same profiles directory) just
>> > nothing shows up in the Web UI...
>> >
>> > What could be happening here?
>> >
>> > *scratching my head
>> >
>>
>
>
> --
> Sent from my iThing
>
