[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552593#comment-15552593
]
Varun Saxena commented on YARN-5585:
bq. Given entities are sorted in ascending order, at some extent
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552548#comment-15552548
]
Rohith Sharma K S commented on YARN-5585:
-
bq. Then the entity id order is really the earliest
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552438#comment-15552438
]
Sangjin Lee commented on YARN-5585:
---
bq. Entity IDs' can be anything. Even a completely alphabetical
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552144#comment-15552144
]
Varun Saxena commented on YARN-5585:
bq. I was thinking to use same REST API for both by using
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550930#comment-15550930
]
Rohith Sharma K S commented on YARN-5585:
-
bq. We also need to be *crystal clear* that timeline
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550218#comment-15550218
]
Sangjin Lee commented on YARN-5585:
---
Thanks [~rohithsharma] for contributing the initial patch for this!
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547980#comment-15547980
]
Varun Saxena commented on YARN-5585:
By the way thinking more over it, we cannot predict how Separator
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546650#comment-15546650
]
Varun Saxena commented on YARN-5585:
bq. I purposefully used VARIABLE_SIZE because prefix can be empty
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546179#comment-15546179
]
Rohith Sharma K S commented on YARN-5585:
-
Thanks Varun for quick review..
bq. Intention behind
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545835#comment-15545835
]
Varun Saxena commented on YARN-5585:
Moreover, changes need to be made for
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545818#comment-15545818
]
Varun Saxena commented on YARN-5585:
Thanks [~rohithsharma] for the patch. Few comments.
# Intention
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545552#comment-15545552
]
Hadoop QA commented on YARN-5585:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533830#comment-15533830
]
Vrushali C commented on YARN-5585:
--
Thanks [~rohithsharma] for the summary.
bq. 2. By default, use
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533740#comment-15533740
]
Rohith Sharma K S commented on YARN-5585:
-
In weekly sync up meeting, had discussed more on
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533365#comment-15533365
]
Vrushali C commented on YARN-5585:
--
I believe one way or the other client does need to send in an ordering
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15532764#comment-15532764
]
Sangjin Lee commented on YARN-5585:
---
Thanks [~rohithsharma] for your comments and input!
I'd like to
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15532592#comment-15532592
]
Varun Saxena commented on YARN-5585:
bq. In a distributed cluster, we can expect source of origin of
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15532511#comment-15532511
]
Rohith Sharma K S commented on YARN-5585:
-
I do still think of concern for introducing new field
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524011#comment-15524011
]
Varun Saxena commented on YARN-5585:
Just to add,
bq. And after providing entityPrefixId , does reader
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523904#comment-15523904
]
Li Lu commented on YARN-5585:
-
Hi [~rohithsharma], some of my understandings
bq. IIUC from user perspective,
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523429#comment-15523429
]
Rohith Sharma K S commented on YARN-5585:
-
Thanks Varun, I got overall picture of solution.
#
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523186#comment-15523186
]
Hadoop QA commented on YARN-5585:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523179#comment-15523179
]
Varun Saxena commented on YARN-5585:
Alternatively, you can change your entity ID in a manner in which
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523177#comment-15523177
]
Varun Saxena commented on YARN-5585:
bq. When scan is performed on rows, the ResultScanner gives in the
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523047#comment-15523047
]
Rohith Sharma K S commented on YARN-5585:
-
Apologized, I was in leave!! Lot more discussions are
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15521715#comment-15521715
]
Joep Rottinghuis commented on YARN-5585:
Good point, something we need to check. We will want to
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15521189#comment-15521189
]
Sangjin Lee commented on YARN-5585:
---
I wasn't sure whether it was possible to update an existing row
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15520959#comment-15520959
]
Joep Rottinghuis commented on YARN-5585:
Will have to look into the solution a bit deeper, but we
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15520952#comment-15520952
]
Joep Rottinghuis commented on YARN-5585:
[~sjlee0] wouldn't the column value filter on entity-id
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517864#comment-15517864
]
Sangjin Lee commented on YARN-5585:
---
To be clear, in case of things like tez vertex id, it should be easy
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517489#comment-15517489
]
Sangjin Lee commented on YARN-5585:
---
I do have one question for us to think about. With the entity id
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517482#comment-15517482
]
Sangjin Lee commented on YARN-5585:
---
Thanks [~varun_saxena] for summarizing the discussion clearly. The
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15514243#comment-15514243
]
Varun Saxena commented on YARN-5585:
Summarizing the solution we decided upon in the call.
* We will
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513781#comment-15513781
]
Sangjin Lee commented on YARN-5585:
---
Thanks for your comments [~varun_saxena]. Yes, we should discuss
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512673#comment-15512673
]
Varun Saxena commented on YARN-5585:
Just to clarify, what I meant by having another index table was
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512200#comment-15512200
]
Sangjin Lee commented on YARN-5585:
---
I am also catching up on this discussion (sorry it got delayed).
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512079#comment-15512079
]
Vrushali C commented on YARN-5585:
--
I have been thinking more on this. I think if there is a concern about
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15511129#comment-15511129
]
Vrushali C commented on YARN-5585:
--
Catching up on this thread. I have tried to read through all the
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510573#comment-15510573
]
Varun Saxena commented on YARN-5585:
I think HBase table schema change as per suggestion given by me
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15494251#comment-15494251
]
Varun Saxena commented on YARN-5585:
Just to summarise the suggestions given for folks to refer to.
*
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472946#comment-15472946
]
Varun Saxena commented on YARN-5585:
But padding isnt required if we do not store DAG ID or vertex ID
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472939#comment-15472939
]
Rohith Sharma K S commented on YARN-5585:
-
Bumping up the priority of the task since it is major
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472922#comment-15472922
]
Rohith Sharma K S commented on YARN-5585:
-
It is right, there is no proper padding for DAG
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472837#comment-15472837
]
Varun Saxena commented on YARN-5585:
IIUC, Tez DAG ID is a combination of YARN App ID and DAG sequence
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472689#comment-15472689
]
Varun Saxena commented on YARN-5585:
Also [~rohithsharma] if its feasible, kindly consolidate all the
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472676#comment-15472676
]
Varun Saxena commented on YARN-5585:
Another option would be to change the row key of entity table to
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472669#comment-15472669
]
Varun Saxena commented on YARN-5585:
Another solution which comes to mind is that we keep another
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472636#comment-15472636
]
Li Lu commented on YARN-5585:
-
bq. DAG ID seems to be generated same way.
Unfortunately this is not true for
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472526#comment-15472526
]
Varun Saxena commented on YARN-5585:
bq. Just realized that a normal converter will not address the use
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472111#comment-15472111
]
Li Lu commented on YARN-5585:
-
Just realized that a normal converter will not address the use case where users
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472083#comment-15472083
]
Li Lu commented on YARN-5585:
-
I think we're overcomplicating the problem here... I believe the general use
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470803#comment-15470803
]
Varun Saxena commented on YARN-5585:
The best solution for this will be that keys are stored in sorted
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470739#comment-15470739
]
Varun Saxena commented on YARN-5585:
bq. But problem is entities may not stored in their creation time
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470687#comment-15470687
]
Rohith Sharma K S commented on YARN-5585:
-
bq. fetch the specific entity mentioned in fromId using
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470657#comment-15470657
]
Varun Saxena commented on YARN-5585:
Hmm...Then what we can do is that fetch the specific entity
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470538#comment-15470538
]
Rohith Sharma K S commented on YARN-5585:
-
bq. Can you tell me the use case ? Listing all DAGs' or
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470469#comment-15470469
]
Varun Saxena commented on YARN-5585:
Thanks Rohith for the patch.
A single column value filter of
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470338#comment-15470338
]
Rohith Sharma K S commented on YARN-5585:
-
One of the challenge for supporting fromId is to iterate
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469802#comment-15469802
]
Rohith Sharma K S commented on YARN-5585:
-
bq. Are we selecting entities whose ID is less than
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469513#comment-15469513
]
Li Lu commented on YARN-5585:
-
Thanks [~rohithsharma]! The approach is generally fine, however, I have some
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468719#comment-15468719
]
Hadoop QA commented on YARN-5585:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467275#comment-15467275
]
Rohith Sharma K S commented on YARN-5585:
-
YARN-5094 has to do some progress to check this patch
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467262#comment-15467262
]
Rohith Sharma K S commented on YARN-5585:
-
YARN web UI use case : Number of containers running in a
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457081#comment-15457081
]
Li Lu commented on YARN-5585:
-
Thanks [~varun_saxena]! I think the discussion so far indicates that
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455180#comment-15455180
]
Varun Saxena commented on YARN-5585:
bq. But I am not sure that does HBase filters can support to scan
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454227#comment-15454227
]
Rohith Sharma K S commented on YARN-5585:
-
bq. Can we translate the fromId request into some HBase
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453314#comment-15453314
]
Li Lu commented on YARN-5585:
-
If there are two flow runs running, I believe the problem is how to define the
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453145#comment-15453145
]
Varun Saxena commented on YARN-5585:
Infact in case of flows within an app, there can be a problem with
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453147#comment-15453147
]
Varun Saxena commented on YARN-5585:
Infact in case of flows within an app, there can be a problem with
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453109#comment-15453109
]
Varun Saxena commented on YARN-5585:
bq. Can we translate the fromId request into some HBase filters
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453070#comment-15453070
]
Li Lu commented on YARN-5585:
-
Can we translate the fromId request into some HBase filters so that we can
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15452583#comment-15452583
]
Varun Saxena commented on YARN-5585:
If the use case is only for apps then the row keys in application
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451835#comment-15451835
]
Rohith Sharma K S commented on YARN-5585:
-
bq. entities may not stored in their creation time order
[
https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450450#comment-15450450
]
Li Lu commented on YARN-5585:
-
OK let me make sure if I fully understand it here: the ultimate use case is
74 matches
Mail list logo