[jira] [Comment Edited] (KUDU-2646) kudu restart the tablets stats from INITIALIZED change to RUNNING cost a few days
[ https://issues.apache.org/jira/browse/KUDU-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724769#comment-16724769 ] jiaqiyang edited comment on KUDU-2646 at 12/19/18 7:44 AM: --- i have the same question i have give out tserver log KUDU-2638 kudu version 1.6.0 was (Author: jiaqiyang): i have the same question i have give out tserver log KUDU-2638 > kudu restart the tablets stats from INITIALIZED change to RUNNING cost a few > days > - > > Key: KUDU-2646 > URL: https://issues.apache.org/jira/browse/KUDU-2646 > Project: Kudu > Issue Type: Bug >Reporter: qinzl_1 >Priority: Major > Fix For: n/a > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KUDU-2646) kudu restart the tablets stats from INITIALIZED change to RUNNING cost a few days
[ https://issues.apache.org/jira/browse/KUDU-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724769#comment-16724769 ] jiaqiyang commented on KUDU-2646: - i have the same question i have give out tserver log KUDU-2638 > kudu restart the tablets stats from INITIALIZED change to RUNNING cost a few > days > - > > Key: KUDU-2646 > URL: https://issues.apache.org/jira/browse/KUDU-2646 > Project: Kudu > Issue Type: Bug >Reporter: qinzl_1 >Priority: Major > Fix For: n/a > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KUDU-2638) kudu cluster restart very long time to reused
[ https://issues.apache.org/jira/browse/KUDU-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724766#comment-16724766 ] jiaqiyang commented on KUDU-2638: - now i'm analysising what block the tablet lifecycle; i think the question in MaintenanceManager modle > kudu cluster restart very long time to reused > - > > Key: KUDU-2638 > URL: https://issues.apache.org/jira/browse/KUDU-2638 > Project: Kudu > Issue Type: Improvement >Reporter: jiaqiyang >Priority: Major > Fix For: n/a > > Attachments: tserverLog.tar.gz > > > when restart my kudu cluster ;all tablet not avalible: > run kudu cluster ksck show that: > Table Summary > > > Name | Status | Total Tablets | Healthy | Under-replicated | Unavailable > + > t1 | HEALTHY | 1 | 1 | 0 | 0 > t2 | UNAVAILABLE | 5 | 0 | 1 | 4 > t3 | UNAVAILABLE | 6 | 2 | 0 | 4 > t3 | UNAVAILABLE | 3 | 0 | 0 | 3 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KUDU-2638) kudu cluster restart very long time to reused
[ https://issues.apache.org/jira/browse/KUDU-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724757#comment-16724757 ] jiaqiyang edited comment on KUDU-2638 at 12/19/18 7:40 AM: --- this is the kudu cluster one tserver log [^tserverLog.tar.gz] the cluster total 19 tservers and 3 masters; 12 sata disk every server 200+ tablet on one tserver; 3 replica every tablet; there is one question : why the cluster restart use long time to table avalible , in the log i see that boostrap very quickly ;but there is very long time spend on major compact; how can we stop the compact then admin compaction after idle time like HBase compaction! was (Author: jiaqiyang): this is the kudu cluster one tserver log [^tserverLog.tar.gz] the cluster total 19 tservers and 3 masters; 12 sata disk every server 200+ tablet on one tserver; there is one question : why the cluster restart use long time to table avalible , in the log i see that boostrap very quickly ;but there is very long time spend on major compact; how can we stop the compact then admin compaction after idle time like HBase compaction! > kudu cluster restart very long time to reused > - > > Key: KUDU-2638 > URL: https://issues.apache.org/jira/browse/KUDU-2638 > Project: Kudu > Issue Type: Improvement >Reporter: jiaqiyang >Priority: Major > Fix For: n/a > > Attachments: tserverLog.tar.gz > > > when restart my kudu cluster ;all tablet not avalible: > run kudu cluster ksck show that: > Table Summary > > > Name | Status | Total Tablets | Healthy | Under-replicated | Unavailable > + > t1 | HEALTHY | 1 | 1 | 0 | 0 > t2 | UNAVAILABLE | 5 | 0 | 1 | 4 > t3 | UNAVAILABLE | 6 | 2 | 0 | 4 > t3 | UNAVAILABLE | 3 | 0 | 0 | 3 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KUDU-2638) kudu cluster restart very long time to reused
[ https://issues.apache.org/jira/browse/KUDU-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724757#comment-16724757 ] jiaqiyang commented on KUDU-2638: - this is the kudu cluster one tserver log [^tserverLog.tar.gz] the cluster total 19 tservers and 3 masters; 12 sata disk every server 200+ tablet on one tserver; there is one question : why the cluster restart use long time to table avalible , in the log i see that boostrap very quickly ;but there is very long time spend on major compact; how can we stop the compact then admin compaction after idle time like HBase compaction! > kudu cluster restart very long time to reused > - > > Key: KUDU-2638 > URL: https://issues.apache.org/jira/browse/KUDU-2638 > Project: Kudu > Issue Type: Improvement >Reporter: jiaqiyang >Priority: Major > Fix For: n/a > > Attachments: tserverLog.tar.gz > > > when restart my kudu cluster ;all tablet not avalible: > run kudu cluster ksck show that: > Table Summary > > > Name | Status | Total Tablets | Healthy | Under-replicated | Unavailable > + > t1 | HEALTHY | 1 | 1 | 0 | 0 > t2 | UNAVAILABLE | 5 | 0 | 1 | 4 > t3 | UNAVAILABLE | 6 | 2 | 0 | 4 > t3 | UNAVAILABLE | 3 | 0 | 0 | 3 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KUDU-2638) kudu cluster restart very long time to reused
[ https://issues.apache.org/jira/browse/KUDU-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaqiyang updated KUDU-2638: Attachment: tserverLog.tar.gz > kudu cluster restart very long time to reused > - > > Key: KUDU-2638 > URL: https://issues.apache.org/jira/browse/KUDU-2638 > Project: Kudu > Issue Type: Improvement >Reporter: jiaqiyang >Priority: Major > Fix For: n/a > > Attachments: tserverLog.tar.gz > > > when restart my kudu cluster ;all tablet not avalible: > run kudu cluster ksck show that: > Table Summary > > > Name | Status | Total Tablets | Healthy | Under-replicated | Unavailable > + > t1 | HEALTHY | 1 | 1 | 0 | 0 > t2 | UNAVAILABLE | 5 | 0 | 1 | 4 > t3 | UNAVAILABLE | 6 | 2 | 0 | 4 > t3 | UNAVAILABLE | 3 | 0 | 0 | 3 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KUDU-2646) kudu restart the tablets stats from INITIALIZED change to RUNNING cost a few days
[ https://issues.apache.org/jira/browse/KUDU-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724669#comment-16724669 ] Grant Henke commented on KUDU-2646: --- Without any details it is hard to asses this problem. Generally the best way to get help with problems is to first report the issue you are seeing with as much context and information as possible on the users mailing list. Any bugs or issues identified there could result in a jira. I will close this Jira for now and we can re-open it if needed. > kudu restart the tablets stats from INITIALIZED change to RUNNING cost a few > days > - > > Key: KUDU-2646 > URL: https://issues.apache.org/jira/browse/KUDU-2646 > Project: Kudu > Issue Type: Bug >Reporter: qinzl_1 >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KUDU-2646) kudu restart the tablets stats from INITIALIZED change to RUNNING cost a few days
[ https://issues.apache.org/jira/browse/KUDU-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke resolved KUDU-2646. --- Resolution: Invalid Fix Version/s: n/a > kudu restart the tablets stats from INITIALIZED change to RUNNING cost a few > days > - > > Key: KUDU-2646 > URL: https://issues.apache.org/jira/browse/KUDU-2646 > Project: Kudu > Issue Type: Bug >Reporter: qinzl_1 >Priority: Major > Fix For: n/a > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KUDU-2646) kudu restart the tablets stats from INITIALIZED change to RUNNING cost a few days
qinzl_1 created KUDU-2646: - Summary: kudu restart the tablets stats from INITIALIZED change to RUNNING cost a few days Key: KUDU-2646 URL: https://issues.apache.org/jira/browse/KUDU-2646 Project: Kudu Issue Type: Bug Reporter: qinzl_1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KUDU-2637) Add a note about leadership imbalance in the faq
[ https://issues.apache.org/jira/browse/KUDU-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Serbin reassigned KUDU-2637: --- Assignee: Alexey Serbin > Add a note about leadership imbalance in the faq > > > Key: KUDU-2637 > URL: https://issues.apache.org/jira/browse/KUDU-2637 > Project: Kudu > Issue Type: Improvement > Components: documentation >Reporter: Grant Henke >Assignee: Alexey Serbin >Priority: Minor > > There have been a few questions on leadership imbalance and whether or not it > is important to monitor and fix. We should update the FAQ section to address > this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KUDU-2645) Diff scanner should perform a merge on the rowset iterators at scan time
Mike Percy created KUDU-2645: Summary: Diff scanner should perform a merge on the rowset iterators at scan time Key: KUDU-2645 URL: https://issues.apache.org/jira/browse/KUDU-2645 Project: Kudu Issue Type: New Feature Components: tablet Reporter: Mike Percy In order to perform a diff scan we will need the MergeIterator to ensure that duplicate ghost rows are not returned in cases where a row was deleted and flushed, then reinserted into a new rowset during the time period covered by the diff scan. In such a case, only one representation of the row should be returned, which is the reinserted one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KUDU-2645) Diff scanner should perform a merge on the rowset iterators at scan time
[ https://issues.apache.org/jira/browse/KUDU-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Percy reassigned KUDU-2645: Assignee: Mike Percy > Diff scanner should perform a merge on the rowset iterators at scan time > > > Key: KUDU-2645 > URL: https://issues.apache.org/jira/browse/KUDU-2645 > Project: Kudu > Issue Type: New Feature > Components: tablet >Reporter: Mike Percy >Assignee: Mike Percy >Priority: Major > > In order to perform a diff scan we will need the MergeIterator to ensure that > duplicate ghost rows are not returned in cases where a row was deleted and > flushed, then reinserted into a new rowset during the time period covered by > the diff scan. In such a case, only one representation of the row should be > returned, which is the reinserted one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KUDU-1563) Add support for INSERT IGNORE
[
https://issues.apache.org/jira/browse/KUDU-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724448#comment-16724448
]
Adar Dembo commented on KUDU-1563:
--
bq. What would be implications be if we could not do this in a backwards
compatible way?
It really depends on the incompatibility itself. IIRC, your draft adds a new
data member to {{KuduWriteOperation}}. This class is allocated by
libkudu_client.so via calls like {{KuduTable::NewInsert}}, and, most of the
time, deallocated by libkudu_client.so after it has taken ownership of the
operation in {{KuduSession::Apply}} and sent it on the wire. However, if the
operation failed, it'll be assigned to a {{KuduError}} and passed back to the
third party application, and the application can choose to take ownership of it
via {{KuduError::release_failed_op}}. At that point, the application is on the
hook for deallocating it, and if the application and libkudu_client.so don't
agree on the size and layout of the class, memory will get corrupted by the
deallocation.
In short, the severity and impact of the incompatibility varies on a case by
case basis, and is pretty difficult to assess thoroughly, which is why I'd err
on the side of either "don't do it", or "do it, and rev the client SONAME's
major version to express the incompatibility".
bq. I am not sure how to solve the PIMPL'ed issue either but I am happy to
investigate.
If you could isolate the changes to just new classes/subclasses (i.e. just a
new {{KuduInsertIgnore}} subclass of {{KuduWriteOperation}}), then you'd be in
the clear.
Barring that, you could implement new variants of {{KuduWriteOperation}} and
friends that are PIMPL'ed, and modify the rest of the client APIs to support
them alongside the existing variant. Users of the C++ client will need to
change their code to use the new variants, but it'll be completely safe from a
backwards compatibility perspective.
The [KDE community wiki page on the
subject|https://community.kde.org/Policies/Binary_Compatibility_Issues_With_C%2B%2B]
has much more useful insight and some examples too.
> Add support for INSERT IGNORE
> -
>
> Key: KUDU-1563
> URL: https://issues.apache.org/jira/browse/KUDU-1563
> Project: Kudu
> Issue Type: New Feature
>Reporter: Dan Burkert
>Assignee: Brock Noland
>Priority: Major
> Labels: newbie
>
> The Java client currently has an [option to ignore duplicate row key errors|
> https://kudu.apache.org/apidocs/org/kududb/client/AsyncKuduSession.html#setIgnoreAllDuplicateRows-boolean-],
> which is implemented by filtering the errors on the client side. If we are
> going to continue to support this feature (and the consensus seems to be that
> we probably should), we should promote it to a first class operation type
> that is handled on the server side. This would have a modest perf.
> improvement since less errors are returned, and it would allow INSERT IGNORE
> ops to be mixed in the same batch as other INSERT, DELETE, UPSERT, etc. ops.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KUDU-2289) Tablet deletion should be throttled
[ https://issues.apache.org/jira/browse/KUDU-2289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adar Dembo updated KUDU-2289: - Affects Version/s: 1.7.0 > Tablet deletion should be throttled > --- > > Key: KUDU-2289 > URL: https://issues.apache.org/jira/browse/KUDU-2289 > Project: Kudu > Issue Type: Improvement > Components: tserver >Affects Versions: 1.7.0 >Reporter: Todd Lipcon >Assignee: Will Berkeley >Priority: Major > Fix For: 1.8.0 > > > Currently if a large amount of data is deleted simultaneously, the master > will not do any throttling of the DeleteTablet requests send to the tservers. > The tservers will use up to the configured number of service threads to work > on deleting tablets. The deletion can be relatively heavy-weight -- lots of > file system operations to hole-punch out the dead tablets, etc. This can have > a negative impact on other concurrent workloads. > It would be desirable to do some throttling either on the master or tserver > side to avoid overwhelming disks and thread resources during heavy deletion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KUDU-2289) Tablet deletion should be throttled
[ https://issues.apache.org/jira/browse/KUDU-2289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adar Dembo resolved KUDU-2289. -- Resolution: Fixed Fix Version/s: 1.8.0 Yes, Will fixed this in commit 165688f83bbb00a41090e909ae4e17c463fa75e4. > Tablet deletion should be throttled > --- > > Key: KUDU-2289 > URL: https://issues.apache.org/jira/browse/KUDU-2289 > Project: Kudu > Issue Type: Improvement > Components: tserver >Reporter: Todd Lipcon >Assignee: Will Berkeley >Priority: Major > Fix For: 1.8.0 > > > Currently if a large amount of data is deleted simultaneously, the master > will not do any throttling of the DeleteTablet requests send to the tservers. > The tservers will use up to the configured number of service threads to work > on deleting tablets. The deletion can be relatively heavy-weight -- lots of > file system operations to hole-punch out the dead tablets, etc. This can have > a negative impact on other concurrent workloads. > It would be desirable to do some throttling either on the master or tserver > side to avoid overwhelming disks and thread resources during heavy deletion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
