[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[
https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-2956:
Component/s: (was: General)
Job Engine
> building trie dictionary blocked on value of length over 4095
> --
>
> Key: KYLIN-2956
> URL: https://issues.apache.org/jira/browse/KYLIN-2956
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Reporter: Wang, Gang
>Assignee: Wang, Gang
>Priority: Major
> Fix For: v2.3.0
>
> Attachments:
> 0001-KYLIN-2956-building-trie-dictionary-blocked-on-value.patch
>
>
> In the new release, Kylin will check the value length when building trie
> dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through
> method:
> private void positiveShortPreCheck(int i, String fieldName) {
> if (!BytesUtil.isPositiveShort(i)) {
> throw new IllegalStateException(fieldName + " is not positive short,
> usually caused by too long dict value.");
> }
> }
> public static boolean isPositiveShort(int i) {
> return (i & 0x7000) == 0;
> }
> And 0x7000 in binary: 0111 , so the
> value length should be less than 0001 0001 ,
> values 4095 in decimalism.
> I wonder why is 0x7000, should 0x8000 ( 1000
> ), support max length: 0111
> (32767)
> be what you want?
> Or 32767 may be too large, I prefer use 0xE000, 0xE000 (
> 1110 ), support max length: 0001
> (8191)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[
https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated KYLIN-2956:
--
Attachment: 0001-KYLIN-2956-building-trie-dictionary-blocked-on-value.patch
> building trie dictionary blocked on value of length over 4095
> --
>
> Key: KYLIN-2956
> URL: https://issues.apache.org/jira/browse/KYLIN-2956
> Project: Kylin
> Issue Type: Bug
> Components: General
>Reporter: Wang, Gang
>Assignee: Wang, Gang
> Attachments:
> 0001-KYLIN-2956-building-trie-dictionary-blocked-on-value.patch
>
>
> In the new release, Kylin will check the value length when building trie
> dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through
> method:
> private void positiveShortPreCheck(int i, String fieldName) {
> if (!BytesUtil.isPositiveShort(i)) {
> throw new IllegalStateException(fieldName + " is not positive short,
> usually caused by too long dict value.");
> }
> }
> public static boolean isPositiveShort(int i) {
> return (i & 0x7000) == 0;
> }
> And 0x7000 in binary: 0111 , so the
> value length should be less than 0001 0001 ,
> values 4095 in decimalism.
> I wonder why is 0x7000, should 0x8000 ( 1000
> ), support max length: 0111
> (32767)
> be what you want?
> Or 32767 may be too large, I prefer use 0xE000, 0xE000 (
> 1110 ), support max length: 0001
> (8191)
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[
https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated KYLIN-2956:
--
Attachment: (was:
0001-KYLIN-2956-building-trie-dictionary-blocked-on-value.patch)
> building trie dictionary blocked on value of length over 4095
> --
>
> Key: KYLIN-2956
> URL: https://issues.apache.org/jira/browse/KYLIN-2956
> Project: Kylin
> Issue Type: Bug
> Components: General
>Reporter: Wang, Gang
>Assignee: Wang, Gang
>
> In the new release, Kylin will check the value length when building trie
> dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through
> method:
> private void positiveShortPreCheck(int i, String fieldName) {
> if (!BytesUtil.isPositiveShort(i)) {
> throw new IllegalStateException(fieldName + " is not positive short,
> usually caused by too long dict value.");
> }
> }
> public static boolean isPositiveShort(int i) {
> return (i & 0x7000) == 0;
> }
> And 0x7000 in binary: 0111 , so the
> value length should be less than 0001 0001 ,
> values 4095 in decimalism.
> I wonder why is 0x7000, should 0x8000 ( 1000
> ), support max length: 0111
> (32767)
> be what you want?
> Or 32767 may be too large, I prefer use 0xE000, 0xE000 (
> 1110 ), support max length: 0001
> (8191)
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[
https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated KYLIN-2956:
--
Attachment: 0001-KYLIN-2956-building-trie-dictionary-blocked-on-value.patch
I think when building trie dictionary, 32767 is too huge as the value length
limit, 8191 should make length. Fix as '0xE000'.
> building trie dictionary blocked on value of length over 4095
> --
>
> Key: KYLIN-2956
> URL: https://issues.apache.org/jira/browse/KYLIN-2956
> Project: Kylin
> Issue Type: Bug
> Components: General
>Reporter: Wang, Gang
>Assignee: Wang, Gang
> Attachments:
> 0001-KYLIN-2956-building-trie-dictionary-blocked-on-value.patch
>
>
> In the new release, Kylin will check the value length when building trie
> dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through
> method:
> private void positiveShortPreCheck(int i, String fieldName) {
> if (!BytesUtil.isPositiveShort(i)) {
> throw new IllegalStateException(fieldName + " is not positive short,
> usually caused by too long dict value.");
> }
> }
> public static boolean isPositiveShort(int i) {
> return (i & 0x7000) == 0;
> }
> And 0x7000 in binary: 0111 , so the
> value length should be less than 0001 0001 ,
> values 4095 in decimalism.
> I wonder why is 0x7000, should 0x8000 ( 1000
> ), support max length: 0111
> (32767)
> be what you want?
> Or 32767 may be too large, I prefer use 0xE000, 0xE000 (
> 1110 ), support max length: 0001
> (8191)
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[
https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated KYLIN-2956:
--
Description:
In the new release, Kylin will check the value length when building trie
dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through
method:
private void positiveShortPreCheck(int i, String fieldName) {
if (!BytesUtil.isPositiveShort(i)) {
throw new IllegalStateException(fieldName + " is not positive short,
usually caused by too long dict value.");
}
}
public static boolean isPositiveShort(int i) {
return (i & 0x7000) == 0;
}
And 0x7000 in binary: 0111 , so the
value length should be less than 0001 0001 ,
values 4095 in decimalism.
I wonder why is 0x7000, should 0x8000 ( 1000
), support max length: 0111
(32767)
be what you want?
Or 32767 may be too large, I prefer use 0xE000, 0xE000 (
1110 ), support max length: 0001
(8191)
was:
In the new release, Kylin will check the value length when building trie
dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through
method:
private void positiveShortPreCheck(int i, String fieldName) {
if (!BytesUtil.isPositiveShort(i)) {
throw new IllegalStateException(fieldName + " is not positive short,
usually caused by too long dict value.");
}
}
public static boolean isPositiveShort(int i) {
return (i & 0x7000) == 0;
}
And 0x7000 in binary: 0111 , so the
value length should be less than 0001 0001 ,
values 4095 in decimalism.
I wonder why is 0x7000, should
0x8000: 1000
support max length: 0111 (32767)
be what you want? And 32767 may be too lagrge, I prefer use 0xE000,
0xE000: 1110 ,
support max length: 0001 (8191)
> building trie dictionary blocked on value of length over 4095
> --
>
> Key: KYLIN-2956
> URL: https://issues.apache.org/jira/browse/KYLIN-2956
> Project: Kylin
> Issue Type: Bug
> Components: General
>Reporter: Wang, Gang
>Assignee: Shaofeng SHI
>
> In the new release, Kylin will check the value length when building trie
> dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through
> method:
> private void positiveShortPreCheck(int i, String fieldName) {
> if (!BytesUtil.isPositiveShort(i)) {
> throw new IllegalStateException(fieldName + " is not positive short,
> usually caused by too long dict value.");
> }
> }
> public static boolean isPositiveShort(int i) {
> return (i & 0x7000) == 0;
> }
> And 0x7000 in binary: 0111 , so the
> value length should be less than 0001 0001 ,
> values 4095 in decimalism.
> I wonder why is 0x7000, should 0x8000 ( 1000
> ), support max length: 0111
> (32767)
> be what you want?
> Or 32767 may be too large, I prefer use 0xE000, 0xE000 (
> 1110 ), support max length: 0001
> (8191)
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[
https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated KYLIN-2956:
--
Description:
In the new release, Kylin will check the value length when building trie
dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through
method:
private void positiveShortPreCheck(int i, String fieldName) {
if (!BytesUtil.isPositiveShort(i)) {
throw new IllegalStateException(fieldName + " is not positive short,
usually caused by too long dict value.");
}
}
public static boolean isPositiveShort(int i) {
return (i & 0x7000) == 0;
}
And 0x7000 in binary: 0111 , so the
value length should be less than 0001 0001 ,
values 4095 in decimalism.
I wonder why is 0x7000, should
0x8000: 1000
support max length: 0111 (32767)
be what you want? And 32767 may be too lagrge, I prefer use 0xE000,
0xE000: 1110 ,
support max length: 0001 (8191)
was:
In the new release, Kylin will check the value length when building trie
dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through
method:
_private void positiveShortPreCheck(int i, String fieldName) {
if (!BytesUtil.isPositiveShort(i)) {
throw new IllegalStateException(fieldName + " is not positive short,
usually caused by too long dict value.");
}
}_
_public static boolean isPositiveShort(int i) {
return (i & 0x7000) == 0;
}
_
And 0x7000 in binary: 0111 , so the
value length should be less than 0001 0001 ,
values 4095 in decimalism.
I wonder why is 0x7000, should
0x8000: 1000
support max length: 0111 (32767)
be what you want? And 32767 may be too lagrge, I prefer use 0xE000,
0xE000: 1110 ,
support max length: 0001 (8191)
> building trie dictionary blocked on value of length over 4095
> --
>
> Key: KYLIN-2956
> URL: https://issues.apache.org/jira/browse/KYLIN-2956
> Project: Kylin
> Issue Type: Bug
> Components: General
>Reporter: Wang, Gang
>Assignee: Wang, Gang
>
> In the new release, Kylin will check the value length when building trie
> dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through
> method:
> private void positiveShortPreCheck(int i, String fieldName) {
> if (!BytesUtil.isPositiveShort(i)) {
> throw new IllegalStateException(fieldName + " is not positive short,
> usually caused by too long dict value.");
> }
> }
> public static boolean isPositiveShort(int i) {
> return (i & 0x7000) == 0;
> }
> And 0x7000 in binary: 0111 , so the
> value length should be less than 0001 0001 ,
> values 4095 in decimalism.
> I wonder why is 0x7000, should
> 0x8000: 1000
> support max length: 0111 (32767)
> be what you want? And 32767 may be too lagrge, I prefer use 0xE000,
> 0xE000: 1110 ,
> support max length: 0001 (8191)
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[
https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Gang updated KYLIN-2956:
--
Description:
In the new release, Kylin will check the value length when building trie
dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through
method:
_private void positiveShortPreCheck(int i, String fieldName) {
if (!BytesUtil.isPositiveShort(i)) {
throw new IllegalStateException(fieldName + " is not positive short,
usually caused by too long dict value.");
}
}_
_public static boolean isPositiveShort(int i) {
return (i & 0x7000) == 0;
}
_
And 0x7000 in binary: 0111 , so the
value length should be less than 0001 0001 ,
values 4095 in decimalism.
I wonder why is 0x7000, should
0x8000: 1000
support max length: 0111 (32767)
be what you want? And 32767 may be too lagrge, I prefer use 0xE000,
0xE000: 1110 ,
support max length: 0001 (8191)
was:
In the new release, Kylin will check the value length when building trie
dictionary, in class _TrieDictionaryBuilder_ method _buildTrieBytes_ , through
method:
_private void positiveShortPreCheck(int i, String fieldName) {
if (!BytesUtil.isPositiveShort(i)) {
throw new IllegalStateException(fieldName + " is not positive short,
usually caused by too long dict value.");
}
} _
_public static boolean isPositiveShort(int i) {
return (i & 0x7000) == 0;
}_
And 0x7000 in binary: 0111 , so the
value length should be less than 0001 0001 ,
values 4095 in decimalism.
I wonder why is 0x7000, should
0x8000: 1000
support max length: 0111 (32767)
be what you want? And 32767 may be too lagrge, I prefer use 0xE000,
0xE000: 1110 ,
support max length: 0001 (8191)
> building trie dictionary blocked on value of length over 4095
> --
>
> Key: KYLIN-2956
> URL: https://issues.apache.org/jira/browse/KYLIN-2956
> Project: Kylin
> Issue Type: Bug
> Components: General
>Reporter: Wang, Gang
>Assignee: Wang, Gang
>
> In the new release, Kylin will check the value length when building trie
> dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through
> method:
> _private void positiveShortPreCheck(int i, String fieldName) {
> if (!BytesUtil.isPositiveShort(i)) {
> throw new IllegalStateException(fieldName + " is not positive short,
> usually caused by too long dict value.");
> }
> }_
> _public static boolean isPositiveShort(int i) {
> return (i & 0x7000) == 0;
> }
> _
> And 0x7000 in binary: 0111 , so the
> value length should be less than 0001 0001 ,
> values 4095 in decimalism.
> I wonder why is 0x7000, should
> 0x8000: 1000
> support max length: 0111 (32767)
> be what you want? And 32767 may be too lagrge, I prefer use 0xE000,
> 0xE000: 1110 ,
> support max length: 0001 (8191)
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
