[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[ https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-2956: Component/s: (was: General) Job Engine > building trie dictionary blocked on value of length over 4095 > -- > > Key: KYLIN-2956 > URL: https://issues.apache.org/jira/browse/KYLIN-2956 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Reporter: Wang, Gang >Assignee: Wang, Gang >Priority: Major > Fix For: v2.3.0 > > Attachments: > 0001-KYLIN-2956-building-trie-dictionary-blocked-on-value.patch > > > In the new release, Kylin will check the value length when building trie > dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through > method: > private void positiveShortPreCheck(int i, String fieldName) { > if (!BytesUtil.isPositiveShort(i)) { > throw new IllegalStateException(fieldName + " is not positive short, > usually caused by too long dict value."); > } > } > public static boolean isPositiveShort(int i) { > return (i & 0x7000) == 0; > } > And 0x7000 in binary: 0111 , so the > value length should be less than 0001 0001 , > values 4095 in decimalism. > I wonder why is 0x7000, should 0x8000 ( 1000 > ), support max length: 0111 > (32767) > be what you want? > Or 32767 may be too large, I prefer use 0xE000, 0xE000 ( > 1110 ), support max length: 0001 > (8191) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[ https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang, Gang updated KYLIN-2956: -- Attachment: 0001-KYLIN-2956-building-trie-dictionary-blocked-on-value.patch > building trie dictionary blocked on value of length over 4095 > -- > > Key: KYLIN-2956 > URL: https://issues.apache.org/jira/browse/KYLIN-2956 > Project: Kylin > Issue Type: Bug > Components: General >Reporter: Wang, Gang >Assignee: Wang, Gang > Attachments: > 0001-KYLIN-2956-building-trie-dictionary-blocked-on-value.patch > > > In the new release, Kylin will check the value length when building trie > dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through > method: > private void positiveShortPreCheck(int i, String fieldName) { > if (!BytesUtil.isPositiveShort(i)) { > throw new IllegalStateException(fieldName + " is not positive short, > usually caused by too long dict value."); > } > } > public static boolean isPositiveShort(int i) { > return (i & 0x7000) == 0; > } > And 0x7000 in binary: 0111 , so the > value length should be less than 0001 0001 , > values 4095 in decimalism. > I wonder why is 0x7000, should 0x8000 ( 1000 > ), support max length: 0111 > (32767) > be what you want? > Or 32767 may be too large, I prefer use 0xE000, 0xE000 ( > 1110 ), support max length: 0001 > (8191) > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[ https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang, Gang updated KYLIN-2956: -- Attachment: (was: 0001-KYLIN-2956-building-trie-dictionary-blocked-on-value.patch) > building trie dictionary blocked on value of length over 4095 > -- > > Key: KYLIN-2956 > URL: https://issues.apache.org/jira/browse/KYLIN-2956 > Project: Kylin > Issue Type: Bug > Components: General >Reporter: Wang, Gang >Assignee: Wang, Gang > > In the new release, Kylin will check the value length when building trie > dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through > method: > private void positiveShortPreCheck(int i, String fieldName) { > if (!BytesUtil.isPositiveShort(i)) { > throw new IllegalStateException(fieldName + " is not positive short, > usually caused by too long dict value."); > } > } > public static boolean isPositiveShort(int i) { > return (i & 0x7000) == 0; > } > And 0x7000 in binary: 0111 , so the > value length should be less than 0001 0001 , > values 4095 in decimalism. > I wonder why is 0x7000, should 0x8000 ( 1000 > ), support max length: 0111 > (32767) > be what you want? > Or 32767 may be too large, I prefer use 0xE000, 0xE000 ( > 1110 ), support max length: 0001 > (8191) > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[ https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang, Gang updated KYLIN-2956: -- Attachment: 0001-KYLIN-2956-building-trie-dictionary-blocked-on-value.patch I think when building trie dictionary, 32767 is too huge as the value length limit, 8191 should make length. Fix as '0xE000'. > building trie dictionary blocked on value of length over 4095 > -- > > Key: KYLIN-2956 > URL: https://issues.apache.org/jira/browse/KYLIN-2956 > Project: Kylin > Issue Type: Bug > Components: General >Reporter: Wang, Gang >Assignee: Wang, Gang > Attachments: > 0001-KYLIN-2956-building-trie-dictionary-blocked-on-value.patch > > > In the new release, Kylin will check the value length when building trie > dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through > method: > private void positiveShortPreCheck(int i, String fieldName) { > if (!BytesUtil.isPositiveShort(i)) { > throw new IllegalStateException(fieldName + " is not positive short, > usually caused by too long dict value."); > } > } > public static boolean isPositiveShort(int i) { > return (i & 0x7000) == 0; > } > And 0x7000 in binary: 0111 , so the > value length should be less than 0001 0001 , > values 4095 in decimalism. > I wonder why is 0x7000, should 0x8000 ( 1000 > ), support max length: 0111 > (32767) > be what you want? > Or 32767 may be too large, I prefer use 0xE000, 0xE000 ( > 1110 ), support max length: 0001 > (8191) > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[ https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang, Gang updated KYLIN-2956: -- Description: In the new release, Kylin will check the value length when building trie dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through method: private void positiveShortPreCheck(int i, String fieldName) { if (!BytesUtil.isPositiveShort(i)) { throw new IllegalStateException(fieldName + " is not positive short, usually caused by too long dict value."); } } public static boolean isPositiveShort(int i) { return (i & 0x7000) == 0; } And 0x7000 in binary: 0111 , so the value length should be less than 0001 0001 , values 4095 in decimalism. I wonder why is 0x7000, should 0x8000 ( 1000 ), support max length: 0111 (32767) be what you want? Or 32767 may be too large, I prefer use 0xE000, 0xE000 ( 1110 ), support max length: 0001 (8191) was: In the new release, Kylin will check the value length when building trie dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through method: private void positiveShortPreCheck(int i, String fieldName) { if (!BytesUtil.isPositiveShort(i)) { throw new IllegalStateException(fieldName + " is not positive short, usually caused by too long dict value."); } } public static boolean isPositiveShort(int i) { return (i & 0x7000) == 0; } And 0x7000 in binary: 0111 , so the value length should be less than 0001 0001 , values 4095 in decimalism. I wonder why is 0x7000, should 0x8000: 1000 support max length: 0111 (32767) be what you want? And 32767 may be too lagrge, I prefer use 0xE000, 0xE000: 1110 , support max length: 0001 (8191) > building trie dictionary blocked on value of length over 4095 > -- > > Key: KYLIN-2956 > URL: https://issues.apache.org/jira/browse/KYLIN-2956 > Project: Kylin > Issue Type: Bug > Components: General >Reporter: Wang, Gang >Assignee: Shaofeng SHI > > In the new release, Kylin will check the value length when building trie > dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through > method: > private void positiveShortPreCheck(int i, String fieldName) { > if (!BytesUtil.isPositiveShort(i)) { > throw new IllegalStateException(fieldName + " is not positive short, > usually caused by too long dict value."); > } > } > public static boolean isPositiveShort(int i) { > return (i & 0x7000) == 0; > } > And 0x7000 in binary: 0111 , so the > value length should be less than 0001 0001 , > values 4095 in decimalism. > I wonder why is 0x7000, should 0x8000 ( 1000 > ), support max length: 0111 > (32767) > be what you want? > Or 32767 may be too large, I prefer use 0xE000, 0xE000 ( > 1110 ), support max length: 0001 > (8191) > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[ https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang, Gang updated KYLIN-2956: -- Description: In the new release, Kylin will check the value length when building trie dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through method: private void positiveShortPreCheck(int i, String fieldName) { if (!BytesUtil.isPositiveShort(i)) { throw new IllegalStateException(fieldName + " is not positive short, usually caused by too long dict value."); } } public static boolean isPositiveShort(int i) { return (i & 0x7000) == 0; } And 0x7000 in binary: 0111 , so the value length should be less than 0001 0001 , values 4095 in decimalism. I wonder why is 0x7000, should 0x8000: 1000 support max length: 0111 (32767) be what you want? And 32767 may be too lagrge, I prefer use 0xE000, 0xE000: 1110 , support max length: 0001 (8191) was: In the new release, Kylin will check the value length when building trie dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through method: _private void positiveShortPreCheck(int i, String fieldName) { if (!BytesUtil.isPositiveShort(i)) { throw new IllegalStateException(fieldName + " is not positive short, usually caused by too long dict value."); } }_ _public static boolean isPositiveShort(int i) { return (i & 0x7000) == 0; } _ And 0x7000 in binary: 0111 , so the value length should be less than 0001 0001 , values 4095 in decimalism. I wonder why is 0x7000, should 0x8000: 1000 support max length: 0111 (32767) be what you want? And 32767 may be too lagrge, I prefer use 0xE000, 0xE000: 1110 , support max length: 0001 (8191) > building trie dictionary blocked on value of length over 4095 > -- > > Key: KYLIN-2956 > URL: https://issues.apache.org/jira/browse/KYLIN-2956 > Project: Kylin > Issue Type: Bug > Components: General >Reporter: Wang, Gang >Assignee: Wang, Gang > > In the new release, Kylin will check the value length when building trie > dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through > method: > private void positiveShortPreCheck(int i, String fieldName) { > if (!BytesUtil.isPositiveShort(i)) { > throw new IllegalStateException(fieldName + " is not positive short, > usually caused by too long dict value."); > } > } > public static boolean isPositiveShort(int i) { > return (i & 0x7000) == 0; > } > And 0x7000 in binary: 0111 , so the > value length should be less than 0001 0001 , > values 4095 in decimalism. > I wonder why is 0x7000, should > 0x8000: 1000 > support max length: 0111 (32767) > be what you want? And 32767 may be too lagrge, I prefer use 0xE000, > 0xE000: 1110 , > support max length: 0001 (8191) > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2956) building trie dictionary blocked on value of length over 4095
[ https://issues.apache.org/jira/browse/KYLIN-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wang, Gang updated KYLIN-2956: -- Description: In the new release, Kylin will check the value length when building trie dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through method: _private void positiveShortPreCheck(int i, String fieldName) { if (!BytesUtil.isPositiveShort(i)) { throw new IllegalStateException(fieldName + " is not positive short, usually caused by too long dict value."); } }_ _public static boolean isPositiveShort(int i) { return (i & 0x7000) == 0; } _ And 0x7000 in binary: 0111 , so the value length should be less than 0001 0001 , values 4095 in decimalism. I wonder why is 0x7000, should 0x8000: 1000 support max length: 0111 (32767) be what you want? And 32767 may be too lagrge, I prefer use 0xE000, 0xE000: 1110 , support max length: 0001 (8191) was: In the new release, Kylin will check the value length when building trie dictionary, in class _TrieDictionaryBuilder_ method _buildTrieBytes_ , through method: _private void positiveShortPreCheck(int i, String fieldName) { if (!BytesUtil.isPositiveShort(i)) { throw new IllegalStateException(fieldName + " is not positive short, usually caused by too long dict value."); } } _ _public static boolean isPositiveShort(int i) { return (i & 0x7000) == 0; }_ And 0x7000 in binary: 0111 , so the value length should be less than 0001 0001 , values 4095 in decimalism. I wonder why is 0x7000, should 0x8000: 1000 support max length: 0111 (32767) be what you want? And 32767 may be too lagrge, I prefer use 0xE000, 0xE000: 1110 , support max length: 0001 (8191) > building trie dictionary blocked on value of length over 4095 > -- > > Key: KYLIN-2956 > URL: https://issues.apache.org/jira/browse/KYLIN-2956 > Project: Kylin > Issue Type: Bug > Components: General >Reporter: Wang, Gang >Assignee: Wang, Gang > > In the new release, Kylin will check the value length when building trie > dictionary, in class TrieDictionaryBuilder method buildTrieBytes, through > method: > _private void positiveShortPreCheck(int i, String fieldName) { > if (!BytesUtil.isPositiveShort(i)) { > throw new IllegalStateException(fieldName + " is not positive short, > usually caused by too long dict value."); > } > }_ > _public static boolean isPositiveShort(int i) { > return (i & 0x7000) == 0; > } > _ > And 0x7000 in binary: 0111 , so the > value length should be less than 0001 0001 , > values 4095 in decimalism. > I wonder why is 0x7000, should > 0x8000: 1000 > support max length: 0111 (32767) > be what you want? And 32767 may be too lagrge, I prefer use 0xE000, > 0xE000: 1110 , > support max length: 0001 (8191) > -- This message was sent by Atlassian JIRA (v6.4.14#64029)