[
https://issues.apache.org/jira/browse/LUCENENET-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12623464#action_12623464
]
Digy commented on LUCENENET-144:
--------------------------------
Just thinking loudly;
The failures of tests TestSpanNearOrdered0[2-5] may be due to loss of precision
while writing float-norms as single byte to index.
For ex, the norm for the first doc in TestSpans ("w1 w2 w3 w4 w5") is
1.0 / System.Math.Sqrt(numTerms[which is 5 in this case]) = 0.447213595499958
which is encoded as 119 in a single byte and decoded as 0.4375 (for second doc:
0.5 --> 120 --> 0.5 (no loss))
So if we change the initialization of NORM_TABLE in Similarity.cs as below
{code}
static Similarity()
{
for (int i = 0; i < 256; i++)
{
NORM_TABLE[i] =
SmallFloat.Byte315ToFloat((byte)i);
}
NORM_TABLE[119] = 0.447213595499958f; //which was
0.4375 originally
}
{code}
all tests pass.
I wonder how these tests pass in java?
DIGY
NORM_TABLE:
000 -> 0.00000000000
001 -> 0.00000000058
002 -> 0.00000000070
003 -> 0.00000000081
004 -> 0.00000000093
005 -> 0.00000000116
006 -> 0.00000000140
007 -> 0.00000000163
008 -> 0.00000000186
009 -> 0.00000000233
010 -> 0.00000000279
011 -> 0.00000000326
012 -> 0.00000000373
013 -> 0.00000000466
014 -> 0.00000000559
015 -> 0.00000000652
016 -> 0.00000000745
017 -> 0.00000000931
018 -> 0.00000001118
019 -> 0.00000001304
020 -> 0.00000001490
021 -> 0.00000001863
022 -> 0.00000002235
023 -> 0.00000002608
024 -> 0.00000002980
025 -> 0.00000003725
026 -> 0.00000004470
027 -> 0.00000005215
028 -> 0.00000005960
029 -> 0.00000007451
030 -> 0.00000008941
031 -> 0.00000010431
032 -> 0.00000011921
033 -> 0.00000014901
034 -> 0.00000017881
035 -> 0.00000020862
036 -> 0.00000023842
037 -> 0.00000029802
038 -> 0.00000035763
039 -> 0.00000041723
040 -> 0.00000047684
041 -> 0.00000059605
042 -> 0.00000071526
043 -> 0.00000083447
044 -> 0.00000095367
045 -> 0.00000119209
046 -> 0.00000143051
047 -> 0.00000166893
048 -> 0.00000190735
049 -> 0.00000238419
050 -> 0.00000286102
051 -> 0.00000333786
052 -> 0.00000381470
053 -> 0.00000476837
054 -> 0.00000572205
055 -> 0.00000667572
056 -> 0.00000762940
057 -> 0.00000953674
058 -> 0.00001144409
059 -> 0.00001335144
060 -> 0.00001525879
061 -> 0.00001907349
062 -> 0.00002288818
063 -> 0.00002670288
064 -> 0.00003051758
065 -> 0.00003814697
066 -> 0.00004577637
067 -> 0.00005340576
068 -> 0.00006103516
069 -> 0.00007629395
070 -> 0.00009155273
071 -> 0.00010681150
072 -> 0.00012207030
073 -> 0.00015258790
074 -> 0.00018310550
075 -> 0.00021362300
076 -> 0.00024414060
077 -> 0.00030517580
078 -> 0.00036621090
079 -> 0.00042724610
080 -> 0.00048828130
081 -> 0.00061035160
082 -> 0.00073242190
083 -> 0.00085449220
084 -> 0.00097656250
085 -> 0.00122070300
086 -> 0.00146484400
087 -> 0.00170898400
088 -> 0.00195312500
089 -> 0.00244140600
090 -> 0.00292968800
091 -> 0.00341796900
092 -> 0.00390625000
093 -> 0.00488281300
094 -> 0.00585937500
095 -> 0.00683593800
096 -> 0.00781250000
097 -> 0.00976562500
098 -> 0.01171875000
099 -> 0.01367188000
100 -> 0.01562500000
101 -> 0.01953125000
102 -> 0.02343750000
103 -> 0.02734375000
104 -> 0.03125000000
105 -> 0.03906250000
106 -> 0.04687500000
107 -> 0.05468750000
108 -> 0.06250000000
109 -> 0.07812500000
110 -> 0.09375000000
111 -> 0.10937500000
112 -> 0.12500000000
113 -> 0.15625000000
114 -> 0.18750000000
115 -> 0.21875000000
116 -> 0.25000000000
117 -> 0.31250000000
118 -> 0.37500000000
119 -> 0.43750000000
120 -> 0.50000000000
121 -> 0.62500000000
122 -> 0.75000000000
123 -> 0.87500000000
124 -> 1.00000000000
125 -> 1.25000000000
126 -> 1.50000000000
127 -> 1.75000000000
128 -> 2.00000000000
129 -> 2.50000000000
130 -> 3.00000000000
131 -> 3.50000000000
132 -> 4.00000000000
133 -> 5.00000000000
134 -> 6.00000000000
135 -> 7.00000000000
136 -> 8.00000000000
137 -> 10.00000000000
138 -> 12.00000000000
139 -> 14.00000000000
140 -> 16.00000000000
141 -> 20.00000000000
142 -> 24.00000000000
143 -> 28.00000000000
144 -> 32.00000000000
145 -> 40.00000000000
146 -> 48.00000000000
147 -> 56.00000000000
148 -> 64.00000000000
149 -> 80.00000000000
150 -> 96.00000000000
151 -> 112.00000000000
152 -> 128.00000000000
153 -> 160.00000000000
154 -> 192.00000000000
155 -> 224.00000000000
156 -> 256.00000000000
157 -> 320.00000000000
158 -> 384.00000000000
159 -> 448.00000000000
160 -> 512.00000000000
161 -> 640.00000000000
162 -> 768.00000000000
163 -> 896.00000000000
164 -> 1024.00000000000
165 -> 1280.00000000000
166 -> 1536.00000000000
167 -> 1792.00000000000
168 -> 2048.00000000000
169 -> 2560.00000000000
170 -> 3072.00000000000
171 -> 3584.00000000000
172 -> 4096.00000000000
173 -> 5120.00000000000
174 -> 6144.00000000000
175 -> 7168.00000000000
176 -> 8192.00000000000
177 -> 10240.00000000000
178 -> 12288.00000000000
179 -> 14336.00000000000
180 -> 16384.00000000000
181 -> 20480.00000000000
182 -> 24576.00000000000
183 -> 28672.00000000000
184 -> 32768.00000000000
185 -> 40960.00000000000
186 -> 49152.00000000000
187 -> 57344.00000000000
188 -> 65536.00000000000
189 -> 81920.00000000000
190 -> 98304.00000000000
191 -> 114688.00000000000
192 -> 131072.00000000000
193 -> 163840.00000000000
194 -> 196608.00000000000
195 -> 229376.00000000000
196 -> 262144.00000000000
197 -> 327680.00000000000
198 -> 393216.00000000000
199 -> 458752.00000000000
200 -> 524288.00000000000
201 -> 655360.00000000000
202 -> 786432.00000000000
203 -> 917504.00000000000
204 -> 1048576.00000000000
205 -> 1310720.00000000000
206 -> 1572864.00000000000
207 -> 1835008.00000000000
208 -> 2097152.00000000000
209 -> 2621440.00000000000
210 -> 3145728.00000000000
211 -> 3670016.00000000000
212 -> 4194304.00000000000
213 -> 5242880.00000000000
214 -> 6291456.00000000000
215 -> 7340032.00000000000
216 -> 8388608.00000000000
217 -> 10485760.00000000000
218 -> 12582910.00000000000
219 -> 14680060.00000000000
220 -> 16777220.00000000000
221 -> 20971520.00000000000
222 -> 25165820.00000000000
223 -> 29360130.00000000000
224 -> 33554430.00000000000
225 -> 41943040.00000000000
226 -> 50331650.00000000000
227 -> 58720260.00000000000
228 -> 67108860.00000000000
229 -> 83886080.00000000000
230 -> 100663300.00000000000
231 -> 117440500.00000000000
232 -> 134217700.00000000000
233 -> 167772200.00000000000
234 -> 201326600.00000000000
235 -> 234881000.00000000000
236 -> 268435500.00000000000
237 -> 335544300.00000000000
238 -> 402653200.00000000000
239 -> 469762000.00000000000
240 -> 536870900.00000000000
241 -> 671088600.00000000000
242 -> 805306400.00000000000
243 -> 939524100.00000000000
244 -> 1073742000.00000000000
245 -> 1342177000.00000000000
246 -> 1610613000.00000000000
247 -> 1879048000.00000000000
248 -> 2147484000.00000000000
249 -> 2684355000.00000000000
250 -> 3221225000.00000000000
251 -> 3758096000.00000000000
252 -> 4294967000.00000000000
253 -> 5368709000.00000000000
254 -> 6442451000.00000000000
255 -> 7516193000.00000000000
> NUnit test for Search/Spans/TestSpans
> -------------------------------------
>
> Key: LUCENENET-144
> URL: https://issues.apache.org/jira/browse/LUCENENET-144
> Project: Lucene.Net
> Issue Type: Bug
> Environment: Lucene.Net 2.3.1
> Reporter: Digy
> Attachments: SmallFloat.patch
>
>
> Bug in function Lucene.Net.Util.SmallFloat.Byte315ToFloat
> Fix for:
> TestSpanNearOrdered02
> TestSpanNearOrdered03
> TestSpanNearOrdered04
> TestSpanNearOrdered05
> DIGY
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
