Section 5.1 Evaluation *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* Textual 0.943 0.669 0.453 0.832 0.782 0.733 0.681 KE4IR *0.971* *0.680* *0.474* *0.854* *0.806* *0.758* *0.713* KE4IR vs. Textual 3.03% 1.71% 4.55% 2.64% 2.99% 3.50% 4.74% p-value (paired t-test) 0.162 0.080 *0.035* *0.002* *0.008* *0.012* *0.015* p-value (approx. random.) 0.500 0.251 0.055 *0.000* *0.007* *0.008* *0.012* *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* Textual 0.943 0.669 0.453 0.832 0.782 0.733 0.681 KE4IR_GU *0.971* *0.691* *0.482* *0.879* *0.831* *0.800* *0.749* KE4IR_GU vs. Textual 3.03% 3.42% 6.49% 5.63% 6.18% 9.18% 9.98% p-value (paired t-test) 0.286 0.105 *0.020* *0.006* *0.007* *0.003* *0.006* p-value (approx. random.) 0.500 0.135 *0.012* *0.003* *0.006* *0.001* *0.004* *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* KE4IR 0.971 0.680 0.474 0.854 0.806 0.758 0.713 KE4IR_GU 0.971 *0.691* *0.482* *0.879* *0.831* *0.800* *0.749* KE4IR_GU vs. KE4IR 0.00% 1.68% 1.86% 2.91% 3.09% 5.49% 5.00% p-value (paired t-test) 0.500 0.244 0.162 0.064 0.077 *0.018* *0.044* p-value (approx. random.) 0.500 0.293 0.211 0.065 0.079 *0.011* *0.039* Section 5.2 Evaluation *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* Textual 0.400 0.420 0.410 0.367 0.344 0.159 0.051 KE4IR *0.550* *0.460* *0.440* *0.496* *0.374* *0.227* *0.077* KE4IR vs. Textual 37.50% 9.52% 7.32% 35.14% 8.65% 42.89% 50.56% p-value (paired t-test) *0.041* 0.204 0.247 *0.001* 0.244 *0.005* 0.057 p-value (approx. random.) 0.125 0.153 0.268 *0.000* 0.257 *0.001* 0.057 *Approach/System* *p@10* *map* F&al. 0.370 0.157 KE4IR *0.440* *0.227* KE4IR vs. F&al. 18.92% 44.43% p-value (paired t-test) 0.096 *0.026* p-value (approx. random.) 0.101 *0.026* Section 5.3 Evaluation Trec 6 title *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* Textual 0.380 0.388 0.308 0.419 0.357 0.189 0.091 KE4IR *0.420* *0.392* *0.318* *0.446* *0.369* *0.203* *0.095* KE4IR vs. Textual 10.53% 1.03% 3.31% 6.49% 3.32% 7.81% 4.40% p-value (paired t-test) 0.161 0.411 0.307 *0.032* 0.269 0.142 0.255 p-value (approx. random.) 0.314 0.438 0.301 *0.027* 0.316 0.156 0.255 Trec 7 title *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* Textual 0.480 0.400 0.378 0.408 0.397 0.169 0.065 KE4IR *0.500* *0.420* *0.394* *0.432* *0.411* *0.182* *0.069* KE4IR vs. Textual 4.17% 5.00% 4.23% 6.02% 3.56% 7.41% 5.05% p-value (paired t-test) 0.330 0.128 0.169 *0.002* 0.170 *0.022* 0.060 p-value (approx. random.) 0.500 0.129 0.164 *0.001* 0.195 *0.021* 0.062 Trec 8 title *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* Textual *0.540* 0.416 0.386 0.450 0.413 0.190 0.074 KE4IR 0.500 *0.428* *0.388* *0.457* *0.414* *0.192* *0.075* KE4IR vs. Textual -7.41% 2.88% 0.52% 1.51% 0.34% 0.89% 1.84% p-value (paired t-test) 0.080 0.284 0.451 0.193 0.464 0.345 0.349 p-value (approx. random.) 0.250 0.266 0.407 0.196 0.465 0.346 0.336 Trec 9 title *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* Textual 0.360 0.284 0.267 0.393 0.302 0.176 *0.110* KE4IR 0.360 *0.292* *0.284* *0.412* *0.309* *0.184* 0.102 KE4IR vs. Textual 0.00% 2.82% 6.11% 4.89% 2.49% 4.75% -7.01% p-value (paired t-test) 0.500 0.299 0.086 0.120 0.310 0.292 0.277 p-value (approx. random.) 0.500 0.388 0.096 0.127 0.317 0.322 0.376 Trec 2001 title *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* Textual 0.380 0.356 0.294 0.336 0.289 0.122 0.053 KE4IR *0.400* *0.388* *0.322* *0.410* *0.325* *0.170* *0.067* KE4IR vs. Textual 5.26% 8.99% 9.52% 22.20% 12.18% 38.97% 25.95% p-value (paired t-test) 0.284 0.052 *0.040* *0.000* *0.022* *0.001* 0.063 p-value (approx. random.) 0.500 0.060 0.051 *0.000* *0.020* *0.000* 0.059 Trec 6 title+description *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* Textual 0.400 0.312 0.264 0.286 0.294 0.118 0.067 KE4IR *0.440* *0.368* *0.312* *0.389* *0.356* *0.176* *0.089* KE4IR vs. Textual 10.00% 17.95% 18.18% 35.97% 21.25% 48.99% 34.44% p-value (paired t-test) 0.080 *0.021* *0.016* *0.000* *0.008* *0.001* 0.057 p-value (approx. random.) 0.250 *0.016* *0.009* *0.000* *0.004* *0.000* 0.050 Trec 7 title+description *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* Textual 0.460 0.388 0.342 0.312 0.366 0.114 0.052 KE4IR *0.480* *0.416* *0.402* *0.394* *0.421* *0.162* *0.065* KE4IR vs. Textual 4.35% 7.22% 17.54% 26.38% 15.05% 41.64% 23.44% p-value (paired t-test) 0.355 0.151 *0.007* *0.000* *0.006* *0.001* *0.002* p-value (approx. random.) 0.500 0.158 *0.006* *0.000* *0.004* *0.000* *0.000* Trec 8 title+description *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* Textual 0.480 0.400 0.368 0.354 0.395 0.150 0.071 KE4IR *0.500* *0.408* *0.382* *0.420* *0.410* *0.178* *0.077* KE4IR vs. Textual 4.17% 2.00% 3.80% 18.62% 3.75% 19.32% 8.79% p-value (paired t-test) 0.284 0.321 0.090 *0.000* 0.055 *0.001* 0.091 p-value (approx. random.) 0.500 0.409 0.127 *0.000* 0.055 *0.000* 0.102 Trec 9 title+description *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* Textual 0.380 0.328 0.288 0.270 0.275 0.118 0.081 KE4IR *0.440* *0.340* *0.290* *0.403* *0.323* *0.184* *0.104* KE4IR vs. Textual 15.79% 3.66% 0.69% 48.99% 17.61% 56.63% 28.26% p-value (paired t-test) *0.042* 0.333 0.460 *0.000* *0.026* *0.000* *0.014* p-value (approx. random.) 0.124 0.304 0.473 *0.000* *0.025* *0.000* *0.010* Trec 2001 title+description *Approach/System* *p@1* *p@5* *p@10* *ndcg* *ndcg@10* *map* *map@10* Textual *0.440* 0.380 0.296 0.266 0.294 0.099 0.058 KE4IR 0.400 *0.388* *0.332* *0.389* *0.343* *0.163* *0.072* KE4IR vs. Textual -9.09% 2.11% 12.16% 46.28% 16.52% 63.47% 23.29% p-value (paired t-test) 0.080 0.344 *0.027* *0.000* *0.005* *0.000* 0.063 p-value (approx. random.) 0.250 0.321 *0.032* *0.000* *0.004* *0.000* 0.058