LSHTC: A Benchmark for Large-Scale Text Classification

You will find here the results of the past challenges.

LSHTC1

Basic

NameAccuracyMacro F-measureMacro PrecisionMacro RecallTree Induced Error
alpaca0.4676320.3412440.3234910.3610593.07858
jhuang0.4632170.354940.3329770.3800053.28079
arthur_general0.4432910.319650.3034130.3377233.29289
XipengQiu0.4430620.3367390.3141290.3628563.36307
Turing0.431680.3232130.3045540.3443063.39343
Dyakonov0.426950.3284170.3043650.3565973.43323
logicators0.4152240.3285910.2967480.368093.57964
ysyky0.4083720.2966510.2720880.326093.4947
zwner0.4052180.281680.2683470.2964073.55427
wdd0.4033260.2941330.2804930.3091673.57761
NakaCristo0.4023220.2865460.2680280.3078133.65049
shawndr0.4018060.3155070.2923820.3426043.71531
alfonsoeromero0.3613250.2852350.2591690.3171314.08865
bbutc0.360350.225740.205590.2502714.40313
illes.solt0.3147080.1994660.197090.20194.43045
leokury0.310780.1704180.1713640.1694834.24974
semelak10.3011750.2477110.2311180.2668724.69521
brouardc0.2917140.1859950.1627410.2170044.48804
controlledchaos0.1519780.1359230.1115450.1739376.38271

Cheap

NameAccuracyMacro F-measureMacro PrecisionMacro RecallTree Induced Error
Turing0.3806190.3171330.2911220.3482493.80803
arthur_general0.3658260.2497360.2367020.264293.96749
jhuang0.3550750.2647020.2538490.2765254.27193
XipengQiu0.3473340.2851960.2579330.3189034.32142
logicators0.3094320.2553580.2301790.2867224.52007
NakaCristo0.2988530.2092760.198470.2213274.66694
brouardc0.2491690.161090.1322660.2059784.99114

Expensive

NameAccuracyMacro F-measureMacro PrecisionMacro RecallTree Induced Error
jhuang0.4678610.3595570.337240.3850363.2621
XipengQiu0.4517490.3584430.3335440.3873583.32434
Turing0.4400520.3380760.318530.3601793.33334
logicators0.421990.3380990.3068640.3764123.47428
NakaCristo0.4038420.3005580.2816290.3222153.71141

Full

NameAccuracyMacro F-measureMacro PrecisionMacro RecallTree Induced Error
alpaca0.5047590.3768620.3588920.3967262.82101
jhuang0.5024370.3861950.3637180.4116332.97652
XipengQiu0.4800170.3842720.358030.4146653.09452
Turing0.4635890.3608990.340790.3835313.13053
zwner0.462930.3600880.3398460.3828933.12767
logicators0.4606080.3714050.3394610.4099853.16841
NakaCristo0.4311070.3229760.303370.345293.46666

LSHTC2

DMOZ

NameAccEBFEBPEBRLBMaFLBMaPLBMaRLBMiFLBMiPLBMiRMGIE
brouardc0.3884890.3890150.3900710.3884890.2648160.3510540.3036860.3885750.3900710.387092.82937
chrishan0.3866040.3871480.3882390.3866040.2266020.408180.2595810.386750.3882390.3852722.82327
coolvegpuff0.3857650.3862880.3873380.3857650.2060020.4198110.2155960.3858520.3873380.3843772.83089
XipengQiu0.3799540.3852190.3814790.3949180.2722810.3833910.3277640.3753720.3587810.3935723.32248
youdongmiao0.3783150.3882450.3798040.4093720.2745050.3690350.339790.3686460.3362090.408013.83157
arthur0.3706810.3711930.3722220.3706810.2079720.3695760.2362860.3707940.3722220.3693772.89059
cheshbon0.3467660.3472660.3482730.3467660.1311090.3852770.1361340.3469370.3482730.3456113.02742
utumno0.2836670.2840110.2847030.2836670.1682410.4388750.1634630.2836110.2847030.2825273.45621
anttip0.2688620.2709540.2698720.2741530.1106460.2887970.119120.2705610.2680470.2731243.50285
vaijanath0.2624140.2630990.2634910.2634140.1295970.2469350.1462560.2619250.2612460.2626063.47579
tsoumakas0.172420.1726640.1731580.172420.07856910.2178730.08010040.1724940.1731580.1718353.89842
MLKD@AUTH0.1629460.1704980.1635860.2054390.037530.02925920.08433760.05795350.03374860.20493829.0915
luisfurquim0.1545930.1548420.1553480.1545930.06721240.326240.06196110.1547520.1553480.154164.01016
billzwc0.1525460.1533890.1531590.1544920.09698250.2059160.1302340.1531660.1523060.1540374.02445
Knn Baseline0.107390.1075410.1078430.107390.0375960.1633630.04134490.1074290.1078430.1070184.28911
vuco0.08717220.08731610.08760540.08717220.02362970.3381760.02009880.08726930.08760540.08693584.41854
cttsai4.79557e-054.79557e-054.79557e-054.79557e-053.26808e-054.59019e-054.50384e-054.77717e-054.79557e-054.75891e-054.9664

Wiki small

NameAccEBFEBPEBRLBMaFLBMaPLBMaRLBMiFLBMiPLBMiRMGIE
arthur0.3739130.433220.4634250.4610230.2316670.3785530.2733970.390440.3814820.3998294.17094
chrishan0.3621040.4217580.4785960.4362250.2133810.3974270.2600980.3756240.3759540.3752944.36359
brouardc0.3535550.4189270.4743960.4360960.241520.3651770.2699110.3842450.3977480.3716294.07641
coolvegpuff0.3514530.4002520.4750490.3969850.1880380.4776910.1835370.3889390.444380.3457973.85817
cheshbon0.2790670.3250880.4301460.2900420.1080010.5609740.09973260.3039620.4187820.2385563.72619
anttip0.2522170.2904030.3725520.2641010.1201950.4033760.113350.2714220.3671760.215283.85877
Knn Baseline0.2491370.3175960.2829530.416390.1757920.2522060.2353990.297860.2508560.366545.70073

Wiki large

NameAccEBFEBPEBRLBMaFLBMaPLBMaRLBMiFLBMiPLBMiRMGIE
arthur0.3466840.4263580.4964130.4578430.186770.369870.2282360.3483530.354280.3426214.64739
chrishan0.3367030.4116010.4961750.415710.1833440.3866450.2275370.3345630.3697240.305514.39208
coolvegpuff0.2828580.348860.4431160.381050.1478330.3102670.1511970.2737320.2601220.2888456.17844
Knn Baseline0.2724380.3471610.3627650.3869410.1486020.3033480.1769780.3015730.3256110.2808394.28833
anttip0.1767980.2101440.3129980.1924740.0612740.2983510.06629640.1611840.287510.1119814.53489

Acronyms of the evaluation measures:
Acc = Accuracy
EBF = Example Based F1-measure
EBP = Example Based Precision
EBR = Example Based Recall
LBMaF = Label Based Macro F1-measure
LBMaP = Label Based Macro Precision
LBMaR = Label Based Macro Recall
LBMiF = Label Based Micro F1-measure
LBMiP = Label Based Micro Precision
LBMiR = Label Based Micro Recall
MGIE = Multi-label Graph Induced Error