|
 
|
[SNa06] Model assessment
Conférence Internationale avec comité de lecture : KNEMO'06 Knowledge Extraction and Modeling, Capri, January 2006,
motcle:
Résumé:
In data mining and machine learning, models come from data and provide insights for understanding data (unsupervised classification) or making prediction (supervised learning).
Thus the scientific status of this kind of models is different from the classical view where a model is a simplified representation of reality provided by an expert of the field . In most data mining applications a good model is a model which not only fits the data but gives good predictions, even if it is not interpretable .
In this context, model validation and model choice need specific indices and approaches. Penalized likelihood measures (AIC, BIC etc.) may not be pertinent when there is no simple distributional assumption on the data and (or) for models like regularized regression, SVM and many others where parameters are constrained.
Complexity measures like the VC-dimension are more adapted, but very difficult to estimate.
In supervised classification, ROC and AUC measures are commonly used . Comparing models should be done on validation (hold-out) sets but resampling is necessary in order to get confidence intervals
Commentaires:
IASC-INTERFACE-IFCS Workshop
BibTeX
| @inproceedings { | |||
| SNa06, | |||
| title | = | "{Model assessment}", | |
| author | = | " G. Saporta and N. Niang Keita ", | |
| booktitle | = | "{KNEMO'06 Knowledge Extraction and Modeling, Capri}", | |
| year | = | 2006, | |
| month | = | "January", | |
| note | = | "{IASC-INTERFACE-IFCS Workshop}", | |
| } | |||
