MELITE¶
Supervised learning evaluation
Multi-model supervised classification and regression evaluator on tabular data.
Pre-stable
MELITE is currently in alpha-stage development.
Why MELITE?¶
Comparing supervised models is easy. Keeping the comparison methodologically sound and reconstructable is harder.
You tune several models, compare their scores, and choose one. Later comes the question: why this model? Answering it depends on two things that are easy to lose — whether tuning stayed clear of the evidence used to report performance, and whether the fold-level results still exist.
MELITE turns that decision into a recorded evaluation workflow. Hyperparameter search stays inside the training portion of each outer split, while performance is measured on held-out outer folds. Model selection uses a task-specific primary metric, and both aggregate and fold-level evidence are persisted. For tunable models, search effort is controlled by an explicitly declared optimization budget, making that effort part of the recorded evaluation design.
For classification, MELITE determines binary or multiclass evaluation from the number of distinct target labels and preserves those original identities. F1-macro remains the optimization and selection metric in both cases. Binary ROC-AUC retains its established definition; multiclass ROC-AUC uses probability-based One-vs-One macro averaging.
Selection and final fitting are deliberately separate. Outer-CV evidence identifies the winning model; the final artifact is then fitted on all available data, with a final full-data search when the model is tunable. The outer-CV score estimates performance; the exported artifact is the deliverable. MELITE reports them as distinct.
You can assemble the same procedure manually with scikit-learn and the underlying estimator libraries. MELITE makes it explicit, repeatable, and inspectable — and keeps the evidence.
MELITE records software and model-evaluation evidence. It does not by itself establish external scientific validity or predictive generalizability.
Machine-Learning Methods Implemented in MELITE¶

What You Provide and Receive¶
MELITE sits between prepared tabular datasets and the fitted model you ultimately export. Numeric feature matrices remain the general contract; CatBoost-only CSV classification may use declared categorical or mixed input. The following contract describes the classification path:
| Stage | You provide | MELITE does | You receive |
|---|---|---|---|
| Input | Numeric feature matrices (X) and target labels (y), or a CSV classification dataset with explicit CatBoost categorical columns. |
Validates each registered dataset and treats it as an independent evaluation unit. | Validated datasets, ready for evaluation. |
| Configuration | Active classifiers, cross-validation settings, dataset metadata, and output paths. | Resolves the evaluation design before classifier fitting begins. | A validated evaluation setup. |
| Evaluation | Nothing further — the evaluation design is already fixed. | Evaluates each active classifier under the same outer cross-validation design. Tunable classifiers perform hyperparameter search only within the training portion of each outer split. Per-class metrics use only held-out outer-fold predictions. | evaluations.csv, aggregate evidence; evaluation_folds.csv, outer-fold evidence; evaluation_class_folds.csv and evaluation_class_summary.csv, class-level evidence; optimization_searches.csv, completed-search evidence; and the optimization and classification metric provenance JSON files. |
| Selection | The evaluation evidence produced by the run. | Selects the classifier with the highest mean outer-CV F1-macro for each dataset. | results.csv, the selected classifier result for each dataset, and evaluation_f1_macro_<dataset>.png, the visual evidence behind the selection. |
| Export | A request to export a selected result. | Fits the selected classifier on all available data using the classifier parameters persisted in results.csv by melite run; no additional hyperparameter search, cross-validation, or classifier selection is performed. Original binary or multiclass label identities are retained. |
Model_<classifier>_<dataset>.pkl, a fitted model artifact distinct from the estimators used to obtain the outer-CV evidence. Its classes_ order defines probability columns during inference. |
Citation¶
Contreras-Torres, F. F., & Murrieta, A. C. (2026). MELITE — Multi-Model Supervised Learning Evaluator. Zenodo. https://doi.org/10.5281/zenodo.20382752
Use CITATION.cff as the authoritative machine-readable citation metadata for MELITE. Citation metadata is updated with each release.
License¶
MELITE is licensed under the
GNU Lesser General Public License v3.0 or later. SPDX identifier: LGPL-3.0-or-later.