Scikit-Learn Workflow

If you install the scikit-learn optional dependency, you can build CompML score objects directly from scikit-learn estimators and datasets.

from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression

from compml.frequentist.tests import CorrelatedTTest
from compml.sklearn_utils import run_cross_val

# X and y come from your dataset
datasets = ((X, y),)

cv_scores = run_cross_val(
    estimators=(
        DummyClassifier(strategy="most_frequent"),
        LogisticRegression(),
    ),
    algorithm_names=("dummy", "logreg"),
    datasets=datasets,
    dataset_names=("my_dataset",),
    scoring="accuracy",
    cv=5,
)

result = CorrelatedTTest(alpha=0.05).run(cv_scores)
print(result.summary())

run_hold_out is also available for hold-out workflows. See compml.sklearn_utils.