estimates_table#

er_evaluation.estimates_table(predictions, samples_weights, estimators={'b_cubed_precision': <function b_cubed_precision_estimator>, 'b_cubed_recall': <function b_cubed_recall_estimator>, 'cluster_f': <function cluster_f_estimator>, 'cluster_precision': <function cluster_precision_estimator>, 'cluster_recall': <function cluster_recall_estimator>, 'pairwise_f': <function pairwise_f_estimator>, 'pairwise_precision': <function pairwise_precision_estimator>, 'pairwise_recall': <function pairwise_recall_estimator>})[source]#

Create table of estimates applied to all combinations of predictions and (sample, weights) pairs.

Parameters:

  • predictions (Dict) – Dictionary of membership vectors.

  • samples_weights (Dict) – Dictionary of dictionaries of the form {“sample”: sample, “weights”: weights}, where sample is the sample membership vector and weights is the pandas Series of sampling weights. See estimators definitions for more information.

  • estimators (Dict) – Dictionary of estimator functions. Each estimator is expected to return a pair (estimate, std).

Returns:

Pandas DataFrame with columns “predition”, “sample_weights”, “estimator”, “value”, and “std”, where value and std are the point estimate and standard deviation estimate for the estimator applied to the given prediction, sample and sampling weights.

Return type:

DataFrame

Examples

>>> import pandas as pd
>>> from er_evaluation.estimators import *
>>> predictions = {"prediction_1": pd.Series(index=[1,2,3,4,5,6,7,8], data=[1,1,2,3,2,4,4,4])}
>>> samples_weights = {"sample_1": {"sample": pd.Series(index=[1,2,3,4,5,8], data=["c1", "c1", "c1", "c2", "c2", "c4"]), "weights": pd.Series(1, index=["c1", "c2", "c4"])}}
>>> estimators = {"precision": pairwise_precision_estimator, "recall": pairwise_recall_estimator}
>>> estimates_table(predictions, samples_weights, estimators) 
    prediction      sample_weights      estimator       value       std
0       prediction_1    sample_1            precision   0.388889        0.254588
1       prediction_1    sample_1            recall          0.296875    0.108253