er_evaluation.metrics#

Evaluation Metrics (Precision, Recall, B-cubed, etc)#

The metrics module provides a set of functions to compute common comparison metrics for entity resolution algorithms. These compare a predicted clustering to a reference clustering. The available metrics are:

  • pairwise precision: the proportion of predicted links that also appear in the reference clustering.

  • pairwise recall: the proportion of links in the reference clustering that also appear in the predicted clustering.

  • cluster precision: the proportion of predicted clusters that are present in the reference clustering.

  • cluster recall: the proportion of reference clusters that are also present in the predicted clustering.

  • F-scores: harmonic mean of precision and recall

  • B-cubed metrics: These are defined as a weighted average of mention-level measures of precision and recall.

Notes:

  • When computing these metrics, all functions in this module first compute the inner join of the reference and predicted clusterings. As such, only records that appear in both clusterings are accounted for by these metrics. All other records are dropped.

  • Records with NA cluster identifier in the reference or predicted clusterings are dropped.

  • The metrics in this module do not provide representative performance estimates. They are only useful for comparing two clusterings, such as a. For representative performance estimates, see the er_evaluation.estimators module.

Functions#

adjusted_rand_score(prediction, reference)

Compute the adjusted Rand index.

b_cubed_f(prediction, reference[, beta])

B-cubed F score for the inner join of two clusterings.

b_cubed_precision(prediction, reference)

B-cubed precision for the inner join of two clusterings, with equal weight placed on each ground truth cluster.

b_cubed_recall(prediction, reference)

B-cubed recall for the inner join of two clusterings, with equal weight placed on each ground truth cluster.

cluster_completeness(prediction, reference)

Cluster completeness score (based on conditional entropy)

cluster_f(prediction, reference[, beta])

Cluster F score for the inner join of two clusterings.

cluster_homogeneity(prediction, reference)

Cluster homogeneity score (based on conditional entropy).

cluster_precision(prediction, reference)

Cluster precision for the inner join of two clusterings.

cluster_recall(prediction, reference)

Cluster recall for the inner join of two clusterings.

cluster_v_measure(prediction, reference[, beta])

Compute the V-measure.

metrics_table(predictions, references[, metrics])

Apply a set of metrics to all combinations of prediction and reference membership vectors.

pairwise_f(prediction, reference[, beta])

Pairwise F score for the inner join of two clusterings.

pairwise_precision(prediction, reference)

Pairwise precision for the inner join of two clusterings.

pairwise_recall(prediction, reference)

Pairwise recall for the inner join of two clusterings.

rand_score(prediction, reference)

Compute the Rand index.