mpcompress.metrics

DataFrameRecords

DataFrameRecords()

Record storage and averaging using pandas DataFrame.

This class stores records in a pandas DataFrame with _id as the index, allowing records to be updated by their _id. It supports computing averages across all stored records.

Example:

Input record: {"_id": id1, "key1": value1, "key2": value2, ...}
Average record: {"key1": avg_value1, "key2": avg_value2, ...}

The record can be updated by _id. Implemented using pandas DataFrame, supporting row updates by _id.

Creates an empty DataFrame for storing records. The _id field will be used as the DataFrame index.

len

__len__()

Get the number of records.

Returns:

Name	Type	Description
`length`	`int`	Number of records stored in the DataFrame.

average

average()

Compute the average of all numeric columns across all records.

Returns:

Name	Type	Description
`average`	`dict`	Dictionary mapping column names to their mean values. Only numeric columns are included in the result.

get_record_by_id

get_record_by_id(_id)

Retrieve a record by its _id.

Parameters:

Name	Type	Description	Default
`_id`	`str`	The identifier of the record to retrieve.	required

Returns:

Name	Type	Description
`record`	`dict`	Dictionary representation of the record with the given _id.

Raises:

Type	Description
`KeyError`	If the _id does not exist in the DataFrame.

update

update(record)

Update or insert a record by _id.

If a record with the given _id exists, it will be updated with new values. Otherwise, a new record will be inserted. The _id field is used as the DataFrame index and is removed from the record data.

Parameters:

Name	Type	Description	Default
`record`	`dict`	Dictionary containing "_id" and other key-value pairs. The "_id" field is required and will be used as the DataFrame index.	required

Raises:

Type	Description
`AssertionError`	If the record does not contain "_id" key.
`KeyError`	If the _id does not exist in the DataFrame.

DictAverageMeter

DictAverageMeter()

Averaging meter for dictionary records.

This class accumulates numeric values from dictionary records and computes their averages. It filters out non-numeric values and handles various numeric types including scalars, single-element tensors, and objects with arithmetic operations.

Example:

Input record: {"key1": value1, "key2": value2, ...}
Average record: {"key1": avg_value1, "key2": avg_value2, ...}

Creates an empty meter with no accumulated values.

average

average()

Compute the average of all accumulated values.

Returns:

Name	Type	Description
`average`	`dict`	Dictionary mapping keys to their average values. Each value is computed as: accumulated_value / count

Raises:

Type	Description
`AttributeError`	If no records have been updated yet (meter is None).

update

update(record, n=1)

Update the meter with a new record.

Parameters:

Name	Type	Description	Default
`record`	`dict`	Dictionary of key-value pairs to accumulate. Non-numeric values are filtered out.	required
`n`	`int`	Number of samples this record represents. Used for weighted averaging. Defaults to 1.	`1`

MeanIoUMetric

MeanIoUMetric(num_classes: int = 21)

Metric for computing mean Intersection over Union (mIoU).

This metric computes the mean IoU across all classes for semantic segmentation tasks. It accumulates a confusion matrix across batches and computes the per-class IoU, then takes the mean.

Parameters:

Name	Type	Description	Default
`num_classes`	`int`	Number of classes in the segmentation task. Defaults to 21.	`21`

compute

compute()

Compute the mean IoU metric.

Returns:

Name	Type	Description
`average`	`dict`	Dictionary containing: "mIoU" (float): Mean Intersection over Union across all classes.

fast_hist

fast_hist(label, prediction, n)

Compute confusion matrix efficiently.

Parameters:

Name	Type	Description	Default
`label`	`ndarray`	Ground truth class indices.	required
`prediction`	`ndarray`	Predicted class indices. Must match label shape.	required
`n`	`int`	Number of classes.	required

Returns:

Name	Type	Description
`hist`	`ndarray`	Confusion matrix of shape [n, n] where entry [i, j] is the count of pixels with true class i and predicted class j.

per_class_iou

per_class_iou(hist)

Compute Intersection over Union for each class.

Parameters:

Name	Type	Description	Default
`hist`	`ndarray`	Confusion matrix of shape [n, n].	required

Returns:

Name	Type	Description
`iou`	`ndarray`	Per-class IoU values of shape [n]. IoU for class i is: IoU_i = hist[i, i] / (sum(hist[i, :]) + sum(hist[:, i]) - hist[i, i])

update

update(preds, target)

Update the metric with a batch of predictions and targets.

Parameters:

Name	Type	Description	Default
`preds`	`Tensor or ndarray`	Predicted class indices. Shape: [B, H, W] or [H, W].	required
`target`	`Tensor or ndarray`	Ground truth class indices. Shape: [B, H, W] or [H, W]. Must match preds shape.	required

TopKAccuracyMetric

TopKAccuracyMetric(topk: List[int] = [1, 5])

Metric for computing top-K accuracy.

This metric tracks the accuracy of predictions where the correct class appears in the top-K predicted classes. It accumulates results across multiple batches and computes the final accuracy percentage.

Parameters:

Name	Type	Description	Default
`topk`	`List[int]`	List of K values to compute accuracy for. The list will be sorted in ascending order. Defaults to [1, 5].	`[1, 5]`

compute

compute() -> Dict[str, float]

Compute the top-K accuracy metrics.

Returns:

Type	Description
`Dict[str, float]`	Dict[str, float]: Dictionary mapping metric names to accuracy percentages. Keys are in the format "top-{k}" (e.g., "top-1", "top-5"). Returns zeros if no samples have been processed.

update

update(predictions: List[List[int]], targets: List[int])

Update the metric with a batch of predictions and targets.

Parameters:

Name	Type	Description	Default
`predictions`	`List[List[int]]`	List of prediction lists, where each inner list contains class indices sorted by confidence (highest first).	required
`targets`	`List[int]`	List of ground truth class indices.	required

create_dist_metrics

create_dist_metrics(metric_names: Union[str, list] = None) -> Dict[str, Callable]

Create distribution-distance metrics dictionary with per-metric lazy cache.

Distribution-distance metrics measure the distance between distributions of image features rather than pixel-level or perceptual similarity.

Supported metrics

FID: Fréchet Inception Distance

Parameters:

Name	Type	Description	Default
`metric_names`	`Union[str, list]`	Name(s) of metrics to create. If None, all available metrics are created. If a string, a single metric is created. If a list, multiple metrics are created. Defaults to None.	`None`

Returns:

Type	Description
`Dict[str, Callable]`	Dict[str, Callable]: Dictionary mapping metric names to their callable functions. Note: Distribution metrics typically require batches of images rather than single image pairs.

Raises:

Type	Description
`ValueError`	If any requested metric name is not supported.

create_img_metrics

create_img_metrics(metric_names: Union[str, list] = None) -> Dict[str, Callable]

Create image quality assessment (IQA) metrics dictionary.

Heavy metrics are instantiated lazily per metric. This function uses a per-metric function cache so repeated requests for the same metric name reuse the cached callable.

Supported metrics

PSNR: Peak Signal-to-Noise Ratio
MS-SSIM: Multi-Scale Structural Similarity Index
MS-SSIM-dB: MS-SSIM in decibel scale
VIF: Visual Information Fidelity
GMSD: Gradient Magnitude Similarity Deviation
LPIPS-Alex: Learned Perceptual Image Patch Similarity (AlexNet backbone)
LPIPS-VGG: Learned Perceptual Image Patch Similarity (VGG backbone)
DISTS: Deep Image Structure and Texture Similarity
PieAPP: Perceptual Image-Error Assessment through Pairwise Preference
AHIQ: Attention-based Hybrid Image Quality
CLIP-SIM: CLIP-based image similarity
TOPIQ-FR: TopIQ Full Reference
TOPIQ-NR: TopIQ No Reference
MUSIQ: Multi-scale Image Quality Transformer

Parameters:

Name	Type	Description	Default
`metric_names`	`Union[str, list]`	Name(s) of metrics to create. If None, all available metrics are created. If a string, a single metric is created. If a list, multiple metrics are created. Defaults to None.	`None`

Returns:

Type	Description
`Dict[str, Callable]`	Dict[str, Callable]: Dictionary mapping metric names to their callable functions. Each callable takes two arguments (img1, img2) and returns a similarity score.

Raises:

Type	Description
`ValueError`	If any requested metric name is not supported.