plismbench.engine.evaluate module#

Compute robustness metrics: cosine similarity and top-k accuracies.

plismbench.engine.evaluate.compute_metrics_ab(fp_a: Path, fp_b: Path, tiles_subset_idx: ndarray, top_k: list[int], device: str, pickles_save_dir: Path, overwrite: bool) → list[float][source]#

Compute metrics between float16 features from slide a and slide b.

plismbench.engine.evaluate.compute_metrics(features_root_dir: Path, metrics_save_dir: Path, extractor: str, top_k: list[int] | None = None, n_tiles: int | None = None, device: str = 'gpu', workers: int = 4, overwrite: bool = False)[source]#

Compute robustness metrics and save it to disk.

Parameters:

  • features_root_dir (pathlib.Path) – The root folder where features will be stored. The final export directory is features_root_dir / extractor

  • metrics_save_dir (pathlib.Path) – Folder containing the output metrics. The final export directory is metrics_save_dir / extractor.

  • extractor (str) – The name of the feature extractor as defined in plismbench.models.__init__.py

  • top_k (list[int] | None = None) – Values of k for top-k accuracy computation.

  • n_tiles (int | None = None) – Number of tiles per slide for metrics computation.

  • device (str = "gpu") – Device on which matrix operations will be performed.

  • workers (int = 4) – Number of workers for cpu parallel computations if device='cpu'.

  • overwrite (bool = False) – Whether to overwrite existing metrics.