Changelog#
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
[Unreleased]#
Fixed#
spatialrefinery.io.xenium:*_he_alignment.csv– the name the Atera “WTA Preview” bundles use instead of 10x’s*_he_imagealignment.csv– was missing from the asset-kind map, so those files classified as"unknown"andkinds=["he_alignment"]silently skipped them.find_xenium_filesalready recognised both spellings.spatialrefinery.core.converter.downsample_plane: halving a single-channel"minisblack"plane silently swapped height and width from the second sub-level onward.cv2.resizedrops a trailing size-1 channel axis ((H, W, 1) -> (H, W)), and the subsequentmoveaxis(img, -1, 0)then transposed that now-2D result instead of restoring the channel axis it had removed. Invisible until now because the one existing"minisblack"caller,BioioImageConverter, alwaysnp.squeezes a single channel down to plain(H, W)before it reachesdownsample_plane.
Added#
spatialrefinery.core: a technology/converter registry (registry), shared spatial-omics helpers (utils), an image-to-pyramidal-OME-TIFF converter (converter, with an optionalcziextra for Zeiss CZI viabioio), and a retrying, atomic-write asset downloader (downloader).spatialrefinery.io.xenium: convert 10x Genomics Xenium bundles to SpatialData zarr stores, optionally with aligned H&E images, tissue segmentation, and Visium-like pseudo-spots; download a Xenium study’s raw assets from acurl -O <url>manifest.spatialrefinery.io.xenium: Xenium protein sub-panels keep their antibody measurements. The cell-feature matrix is now read withgex_only=Falseand the table is split by feature type:varis restricted to the gene panel, and anyProtein Expressionrows move intotable.obsm["protein_expression"]– a cells x antibodies DataFrame columned by antibody name, withuns["protein_expression"]recording theirnames,gene_idsandmetric. Antibodies cannot stay invar: one can carry the same name as a gene targeting the same molecule, which makesvar_namesnon-unique,table[:, name]ambiguous, and would havecreate_pseudo_spotsemit every such gene twice into the spot table. Their values areMEAN_PER_CELL_STAINintensities, not counts, so a mixed table also madeX.sum(axis=1)meaningless against the transcript-onlyobs["total_counts"]. Control and codeword feature types are dropped, as before, their per-cell totals already being inobs. Samples without a protein sub-panel are unaffected – their table is bit-identical to what the previousgex_only=Trueread produced, and gains noobsmentry – and panel sizes are read off the data, so nothing assumes a particular gene count. Pseudo-spot tables carry no protein channel: proteins are measured per cell by antibody stain, with nothing intranscriptsto aggregate. A Xenium protein store used as ageojson_to_spatialdatatemplate therefore contributes a gene-panel-onlyvar.spatialrefinery.io.visium: download a 10x Visium / CytAssist study’s raw assets from acurl -O <url>manifest, unpacking thespatial,analysisanddeconvolutiontarballs in place. The two*_feature_bc_matrix.tar.gzarchives are left packed, being MTX copies of the.h5files fetched alongside. Each study is verified on disk afterwards reporting missing required assets and whether a.cloupewas found rather than failing a batch on one incomplete study.BaseDownloader.verify_bundleandBundleCheck: per-study bundle verification driven by four class-level tuples (required_kinds,expected_kinds,required_members,expected_members), logged from apost_processthat is no longer a no-op. It reports asset kinds and extracted members separately, because a missing kind means an asset was never downloaded whereas a missing member means an archive did not unpack – a failure invisible in the download results, since the bytes arrived intact. Xenium verifies the six membersspatialdata_io.xeniumopens, and flags an H&E that has no alignment CSV: it converts without error but lands on an Identity transform, silently unaligned, and 3 of 65 sample bundles are in that state.spatialrefinery.core.utils.collapse_url_slashes, applied inRemoteAsset.from_url: some published 10x manifests carry a doubled path separator (.../spatial-exp/3.1.3//<study>/...), which the CDN answers with403 Forbidden. It was invisible before because the study name still parsed correctly, so all 8 assets of one Visium study failed as an apparent permissions error.spatialrefinery.core.utils.safe_extract_tarandtar_root_dir, andBaseDownloader.extract_archive/require_extract: tar archives are now recognised alongside zips (Path.suffixreports.gzfora.tar.gz, so tarballs were previously never unpacked), a failed extraction of a load-bearing archive fails its asset instead of only warning, and a flat archive is given a directory of its own so concurrent extractions cannot overwrite each other.spatialrefinery.segmentation: nucleus segmentation on H&E whole-slide images via InstanSeg (instanseg), and export of the resulting boundaries as a SpatialData zarr store carrying the slide image, the nucleus polygons, and a table over a template gene panel (to_spatialdata). Needs the optionalsegmentationextra. The store is named for the slide’s stem, soslide.ome.tifyieldsslide.zarr.Tutorial notebook for the segmentation pipeline, taking an OME-TIFF through segmentation to a written SpatialData zarr.
spatialrefinery.core.converter: read Olympus cellSens (.vsi), PerkinElmer/Akoya QPTIFF (.qptiff), and Zeiss ZVI/AFI (.zvi,.afi) whole-slide/microscopy images viaslideio, joiningopenslideandbioioas a thirdImageConverterbackend (SlideioImageConverter). Streams level 0 the same wayOpenSlideImageConverterdoes – full-width bands halved on the way past into an on-disk memmap – via a newSlideioTiledSource, so a whole-slide plane is never materialised in full. A multichannel (non-RGB) scene is written as one page per channel, whichtifffilerequires its tile iterator to fill in page-major order (every tile of channel 0 before any of channel 1);SlideioTiledSourcereads and stages one channel at a time to match.slideiois a required dependency, not an optional extra: its wheels arenumpy-only and BSD-3-Clause, matching this project’s license, though it currently ships none formanylinuxaarch64 or glibc < 2.28. DICOM WSI (.dcm) is deliberately not registered: it is normally a directory of instances, which suffix-based dispatch does not address, and it has not been exercised against a real file.
Changed#
spatialrefinery.core.converter.OpenSlideImageConverterand the newSlideioImageConverternow raise if the source reports no physical pixel size (mpp-x/mpp-y, orslideio’s exactly-1.0-metre/pixel no-metadata fallback), instead of silently writing the slide at an assumed 1.0 um/px. A slide converted at the wrong scale is exactly the “silent bad sample” this project’sCLAUDE.mdsays must not happen – every downstream pseudo-spot size and Phoenix training target derives from it. Both converters, andconvert_to_ometiff, take anmpp=override (a single value, or an(x, y)pair, in micrometres) for a source that genuinely carries none;scripts/convert_to_ometiff.pyexposes it as--mpp.