Summary
Data management
Debian Science Data Management packages
This metapackage will install packages to assist with data management
tasks, such as obtaining data from remote resources, keeping data
under version control, etc.
Description
For a better overview of the project's availability as a Debian package, each head row has a color code according to this scheme:
If you discover a project which looks like a good candidate for Debian Science
to you, or if you have prepared an unofficial Debian package, please do not hesitate to
send a description of that project to the Debian Science mailing list
Links to other tasks
|
Debian Science Data management packages
Official Debian packages with high relevance
Datalad
data files management and distribution platform
|
Versions of package datalad |
Release | Version | Architectures |
bullseye | 0.14.0-1 | all |
bookworm | 0.18.1-2 | all |
sid | 0.18.1-2 | all |
stretch | 0.4.1-1 | all |
buster | 0.11.2-2 | all |
|
License: DFSG free
|
DataLad is a data management and distribution platform providing
access to a wide range of data resources already available online.
Using git-annex as its backend for data logistics it provides following
facilities built-in or available through additional extensions
-
command line and Python interfaces for manipulation of collections of
datasets (install, uninstall, update, publish, save, etc.) and
separate files/directories (add, get)
-
extract, aggregate, and search through various sources of metadata
(xmp, EXIF, etc; install datalad-neuroimaging for DICOM, BIDS, NIfTI
support)
- crawl web sites to automatically prepare and update git-annex
repositories with content from online websites, S3, etc (install
datalad-crawler)
|
|
Datalad-container
DataLad extension for working with containerized environments
|
Versions of package datalad-container |
Release | Version | Architectures |
bookworm | 1.1.7-1 | all |
sid | 1.1.9-1 | all |
buster | 0.2.2-2 | all |
bullseye | 1.1.2-1 | all |
|
License: DFSG free
|
This extension enhances DataLad (http://datalad.org) for working with
computational containers.
|
|
Git-annex
manage files with git, without checking their contents into git
|
Versions of package git-annex |
Release | Version | Architectures |
sid | 10.20221003-3 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
buster | 7.20190129-3 | amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x |
buster-backports | 8.20200330-1~bpo10+1 | amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x |
bullseye | 8.20210223-2 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
bookworm | 10.20221003-3 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
jessie | 5.20141125+deb8u1 | amd64,armel,armhf,i386 |
jessie-security | 5.20141125+oops-1+deb8u2 | amd64,armel,armhf,i386 |
stretch-security | 6.20170101-1+deb9u1 | amd64,i386 |
stretch | 6.20170101-1+deb9u2 | amd64,arm64,i386,mips,mips64el,mipsel,ppc64el,s390x |
stretch-backports | 6.20180913-1~bpo9+1 | mipsel |
stretch-backports | 7.20181211-2~bpo9+1 | mips |
stretch-backports | 7.20190129-2~bpo9+1 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x |
Debtags of package git-annex: |
devel | rcs |
role | program |
works-with | file |
|
License: DFSG free
|
git-annex allows managing large files with git, without storing the file
contents in git. It can sync, backup, and archive your data, offline
and online. Checksums and encryption keep your data safe and secure. Bring
the power and distributed nature of git to bear on your large files with
git-annex.
It can store large files in many places, from local hard drives, to a
large number of cloud storage services, including S3, WebDAV,
and rsync, with dozens of cloud storage providers usable via plugins.
Files can be stored encrypted with gpg, so that the cloud storage
provider cannot see your data. git-annex keeps track of where each file
is stored, so it knows how many copies are available, and has many
facilities to ensure your data is preserved.
git-annex can also be used to keep a folder in sync between computers,
noticing when files are changed, and automatically committing them
to git and transferring them to other computers. The git-annex webapp
makes it easy to set up and use git-annex this way.
|
|
Hdf5-filter-plugin
external filters for HDF5: LZ4, BZip2, Bitshuffle
|
Versions of package hdf5-filter-plugin |
Release | Version | Architectures |
sid | 0.0~git20221111.49e3b65-4 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
bookworm | 0.0~git20221111.49e3b65-4 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
|
License: DFSG free
|
The external filter mechanism introduced with HDF5 1.8.12 allows
applications to utilize custom filters not shipped by the HDF5
core library without recompiling your application. This package
provides external filters for HDF5 for
- the LZ4 compression algorithm
- BZip2 compression
|
|
Hdf5-filter-plugin-zfp-serial
Compression plugin for the HDF5 library using ZFP compression
|
Versions of package hdf5-filter-plugin-zfp-serial |
Release | Version | Architectures |
sid | 1.1.0+git20221021-4 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el |
bookworm | 1.1.0+git20221021-4 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el |
|
License: DFSG free
|
H5Z-ZFP is a compression filter for HDF5 using the ZFP compression library,
supporting lossy and lossless compression of floating point and integer data
to meet bitrate, accuracy, and/or precision targets.
|
|
|