Summary
Data Management
Debian Science Data Management packages
This metapackage will install packages to assist with data management
tasks, such as obtaining data from remote resources, keeping data
under version control, etc.
Description
For a better overview of the project's availability as a Debian package, each head row has a color code according to this scheme:
If you discover a project which looks like a good candidate for Debian Science
to you, or if you have prepared an unofficial Debian package, please do not hesitate to
send a description of that project to the Debian Science mailing list
Links to other tasks
|
Debian Science Data Management packages
Official Debian packages with high relevance
datalad
data files management and distribution platform
|
Versions of package datalad |
Release | Version | Architectures |
sid | 0.19.6-2 | all |
stretch | 0.4.1-1 | all |
trixie | 0.19.6-2 | all |
bookworm | 0.18.1-2 | all |
bullseye | 0.14.0-1 | all |
buster | 0.11.2-2 | all |
upstream | 1.0.2 |
|
License: DFSG free
|
DataLad is a data management and distribution platform providing
access to a wide range of data resources already available online.
Using git-annex as its backend for data logistics it provides following
facilities built-in or available through additional extensions
- command line and Python interfaces for manipulation of collections of
datasets (install, uninstall, update, publish, save, etc.) and
separate files/directories (add, get)
- extract, aggregate, and search through various sources of metadata
(xmp, EXIF, etc; install datalad-neuroimaging for DICOM, BIDS, NIfTI
support)
- crawl web sites to automatically prepare and update git-annex
repositories with content from online websites, S3, etc (install
datalad-crawler)
|
|
datalad-container
DataLad extension for working with containerized environments
|
Versions of package datalad-container |
Release | Version | Architectures |
sid | 1.2.5-1 | all |
buster | 0.2.2-2 | all |
bullseye | 1.1.2-1 | all |
bookworm | 1.1.9-1 | all |
trixie | 1.2.5-1 | all |
|
License: DFSG free
|
This extension enhances DataLad (http://datalad.org) for working with
computational containers.
|
|
git-annex
gestion de fichiers avec git, sans vérification de leur contenu dans git
|
Versions of package git-annex |
Release | Version | Architectures |
sid | 10.20240129-1 | amd64,arm64,i386,mips64el,ppc64el,riscv64,s390x |
bullseye | 8.20210223-2 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
buster-backports | 8.20200330-1~bpo10+1 | amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x |
buster | 7.20190129-3 | amd64,arm64,armhf,i386 |
stretch-backports | 7.20190129-2~bpo9+1 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x |
stretch-backports | 7.20181211-2~bpo9+1 | mips |
stretch-backports | 6.20180913-1~bpo9+1 | mipsel |
stretch | 6.20170101-1+deb9u2 | amd64,arm64,i386,mips,mips64el,mipsel,ppc64el,s390x |
stretch-security | 6.20170101-1+deb9u1 | amd64,i386 |
jessie-security | 5.20141125+oops-1+deb8u2 | amd64,armel,armhf,i386 |
jessie | 5.20141125+deb8u1 | amd64,armel,armhf,i386 |
trixie | 10.20240129-1 | amd64,arm64,i386,mips64el,ppc64el,s390x |
bookworm-backports | 10.20240129-1~bpo12+1 | amd64,arm64,armel,i386,mips64el,mipsel,ppc64el,s390x |
bookworm | 10.20230126-3 | amd64,arm64,i386,mips64el,mipsel,ppc64el,s390x |
Debtags of package git-annex: |
devel | rcs |
role | program |
works-with | file |
|
License: DFSG free
|
Git-annex permet la gestion de grands fichiers avec git, sans stocker leur
contenu dans git. Il peut synchroniser, restaurer et archiver les données
en ligne ou hors ligne. Les sommes de contrôle et le chiffrement rendent
les données sécurisées. Utilisez la puissance et la nature distribuée de
git pour prendre en charge de grands fichiers avec git-annex.
Il peut stocker de gros fichiers à divers endroits, depuis les disques
durs locaux à un grand nombre de services de stockage en ligne, y compris
S3, WebDAV ou rsync et des douzaines de fournisseurs de stockage en ligne
utilisables à partir de greffons. Les fichiers peuvent être stockés
chiffrés avec gpg, de telle sorte que le fournisseur de stockage en ligne
ne puisse pas voir les données. Git-annex conserve la trace de l'endroit où
est stocké chaque fichier, afin de savoir combien de copies sont
disponibles, et possède de nombreuses fonctionnalités pour assurer la
préservation des données
Git-annex peut aussi être utilisé pour assurer la synchronisation d'un
dossier entre plusieurs ordinateurs, en détectant les modifications de
fichier et en les transmettant automatiquement à git pour transfert aux
autres ordinateurs. L'application web de git-annex facilite la
configuration et l'utilisation de git-annex à cette fin.
|
|
hdf5-filter-plugin
external filters for HDF5: LZ4, BZip2, Bitshuffle
|
Versions of package hdf5-filter-plugin |
Release | Version | Architectures |
bookworm | 0.0~git20221111.49e3b65-4 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
trixie | 0.0~git20221111.49e3b65-4 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x |
sid | 0.0~git20221111.49e3b65-4 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x |
|
License: DFSG free
|
The external filter mechanism introduced with HDF5 1.8.12 allows
applications to utilize custom filters not shipped by the HDF5
core library without recompiling your application. This package
provides external filters for HDF5 for
- the LZ4 compression algorithm
- BZip2 compression
|
|
hdf5-filter-plugin-blosc-serial
blocking, shuffling and lossless compression library
|
Versions of package hdf5-filter-plugin-blosc-serial |
Release | Version | Architectures |
sid | 0.0~git20220616.9683f7d-5 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x |
trixie | 0.0~git20220616.9683f7d-5 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x |
bookworm | 0.0~git20220616.9683f7d-5 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
|
License: DFSG free
|
This package contains a filter for HDF5 that uses the Blosc compressor.
By installing this filter, you can read and write HDF5 files with
Blosc-compressed datasets.
|
|
hdf5-filter-plugin-zfp-serial
Compression plugin for the HDF5 library using ZFP compression
|
Versions of package hdf5-filter-plugin-zfp-serial |
Release | Version | Architectures |
sid | 1.1.1-2 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64 |
bookworm | 1.1.0+git20221021-4 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el |
experimental | 1.1.0+git20230428-0+exp2 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x |
trixie | 1.1.1-2 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el |
|
License: DFSG free
|
H5Z-ZFP is a compression filter for HDF5 using the ZFP compression library,
supporting lossy and lossless compression of floating point and integer data
to meet bitrate, accuracy, and/or precision targets.
|
|
nexus-tools
NeXus scientific data file format - applications
|
Versions of package nexus-tools |
Release | Version | Architectures |
trixie | 4.4.3-6 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x |
bookworm | 4.4.3-5 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
jessie | 4.3.2-svn1921-2 | amd64,armel,armhf,i386 |
bullseye | 4.4.3-5 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
sid | 4.4.3-6 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x |
|
License: DFSG free
|
NeXus is a common data format for neutron, X-ray, and muon science. It
is being developed as an international standard by scientists and
programmers representing major scientific facilities in Europe, Asia,
Australia, and North America in order to facilitate greater cooperation
in the analysis and visualization of neutron, X-ray, and muon data.
This is the package containing some applications for reading and writing
NeXus files.
|
|
plfit
fitting power-law distributions to empirical data -- interfaces
|
Versions of package plfit |
Release | Version | Architectures |
sid | 0.9.6+ds-1 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x |
bookworm | 0.9.4+ds-1 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el |
trixie | 0.9.4+ds-1 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el |
|
License: DFSG free
|
The plfit software fits power-law distributions to empirical (discrete or
continuous) data, according to the method of Clauset, Shalizi and Newman
[SIAM Review 51, 661-703 (2009)].
This package provides two command line utilities, plfit and plgen.
The package is enhanced by the following packages:
plfit-doc
|
|
python3-jdata
JData encoder/decoder for python 3
|
Versions of package python3-jdata |
Release | Version | Architectures |
sid | 0.3.6-1 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x |
bullseye | 0.3.6-1 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
bookworm | 0.3.6-1 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
trixie | 0.3.6-1 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x |
|
License: DFSG free
|
The JData Specification (https://github.com/fangq/jdata/) defines a
lightweight language-independent data annotation interface targeted at
storing and sharing complex data structures across different programming
languages such as MATLAB, JavaScript, python etc. Using JData formats, a
complex python data structure can be encoded as a dict object that is
easily serialized as a JSON/binary JSON file and share such data between
programs of different languages.
|
|
python3-mdp
boite à outils modulaire pour le traitement de données
|
Versions of package python3-mdp |
Release | Version | Architectures |
jessie | 3.3-2 | all |
stretch | 3.5-1 | all |
bullseye | 3.6-1.1 | all |
bookworm | 3.6-2 | amd64,arm64,mips64el,ppc64el |
trixie | 3.6-7 | all |
sid | 3.6-8 | all |
|
License: DFSG free
|
Il s’agit d’un cadriciel en Python de traitement de données pour
construire des logiciels complexes de traitement de données en combinant
des algorithmes d’apprentissage automatique largement utilisés dans des
tuyauteries et réseaux. Les algorithmes implémentés incluent l'analyse en
composantes principales (PCA), l'analyse en composantes indépendantes
(ICA), Slow Feature Analysis (SFA — analyse des variations lentes),
Independent Slow Feature Analysis (ISFA), Growing Neural Gas (GNG — réseau
neuronal artificiel incrémental), l’analyse factorielle, l’analyse
discriminante linéaire de Fisher (FDA) et les classifieurs gaussiens.
|
|
python3-nxs
format NeXus de fichiers de données scientifiques – liaisons de Python 3
|
Versions of package python3-nxs |
Release | Version | Architectures |
sid | 4.4.1-4 | all |
trixie | 4.4.1-4 | all |
bookworm | 4.4.1-4 | all |
bullseye | 4.4.1-3 | all |
|
License: DFSG free
|
NeXus est un format de données courant pour la science des rayons X, des
neutrons et des muons. Il a été développé comme norme internationale par
les scientifiques et les programmeurs des institutions scientifiques
majeures d’Europe, d’Asie, d’Australie et d’Amérique du Nord dans le but
d’améliorer la coopération pour l’analyse et la visualisation de données
de neutrons, de rayons X et de muons.
Ce paquet fournit les liaisons de Python 3.
|
|
python3-pyzoltan
Wrapper for the Zoltan data management library
|
Versions of package python3-pyzoltan |
Release | Version | Architectures |
bullseye | 1.0.1-2+deb11u1 | amd64,arm64,ppc64el,s390x |
bookworm | 1.0.1-5+deb12u1 | amd64,arm64,ppc64el,s390x |
trixie | 1.0.1-9 | amd64,arm64,ppc64el,s390x |
sid | 1.0.1-9 | amd64,arm64,ppc64el,riscv64,s390x |
|
License: DFSG free
|
PyZoltan is as the name suggests, is a Python wrapper for the
Zoltan data management library.
In PyZoltan, only specific routines and objects are wrapped.
The following features of Zoltan are currently supported:
- Dynamic load balancing using geometric algorithms
- Unstructured point-to-point communication
- Distributed data directories
|
|
virtuoso-opensource
gestionnaire de bases de données à performances élevées
|
Versions of package virtuoso-opensource |
Release | Version | Architectures |
bookworm | 7.2.5.1+dfsg1-0.3 | all |
sid | 7.2.5.1+dfsg1-0.8 | all |
experimental | 7.2.12+dfsg-0.1 | all |
jessie | 6.1.6+dfsg2-2 | all |
buster | 6.1.6+dfsg2-4 | all |
stretch | 6.1.6+dfsg2-4 | all |
bullseye | 7.2.5.1+dfsg1-0.1 | all |
upstream | 7.2.12 |
Debtags of package virtuoso-opensource: |
role | metapackage, program |
works-with | db |
|
License: DFSG free
|
OpenLink Virtuoso est un gestionnaire de bases de données relationnel-objet
à hautes performances. Il comporte la gestion des transactions, un
compilateur SQL intelligent, la possibilité de sauvegardes à chaud, la
gestion de SQL:1999, un langage puissant à procédures stockées qui gère
Java ou .Net côté serveur et bien plus. Il gère toutes les principales
interfaces d'accès aux données, telles que ODBC, JDBC, ADO.NET et OLE/DB.
Virtuoso gère SPARQL embarqué dans SQL pour des requêtes sur des données
RDF stockées dans ses bases de données. SPARQL bénéficie d'une gestion bas
niveau incluse dans le moteur de bases de données lui-même,
comme les règles de typage conformes à SPARQL et un type dédié
de données IRI.
L'installation de ce méta-paquet permet d'installer l'ensemble complet des
paquets qui constituent Virtuoso OSE (« Open Source Edition »).
|
|
visidata
exploration rapide de données colonnaires dans un terminal
|
Versions of package visidata |
Release | Version | Architectures |
bullseye | 2.2.1-1 | all |
bookworm | 2.11-1 | all |
sid | 3.0.2-1 | all |
buster | 1.5.2-1 | all |
trixie | 3.0.2-1 | all |
|
License: DFSG free
|
VisiData est un utilitaire générique pour explorer, nettoyer, restructurer
et analyser des données colonnaires. Les sources actuellement gérées sont
TSV, CSV, texte de largeur fixe, JSON, SQLite, HTTP, HTML, .xls et .xlsx
(Microsoft Excel).
|
|
Official Debian packages with lower relevance
libnexus-dev
NeXus scientific data file format - development libraries
|
Versions of package libnexus-dev |
Release | Version | Architectures |
sid | 4.4.3-6 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x |
bookworm | 4.4.3-5 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
trixie | 4.4.3-6 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x |
bullseye | 4.4.3-5 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x |
|
License: DFSG free
|
NeXus is a common data format for neutron, X-ray, and muon science. It
is being developed as an international standard by scientists and
programmers representing major scientific facilities in Europe, Asia,
Australia, and North America in order to facilitate greater cooperation
in the analysis and visualization of neutron, X-ray, and muon data.
This is the package containing the development libraries.
|
|
libnexus-java
NeXus scientific data file format - java libraries
|
Versions of package libnexus-java |
Release | Version | Architectures |
sid | 4.4.3-6 | all |
bullseye | 4.4.3-5 | all |
bookworm | 4.4.3-5 | all |
trixie | 4.4.3-6 | all |
|
License: DFSG free
|
NeXus is a common data format for neutron, X-ray, and muon science. It
is being developed as an international standard by scientists and
programmers representing major scientific facilities in Europe, Asia,
Australia, and North America in order to facilitate greater cooperation
in the analysis and visualization of neutron, X-ray, and muon data.
This is the package containing the java libraries.
|
|
libplfit-dev
fitting power-law distributions to empirical data -- development
|
Versions of package libplfit-dev |
Release | Version | Architectures |
sid | 0.9.6+ds-1 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x |
bookworm | 0.9.4+ds-1 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el |
trixie | 0.9.4+ds-1 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el |
|
License: DFSG free
|
The plfit software fits power-law distributions to empirical (discrete or
continuous) data, according to the method of Clauset, Shalizi and Newman
[SIAM Review 51, 661-703 (2009)].
This package contains the header files, static libraries and symbolic
links that developers using the plfit library will need.
The package is enhanced by the following packages:
plfit-doc
|
|
python3-openpyxl
Python 3 module to read/write OpenXML xlsx/xlsm files
|
Versions of package python3-openpyxl |
Release | Version | Architectures |
trixie | 3.1.2+dfsg-6 | all |
bullseye | 3.0.3-1 | all |
bookworm | 3.0.9-1 | all |
buster | 2.4.9-1 | all |
sid | 3.1.2+dfsg-6 | all |
stretch | 2.3.0-3 | all |
|
License: DFSG free
|
Openpyxl is a pure Python 3 module to read/write Excel 2007 (OpenXML)
xlsx/xlsm files.
This package contains the module itself.
|
|
python3-opentsne
t-Distributed Stochastic Neighbor Embedding algorithm
|
Versions of package python3-opentsne |
Release | Version | Architectures |
sid | 1.0.0-1 | amd64,arm64,armel,armhf,mips64el,ppc64el,riscv64,s390x |
sid | 0.5.0-2 | i386 |
upstream | 1.0.1 |
|
License: DFSG free
|
Modular Python implementation of t-Distributed Stochasitc Neighbor
Embedding (t-SNE), a popular dimensionality-reduction algorithm for
visualizing high-dimensional data sets. openTSNE incorporates the
latest improvements to the t-SNE algorithm, including the ability to
add new data points to existing embeddings, massive speed
improvements, enabling t-SNE to scale to millions of data points and
various tricks to improve global alignment of the
resulting visualizations.
|
|
python3-plfit
fitting power-law distributions to empirical data -- Python
|
Versions of package python3-plfit |
Release | Version | Architectures |
sid | 0.9.6+ds-1 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x |
trixie | 0.9.4+ds-1 | amd64,arm64,armel,armhf,i386,mips64el,ppc64el |
bookworm | 0.9.4+ds-1 | amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el |
|
License: DFSG free
|
The plfit software fits power-law distributions to empirical (discrete or
continuous) data, according to the method of Clauset, Shalizi and Newman
[SIAM Review 51, 661-703 (2009)].
This package provides a Python module.
The package is enhanced by the following packages:
plfit-doc
|
|
|