Debian Science Project
Summary
Distributed Computing
Debian Science Distributed Computing packages

This metapackage will install Debian Science packages useful for various types of distributed computing, such as grid-, cloud-, cluster- and parallel-computing.

Description

For a better overview of the project's availability as a Debian package, each head row has a color code according to this scheme:

If you discover a project which looks like a good candidate for Debian Science to you, or if you have prepared an unofficial Debian package, please do not hesitate to send a description of that project to the Debian Science mailing list

Links to other tasks

Debian Science Distributed Computing packages

Official Debian packages with high relevance

dmtcp
Checkpoint/Restart functionality for Linux processes
Maintainer: Kapil Arya
Versions of package dmtcp
ReleaseVersionArchitectures
jessie2.3.1-6amd64,armhf,i386
Debtags of package dmtcp:
roleprogram
Popcon: 3 users (0 upd.)*
Versions and Archs
License: DFSG free
Svn

DMTCP (Distributed MultiThreaded Checkpointing) is a tool to transparently checkpointing the state of an arbitrary group of programs including multi-threaded and distributed computations. It operates directly on the user binary executable, with no Linux kernel modules or other kernel mods.

Among the applications supported by DMTCP are Open MPI, MATLAB, Python, Perl, and many programming languages and shell scripting languages. DMTCP also supports GNU screen sessions, including vim/cscope and emacs. With the use of TightVNC, it can also checkpoint and restart X-Window applications, as long as they do not use extensions (e.g.: no OpenGL, no video).

This package contains DMTCP binaries.

Please cite: Jason Ansel, Kapil Arya, and Gene Cooperman: DMTCP: Transparent Checkpointing for Cluster Computations and the Desktop (2009)
environment-modules
Modular system for handling environment variables
Maintainer: Alastair McKinstry
Versions of package environment-modules
ReleaseVersionArchitectures
stretch3.2.10-10amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
jessie3.2.10-9amd64,armel,armhf,i386
sid5.4.0-1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
trixie5.4.0-1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
bookworm5.2.0-1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
bullseye4.6.1-1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
buster4.2.2-1amd64,arm64,armhf,i386
Popcon: 53 users (73 upd.)*
Versions and Archs
License: DFSG free
Git

The Modules package provides for the dynamic modification of a user's environment via modulefiles. Each modulefile contains the information needed to configure the shell for an application. Once the Modules package is initialized, the environment can be modified dynamically on a per-module basis using the module command which interprets modulefiles. Typically modulefiles instruct the module command to alter or set shell environment variables such as PATH, MANPATH, etc. modulefiles may be shared by many users on a system and users may have their own collection to supplement or replace the shared modulefiles. The modules environment is common on SGI/Crays and many workstation farms.

globus-core
Globus Toolkit - Globus Core - Transitional Package
Maintainer: Mattias Ellert
Versions of package globus-core
ReleaseVersionArchitectures
jessie9.0amd64,armel,armhf,i386
stretch9.1amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
Popcon: 0 users (1 upd.)*
Versions and Archs
License: DFSG free
Svn

The Globus Toolkit is an open source software toolkit used for building Grid systems and applications. It is being developed by the Globus Alliance and many others all over the world. A growing number of projects and companies are using the Globus Toolkit to unlock the potential of grids for their cause.

The globus-core package is no longer needed with Globus Toolkit 6. This transitional package provides a dummy pkg-config file to fulfill dependencies during the transition. In can be removed when no longer needed.

gridengine-client
Utilities for Grid Engine queue management
Versions of package gridengine-client
ReleaseVersionArchitectures
stretch8.1.9+dfsg-4+deb9u2amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
buster8.1.9+dfsg-9amd64,arm64,armhf,i386
bookworm8.1.9+dfsg-10amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
sid8.1.9+dfsg-11.1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
trixie8.1.9+dfsg-11amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x
bullseye8.1.9+dfsg-9.1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
Debtags of package gridengine-client:
roleprogram
Popcon: 388 users (3 upd.)*
Versions and Archs
License: DFSG free
Git

Grid Engine is software that facilitates "distributed resource management" (DRM). Far more than just simple load-balancing tools or batch scheduling mechanisms, DRM software typically provides the following key features across large sets of distributed resources:

  • Policy based allocation of distributed resources (CPU time, software licenses, etc.)
  • Batch queuing & scheduling
  • Support diverse server hardware, OS and architectures
  • Load balancing & remote job execution
  • Detailed job accounting statistics
  • Fine-grained user specifiable resources
  • Suspend/resume/migrate jobs
  • Tools for reporting Job/Host/Cluster status
  • Job Arrays
  • Integration & control of parallel jobs

This package contains the Grid Engine user utilities.

gridengine-exec
Distributed resource management - Execution Server
Versions of package gridengine-exec
ReleaseVersionArchitectures
sid8.1.9+dfsg-11.1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
buster8.1.9+dfsg-9amd64,arm64,armhf,i386
bullseye8.1.9+dfsg-9.1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
stretch8.1.9+dfsg-4+deb9u2amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
trixie8.1.9+dfsg-11amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x
bookworm8.1.9+dfsg-10amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
Debtags of package gridengine-exec:
roleprogram
Popcon: 66 users (2 upd.)*
Versions and Archs
License: DFSG free
Git

Grid Engine is software that facilitates "distributed resource management" (DRM). Far more than just simple load-balancing tools or batch scheduling mechanisms, DRM software typically provides the following key features across large sets of distributed resources:

  • Policy based allocation of distributed resources (CPU time, software licenses, etc.)
  • Batch queuing & scheduling
  • Support diverse server hardware, OS and architectures
  • Load balancing & remote job execution
  • Detailed job accounting statistics
  • Fine-grained user specifiable resources
  • Suspend/resume/migrate jobs
  • Tools for reporting Job/Host/Cluster status
  • Job Arrays
  • Integration & control of parallel jobs

This package contains the Grid Engine execution server.

gridengine-master
Distributed resource management - Master Server
Versions of package gridengine-master
ReleaseVersionArchitectures
bookworm8.1.9+dfsg-10amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
trixie8.1.9+dfsg-11amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x
bullseye8.1.9+dfsg-9.1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
sid8.1.9+dfsg-11.1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
stretch8.1.9+dfsg-4+deb9u2amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
buster8.1.9+dfsg-9amd64,arm64,armhf,i386
Popcon: 30 users (1 upd.)*
Versions and Archs
License: DFSG free
Git

Grid Engine is software that facilitates "distributed resource management" (DRM). Far more than just simple load-balancing tools or batch scheduling mechanisms, DRM software typically provides the following key features across large sets of distributed resources:

  • Policy based allocation of distributed resources (CPU time, software licenses, etc.)
  • Batch queuing & scheduling
  • Support diverse server hardware, OS and architectures
  • Load balancing & remote job execution
  • Detailed job accounting statistics
  • Fine-grained user specifiable resources
  • Suspend/resume/migrate jobs
  • Tools for reporting Job/Host/Cluster status
  • Job Arrays
  • Integration & control of parallel jobs

This package contains the Grid Engine master server.

gridengine-qmon
Graphical utilities for Grid Engine queue management
Versions of package gridengine-qmon
ReleaseVersionArchitectures
buster8.1.9+dfsg-9amd64,arm64,armhf,i386
stretch8.1.9+dfsg-4+deb9u2amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
bookworm8.1.9+dfsg-10amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
trixie8.1.9+dfsg-11amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x
sid8.1.9+dfsg-11.1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
bullseye8.1.9+dfsg-9.1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
Debtags of package gridengine-qmon:
uitoolkitmotif
Popcon: 331 users (2 upd.)*
Versions and Archs
License: DFSG free
Git

Grid Engine is software that facilitates "distributed resource management" (DRM). Far more than just simple load-balancing tools or batch scheduling mechanisms, DRM software typically provides the following key features across large sets of distributed resources:

  • Policy based allocation of distributed resources (CPU time, software licenses, etc.)
  • Batch queuing & scheduling
  • Support diverse server hardware, OS and architectures
  • Load balancing & remote job execution
  • Detailed job accounting statistics
  • Fine-grained user specifiable resources
  • Suspend/resume/migrate jobs
  • Tools for reporting Job/Host/Cluster status
  • Job Arrays
  • Integration & control of parallel jobs

This package contains the graphical Grid Engine administration frontend.

htcondor
transitional dummy package
Versions of package htcondor
ReleaseVersionArchitectures
stretch8.4.11~dfsg.1-1amd64,arm64,armel,armhf,i386,mips,mipsel,ppc64el,s390x
stretch-security8.4.11~dfsg.1-1+deb9u2amd64,arm64,armel,armhf,i386
buster-security8.6.8~dfsg.1-2+deb10u1amd64,arm64,armhf,i386
jessie8.2.3~dfsg.1-6amd64,armel,armhf,i386
buster8.6.8~dfsg.1-2+deb10u1amd64,arm64,armhf,i386
sid23.4.0+dfsg-1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
upstream23.7.2
Popcon: 80 users (0 upd.)*
Newer upstream!
License: DFSG free
Git

This package aids upgrades of existing htcondor installations to the package name "condor". The package is empty and it can safely be removed.

Please cite: Michael Litzkow, Miron Livny and Matt Mutka: Condor - A Hunter of Idle Workstations (eprint) Proceedings of the 8th International Conference of Distributed Computing Systems :104-111 (1988)
ipython3
interfaz de comandos interactiva de Python 3
Versions of package ipython3
ReleaseVersionArchitectures
bullseye7.20.0-1+deb11u1all
bullseye-security7.20.0-1+deb11u1all
jessie2.3.0-2all
sid8.20.0-1all
stretch-security5.1.0-3+deb9u1all
buster5.8.0-1+deb10u1all
buster-security5.8.0-1+deb10u1all
stretch5.1.0-3all
trixie8.20.0-1all
bookworm8.5.0-4all
upstream8.25.0
Popcon: 1197 users (354 upd.)*
Newer upstream!
License: DFSG free
Git

IPython se puede usar como reemplazo de la línea de órdenes estándar de Python, o como un entorno completo de trabajo para cálculos científicos (como Matlab o Mathematica) cuando se usa junto a las herramientas científicas y numéricas de Python. Permite tener introspección de objetos dinámica, líneas de entrada y salida numeradas, un sistema de macros, registro de sesiones, restauración de sesiones, acceso completo a la línea de comandos del sistema, reportes prolijos y a color de los volcados de pila, paréntesis automáticos, comillas automáticas y se puede incrustar dentro de otros programas de Python.

Este paquete contiene propiamente la línea de comandos para Python 3.

The package is enhanced by the following packages: isympy3 python3-matplotlib python3-sympy
Screenshots of package ipython3
mpich
Implementation of the MPI Message Passing Interface standard
Versions of package mpich
ReleaseVersionArchitectures
trixie4.2.0-6amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
jessie3.1-5amd64,armel,armhf,i386
stretch3.2-7amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
buster3.3-3amd64,arm64,armhf,i386
bullseye3.4.1-5~deb11u1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
bookworm4.0.2-3amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
sid4.2.0-6amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
experimental4.2.1-2amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
upstream4.2.1
Debtags of package mpich:
adminbenchmarking, monitoring
networkhiavailability, load-balancing, scanner
scopeutility
usetransmission
Popcon: 69 users (83 upd.)*
Newer upstream!
License: DFSG free
Git

MPICH is a high-performance and widely portable implementation of the MPI-3.1 standard from the Argonne National Laboratory. It efficiently supports different computation and communication platforms including commodity clusters, SMPs, massively parallel systems, and high-speed networks. This release has all MPI 3.1 functions and features required by the standard with the exception of support for the "external32" portable I/O format and user-defined data representations for I/O.

This package includes the program binaries necessary to run MPICH programs.

mpich2python
MPI-enhanced Python interpreter (MPICH2 based version)
Versions of package mpich2python
ReleaseVersionArchitectures
jessie2.9.4-3amd64,armel,armhf,i386
Popcon: 0 users (0 upd.)*
Versions and Archs
License: DFSG free
Git

The package provides a python interpreter with MPI (Message Passing Interface, message-based parallel programming) support.

nordugrid-arc-client
ARC command line clients
Maintainer: Mattias Ellert
Versions of package nordugrid-arc-client
ReleaseVersionArchitectures
bullseye6.10.2-1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
buster5.4.3-2amd64,arm64,armhf,i386
stretch5.3.0~rc1-1amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
jessie4.2.0-2amd64,armel,armhf,i386
sid6.19.0-1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
trixie6.19.0-1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x
bookworm6.17.0-3amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
Popcon: 2 users (0 upd.)*
Versions and Archs
License: DFSG free
Git

NorduGrid is a collaboration aiming at development, maintenance and support of the middleware, known as the Advanced Resource Connector (ARC).

This client package contains all the CLI tools that are needed to operate with x509 proxies, submit and manage jobs and handle data transfers.

openmpi-bin
high performance message passing library -- binaries
Versions of package openmpi-bin
ReleaseVersionArchitectures
jessie1.6.5-9.1+deb8u1amd64,armel,armhf,i386
sid4.1.6-13.3amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
trixie4.1.6-13.3amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x
buster3.1.3-11amd64,arm64,armhf,i386
stretch2.0.2-2amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
bookworm4.1.4-3amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
bullseye4.1.0-10amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
experimental5.0.3-3amd64,arm64,mips64el,ppc64el,riscv64,s390x
upstream5.0.3
Debtags of package openmpi-bin:
admincluster
fieldbiology, chemistry, mathematics, physics
interfacecommandline
roleprogram
scopeutility
Popcon: 1028 users (711 upd.)*
Newer upstream!
License: DFSG free
Git

Open MPI is a project combining technologies and resources from several other projects (FT-MPI, LA-MPI, LAM/MPI, and PACX-MPI) in order to build the best MPI library available. A completely new MPI-3.1 compliant implementation, Open MPI offers advantages for system and software vendors, application developers and computer science researchers.

Features:

  • Full MPI-3.1 standards conformance
  • Thread safety and concurrency
  • Dynamic process spawning
  • High performance on all platforms
  • Reliable and fast job management
  • Network and process fault tolerance
  • Support network heterogeneity
  • Single library supports all networks
  • Run-time instrumentation
  • Many job schedulers supported
  • Internationalized error messages
  • Component-based design, documented APIs

This package contains the Open MPI utility programs.

openmpipython
MPI-enhanced Python interpreter (OpenMPI based version)
Versions of package openmpipython
ReleaseVersionArchitectures
jessie2.9.4-3amd64,armel,armhf,i386
Popcon: 0 users (0 upd.)*
Versions and Archs
License: DFSG free
Git

The package provides a python interpreter with MPI (Message Passing Interface, message-based parallel programming) support.

python3-scoop
Python3 library for concurrent parallel programming
Versions of package python3-scoop
ReleaseVersionArchitectures
buster0.7.1.1-3all
bullseye0.7.1.1-7all
stretch0.7.1.1-1all
Popcon: 0 users (1 upd.)*
Versions and Archs
License: DFSG free
Git

SCOOP (Scalable COncurrent Operations in Python) is a distributed task module allowing concurrent parallel programming on various environments, from heterogeneous grids to supercomputers.

This package contains the Python3 modules.

Official Debian packages with lower relevance

coop-computing-tools
cooperative computing tools
Versions of package coop-computing-tools
ReleaseVersionArchitectures
buster7.0.9-2amd64,arm64,armhf,i386
bookworm9.9-2amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
sid9.9-4.1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
trixie9.9-4.1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,s390x
bullseye7.1.2-5amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
stretch4.0-1.1amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
Debtags of package coop-computing-tools:
uitoolkitncurses
Popcon: 3 users (1 upd.)*
Versions and Archs
License: DFSG free
Git

This is a collection of software that help users to share resources in a complex, heterogeneous, and unreliable computing environment. This includes:

  • Chirp: A personal filesystem and I/O protocol that allows unprivileged users to share space securely, efficiently, and conveniently. When combined with Parrot, Chirp allows users to create custom wide-area distributed filesystems.
  • Parrot: A transparent user-level virtual filesystem that allows any ordinary program to be attached to a remote storage device such as an FTP server or a Chirp server.
  • Makeflow: A workflow system for parallel and distributed computing that uses a language very similar to Make.
  • Work Queue: A system and API for building master-worker style programs that scale up to thousands of processors.
  • All Pairs: A computational abstraction for running very large Cartesian products.
  • Wavefront: A computational abstraction for running very large dynamic programming problems.
  • The Fault Tolerant Shell: A high-level programming language that allows users to combine the ease of shell scripting, the power of distributed programming, and the precision of compiled languages. Basically, parallel programming and exception handling for scripts.
Please cite: Douglas Thain, Christopher Moretti, and Jeffrey Hemmes: Chirp: A Practical Global Filesystem for Cluster and Grid Computing. (2009)
hpcc
HPC Challenge benchmark
Versions of package hpcc
ReleaseVersionArchitectures
jessie1.4.1-3amd64,armel,armhf,i386
stretch1.4.3-1amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
buster1.5.0-1amd64,arm64,armhf,i386
bookworm1.5.0-3amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
sid1.5.0-4amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
trixie1.5.0-4amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
Debtags of package hpcc:
adminbenchmarking
roleprogram
Popcon: 4 users (7 upd.)*
Versions and Archs
License: DFSG free
Git

The High Performance Computing (HPC) Challenge benchmark runs a suite of 7 tests that measure the performance of CPU, memory and network for HPC clusters. Amongst others, it includes the High-Performance LINPACK (HPL) benchmark, used by the Top500 ranking (http://www.top500.org/).

python3-mpi4py
bindings of the Message Passing Interface (MPI) standard
Versions of package python3-mpi4py
ReleaseVersionArchitectures
sid3.1.6-1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
trixie3.1.6-1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
bookworm3.1.4-2amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
bullseye3.0.3-8amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
buster2.0.0-3amd64,arm64,armhf,i386
stretch2.0.0-2.1+deb9u1amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
jessie1.3.1+hg20131106-2amd64,armel,armhf,i386
Popcon: 650 users (134 upd.)*
Versions and Archs
License: DFSG free
Git

MPI for Python (mpi4py) provides bindings of the Message Passing Interface (MPI) standard for the Python programming language, allowing any Python program to exploit multiple processors.

mpi4py is constructed on top of the MPI-1/MPI-2 specification and provides an object oriented interface which closely follows MPI-2 C++ bindings. It supports point-to-point (sends, receives) and collective (broadcasts, scatters, gathers) communications of any picklable Python object as well as optimized communications of Python object exposing the single-segment buffer interface (NumPy arrays, builtin bytes/string/array objects).

No known packages available

psom
pipeline system for Octave and Matlab
Responsible: NeuroDebian Team
License: MIT/Expat
Debian package not available
Language: Matlab/Octave

PSOM is a lightweight library to manage complex multi-stage data processing. A pipeline is a collection of jobs, i.e. Matlab or Octave code with a well identified set of options that are using files for inputs and outputs. To use PSOM, the only requirement is to generate a description of a pipeline in the form of a simple Matlab/Octave structure. PSOM then automatically offers the following services:

  • Run jobs in parallel using multiple CPUs or within a distributed computing environment.
  • Generate log files and keep track of the pipeline execution. These logs are detailed enough to fully reproduce the analysis.
  • Handle job failures: successful completion of jobs is checked and failed jobs can be restarted.
  • Handle updates of the pipeline: change options or add jobs and let PSOM figure out what to reprocess.
*Popularitycontest results: number of people who use this package regularly (number of people who upgraded this package recently) out of 237679