Debian Science Project
Summary
Distributed Computing
Debian Science Distributed Computing packages

This metapackage will install Debian Science packages useful for various types of distributed computing, such as grid-, cloud-, cluster- and parallel-computing.

Description

For a better overview of the project's availability as a Debian package, each head row has a color code according to this scheme:

If you discover a project which looks like a good candidate for Debian Science to you, or if you have prepared an unofficial Debian package, please do not hesitate to send a description of that project to the Debian Science mailing list

Links to other tasks

Debian Science Distributed Computing packages

Official Debian packages with high relevance

dmtcp
??? missing short description for package dmtcp :-(
Maintainer: Kapil Arya
Versions of package dmtcp
ReleaseVersionArchitectures
jessie2.3.1-6amd64,armhf,i386
Debtags of package dmtcp:
roleprogram
Popcon: 2 users (0 upd.)*
Versions and Archs
License: DFSG free
Svn
Please cite: Jason Ansel, Kapil Arya, and Gene Cooperman: DMTCP: Transparent Checkpointing for Cluster Computations and the Desktop (2009)
environment-modules
Modular system for handling environment variables
Maintainer: Alastair McKinstry
Versions of package environment-modules
ReleaseVersionArchitectures
jessie3.2.10-9amd64,armel,armhf,i386
buster4.2.2-1amd64,arm64,armhf,i386
trixie5.5.0-1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
bookworm5.2.0-1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
sid5.5.0-1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
stretch3.2.10-10amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
bullseye4.6.1-1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
Popcon: 32 users (89 upd.)*
Versions and Archs
License: DFSG free
Git

The Modules package provides for the dynamic modification of a user's environment via modulefiles. Each modulefile contains the information needed to configure the shell for an application. Once the Modules package is initialized, the environment can be modified dynamically on a per-module basis using the module command which interprets modulefiles. Typically modulefiles instruct the module command to alter or set shell environment variables such as PATH, MANPATH, etc. modulefiles may be shared by many users on a system and users may have their own collection to supplement or replace the shared modulefiles. The modules environment is common on SGI/Crays and many workstation farms.

globus-core
??? missing short description for package globus-core :-(
Maintainer: Mattias Ellert
Versions of package globus-core
ReleaseVersionArchitectures
jessie9.0amd64,armel,armhf,i386
stretch9.1amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
Popcon: 0 users (0 upd.)*
Versions and Archs
License: DFSG free
Svn
gridengine-client
Utilities for Grid Engine queue management
Versions of package gridengine-client
ReleaseVersionArchitectures
sid8.1.9+dfsg-11.1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
bookworm8.1.9+dfsg-10amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
stretch8.1.9+dfsg-4+deb9u2amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
buster8.1.9+dfsg-9amd64,arm64,armhf,i386
bullseye8.1.9+dfsg-9.1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
Debtags of package gridengine-client:
roleprogram
Popcon: 64 users (0 upd.)*
Versions and Archs
License: DFSG free
Git

Grid Engine is software that facilitates "distributed resource management" (DRM). Far more than just simple load-balancing tools or batch scheduling mechanisms, DRM software typically provides the following key features across large sets of distributed resources:

  • Policy based allocation of distributed resources (CPU time, software licenses, etc.)
  • Batch queuing & scheduling
  • Support diverse server hardware, OS and architectures
  • Load balancing & remote job execution
  • Detailed job accounting statistics
  • Fine-grained user specifiable resources
  • Suspend/resume/migrate jobs
  • Tools for reporting Job/Host/Cluster status
  • Job Arrays
  • Integration & control of parallel jobs

This package contains the Grid Engine user utilities.

gridengine-exec
Distributed resource management - Execution Server
Versions of package gridengine-exec
ReleaseVersionArchitectures
sid8.1.9+dfsg-11.1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
buster8.1.9+dfsg-9amd64,arm64,armhf,i386
stretch8.1.9+dfsg-4+deb9u2amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
bookworm8.1.9+dfsg-10amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
bullseye8.1.9+dfsg-9.1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
Debtags of package gridengine-exec:
roleprogram
Popcon: 56 users (0 upd.)*
Versions and Archs
License: DFSG free
Git

Grid Engine is software that facilitates "distributed resource management" (DRM). Far more than just simple load-balancing tools or batch scheduling mechanisms, DRM software typically provides the following key features across large sets of distributed resources:

  • Policy based allocation of distributed resources (CPU time, software licenses, etc.)
  • Batch queuing & scheduling
  • Support diverse server hardware, OS and architectures
  • Load balancing & remote job execution
  • Detailed job accounting statistics
  • Fine-grained user specifiable resources
  • Suspend/resume/migrate jobs
  • Tools for reporting Job/Host/Cluster status
  • Job Arrays
  • Integration & control of parallel jobs

This package contains the Grid Engine execution server.

gridengine-master
Distributed resource management - Master Server
Versions of package gridengine-master
ReleaseVersionArchitectures
stretch8.1.9+dfsg-4+deb9u2amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
buster8.1.9+dfsg-9amd64,arm64,armhf,i386
sid8.1.9+dfsg-11.1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
bookworm8.1.9+dfsg-10amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
bullseye8.1.9+dfsg-9.1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
Popcon: 27 users (0 upd.)*
Versions and Archs
License: DFSG free
Git

Grid Engine is software that facilitates "distributed resource management" (DRM). Far more than just simple load-balancing tools or batch scheduling mechanisms, DRM software typically provides the following key features across large sets of distributed resources:

  • Policy based allocation of distributed resources (CPU time, software licenses, etc.)
  • Batch queuing & scheduling
  • Support diverse server hardware, OS and architectures
  • Load balancing & remote job execution
  • Detailed job accounting statistics
  • Fine-grained user specifiable resources
  • Suspend/resume/migrate jobs
  • Tools for reporting Job/Host/Cluster status
  • Job Arrays
  • Integration & control of parallel jobs

This package contains the Grid Engine master server.

gridengine-qmon
Graphical utilities for Grid Engine queue management
Versions of package gridengine-qmon
ReleaseVersionArchitectures
buster8.1.9+dfsg-9amd64,arm64,armhf,i386
sid8.1.9+dfsg-11.1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
bookworm8.1.9+dfsg-10amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
bullseye8.1.9+dfsg-9.1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
stretch8.1.9+dfsg-4+deb9u2amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
Debtags of package gridengine-qmon:
uitoolkitmotif
Popcon: 17 users (0 upd.)*
Versions and Archs
License: DFSG free
Git

Grid Engine is software that facilitates "distributed resource management" (DRM). Far more than just simple load-balancing tools or batch scheduling mechanisms, DRM software typically provides the following key features across large sets of distributed resources:

  • Policy based allocation of distributed resources (CPU time, software licenses, etc.)
  • Batch queuing & scheduling
  • Support diverse server hardware, OS and architectures
  • Load balancing & remote job execution
  • Detailed job accounting statistics
  • Fine-grained user specifiable resources
  • Suspend/resume/migrate jobs
  • Tools for reporting Job/Host/Cluster status
  • Job Arrays
  • Integration & control of parallel jobs

This package contains the graphical Grid Engine administration frontend.

htcondor
transitional dummy package
Versions of package htcondor
ReleaseVersionArchitectures
trixie23.6.2+dfsg-2amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
jessie8.2.3~dfsg.1-6amd64,armel,armhf,i386
stretch8.4.11~dfsg.1-1amd64,arm64,armel,armhf,i386,mips,mipsel,ppc64el,s390x
stretch-security8.4.11~dfsg.1-1+deb9u2amd64,arm64,armel,armhf,i386
buster8.6.8~dfsg.1-2+deb10u1amd64,arm64,armhf,i386
sid23.6.2+dfsg-2amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
buster-security8.6.8~dfsg.1-2+deb10u1amd64,arm64,armhf,i386
upstream24.2.2
Popcon: 81 users (1 upd.)*
Newer upstream!
License: DFSG free
Git

This package aids upgrades of existing htcondor installations to the package name "condor". The package is empty and it can safely be removed.

Please cite: Michael Litzkow, Miron Livny and Matt Mutka: Condor - A Hunter of Idle Workstations (eprint) Proceedings of the 8th International Conference of Distributed Computing Systems :104-111 (1988)
ipython3
Enhanced interactive Python 3 shell
Maintainer: Debian Python Team
Versions of package ipython3
ReleaseVersionArchitectures
bullseye7.20.0-1+deb11u1all
jessie2.3.0-2all
stretch5.1.0-3all
stretch-security5.1.0-3+deb9u1all
buster5.8.0-1+deb10u1all
buster-security5.8.0-1+deb10u1all
bullseye-security7.20.0-1+deb11u1all
bookworm8.5.0-4all
trixie8.29.0-1all
sid8.30.0-1all
Popcon: 1776 users (866 upd.)*
Versions and Archs
License: DFSG free
Git

IPython can be used as a replacement for the standard Python shell, or it can be used as a complete working environment for scientific computing (like Matlab or Mathematica) when paired with the standard Python scientific and numerical tools. It supports dynamic object introspections, numbered input/output prompts, a macro system, session logging, session restoring, complete system shell access, verbose and colored traceback reports, auto-parentheses, auto-quoting, and is embeddable in other Python programs.

This package contains the actual terminal shell for Python 3.

The package is enhanced by the following packages: isympy3 python3-matplotlib python3-sympy
Screenshots of package ipython3
mpich
Implementation of the MPI Message Passing Interface standard
Versions of package mpich
ReleaseVersionArchitectures
bullseye3.4.1-5~deb11u1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
buster3.3-3amd64,arm64,armhf,i386
stretch3.2-7amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
jessie3.1-5amd64,armel,armhf,i386
experimental4.2.1-3amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
sid4.2.1-2amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
trixie4.2.0-14amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
bookworm4.0.2-3amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
upstream4.2.3
Debtags of package mpich:
adminbenchmarking, monitoring
networkhiavailability, load-balancing, scanner
scopeutility
usetransmission
Popcon: 63 users (45 upd.)*
Newer upstream!
License: DFSG free
Git

MPICH is a high-performance and widely portable implementation of the MPI-3.1 standard from the Argonne National Laboratory. It efficiently supports different computation and communication platforms including commodity clusters, SMPs, massively parallel systems, and high-speed networks. This release has all MPI 3.1 functions and features required by the standard with the exception of support for the "external32" portable I/O format and user-defined data representations for I/O.

This package includes the program binaries necessary to run MPICH programs.

mpich2python
??? missing short description for package mpich2python :-(
Versions of package mpich2python
ReleaseVersionArchitectures
jessie2.9.4-3amd64,armel,armhf,i386
Popcon: users ( upd.)*
Versions and Archs
License: DFSG free
Git
nordugrid-arc-client
ARC command line clients
Maintainer: Mattias Ellert
Versions of package nordugrid-arc-client
ReleaseVersionArchitectures
jessie4.2.0-2amd64,armel,armhf,i386
stretch5.3.0~rc1-1amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
buster5.4.3-2amd64,arm64,armhf,i386
bullseye6.10.2-1amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
sid6.21.0-1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
trixie6.21.0-1amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
bookworm6.17.0-3amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
Popcon: 2 users (1 upd.)*
Versions and Archs
License: DFSG free
Git

NorduGrid is a collaboration aiming at development, maintenance and support of the middleware, known as the Advanced Resource Connector (ARC).

This client package contains all the CLI tools that are needed to operate with x509 proxies, submit and manage jobs and handle data transfers.

openmpi-bin
high performance message passing library -- binaries
Versions of package openmpi-bin
ReleaseVersionArchitectures
jessie1.6.5-9.1+deb8u1amd64,armel,armhf,i386
sid5.0.6-3amd64,arm64,mips64el,ppc64el,riscv64,s390x
trixie5.0.6-3amd64,arm64,mips64el,ppc64el,riscv64,s390x
bookworm4.1.4-3amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
bullseye4.1.0-10amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
buster3.1.3-11amd64,arm64,armhf,i386
stretch2.0.2-2amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
Debtags of package openmpi-bin:
admincluster
fieldbiology, chemistry, mathematics, physics
interfacecommandline
roleprogram
scopeutility
Popcon: 1061 users (671 upd.)*
Versions and Archs
License: DFSG free
Git

Open MPI is a project combining technologies and resources from several other projects (FT-MPI, LA-MPI, LAM/MPI, and PACX-MPI) in order to build the best MPI library available. A completely new MPI-3.1 compliant implementation, Open MPI offers advantages for system and software vendors, application developers and computer science researchers.

Features:

  • Full MPI-3.1 standards conformance
  • Thread safety and concurrency
  • Dynamic process spawning
  • High performance on all platforms
  • Reliable and fast job management
  • Network and process fault tolerance
  • Support network heterogeneity
  • Single library supports all networks
  • Run-time instrumentation
  • Many job schedulers supported
  • Internationalized error messages
  • Component-based design, documented APIs

This package contains the Open MPI utility programs.

openmpipython
??? missing short description for package openmpipython :-(
Versions of package openmpipython
ReleaseVersionArchitectures
jessie2.9.4-3amd64,armel,armhf,i386
Popcon: 0 users (0 upd.)*
Versions and Archs
License: DFSG free
Git
python3-scoop
Python3 library for concurrent parallel programming
Versions of package python3-scoop
ReleaseVersionArchitectures
bullseye0.7.1.1-7all
buster0.7.1.1-3all
stretch0.7.1.1-1all
Popcon: 0 users (0 upd.)*
Versions and Archs
License: DFSG free
Git

SCOOP (Scalable COncurrent Operations in Python) is a distributed task module allowing concurrent parallel programming on various environments, from heterogeneous grids to supercomputers.

This package contains the Python3 modules.

Official Debian packages with lower relevance

coop-computing-tools
cooperative computing tools
Maintainer: Alastair McKinstry
Versions of package coop-computing-tools
ReleaseVersionArchitectures
sid9.9-4.1armel,armhf
stretch4.0-1.1amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
sid7.13.1-2amd64,arm64,i386,mips64el,ppc64el,riscv64,s390x
bookworm9.9-2amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
bullseye7.1.2-5amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
buster7.0.9-2amd64,arm64,armhf,i386
Debtags of package coop-computing-tools:
uitoolkitncurses
Popcon: 2 users (1 upd.)*
Versions and Archs
License: DFSG free
Git

This is a collection of software that help users to share resources in a complex, heterogeneous, and unreliable computing environment. This includes:

  • Chirp: A personal filesystem and I/O protocol that allows unprivileged users to share space securely, efficiently, and conveniently. When combined with Parrot, Chirp allows users to create custom wide-area distributed filesystems.
  • Parrot: A transparent user-level virtual filesystem that allows any ordinary program to be attached to a remote storage device such as an FTP server or a Chirp server.
  • Makeflow: A workflow system for parallel and distributed computing that uses a language very similar to Make.
  • Work Queue: A system and API for building master-worker style programs that scale up to thousands of processors.
  • All Pairs: A computational abstraction for running very large Cartesian products.
  • Wavefront: A computational abstraction for running very large dynamic programming problems.
  • The Fault Tolerant Shell: A high-level programming language that allows users to combine the ease of shell scripting, the power of distributed programming, and the precision of compiled languages. Basically, parallel programming and exception handling for scripts.
Please cite: Douglas Thain, Christopher Moretti, and Jeffrey Hemmes: Chirp: A Practical Global Filesystem for Cluster and Grid Computing. (2009)
hpcc
HPC Challenge benchmark
Versions of package hpcc
ReleaseVersionArchitectures
sid1.5.0-4amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
trixie1.5.0-4amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
stretch1.4.3-1amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
bookworm1.5.0-3amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
buster1.5.0-1amd64,arm64,armhf,i386
jessie1.4.1-3amd64,armel,armhf,i386
Debtags of package hpcc:
adminbenchmarking
roleprogram
Popcon: 2 users (7 upd.)*
Versions and Archs
License: DFSG free
Git

The High Performance Computing (HPC) Challenge benchmark runs a suite of 7 tests that measure the performance of CPU, memory and network for HPC clusters. Amongst others, it includes the High-Performance LINPACK (HPL) benchmark, used by the Top500 ranking (http://www.top500.org/).

python3-mpi4py
bindings of the Message Passing Interface (MPI) standard
Versions of package python3-mpi4py
ReleaseVersionArchitectures
jessie1.3.1+hg20131106-2amd64,armel,armhf,i386
bookworm3.1.4-2amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
bullseye3.0.3-8amd64,arm64,armel,armhf,i386,mips64el,mipsel,ppc64el,s390x
trixie4.0.1-6amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
sid4.0.1-6amd64,arm64,armel,armhf,i386,mips64el,ppc64el,riscv64,s390x
experimental4.0.1-6.exp1amd64,arm64,mips64el,ppc64el,riscv64
buster2.0.0-3amd64,arm64,armhf,i386
stretch2.0.0-2.1+deb9u1amd64,arm64,armel,armhf,i386,mips,mips64el,mipsel,ppc64el,s390x
Popcon: 847 users (140 upd.)*
Versions and Archs
License: DFSG free
Git

MPI for Python (mpi4py) provides bindings of the Message Passing Interface (MPI) standard for the Python programming language, allowing any Python program to exploit multiple processors.

mpi4py is constructed on top of the MPI-1/MPI-2 specification and provides an object oriented interface which closely follows MPI-2 C++ bindings. It supports point-to-point (sends, receives) and collective (broadcasts, scatters, gathers) communications of any picklable Python object as well as optimized communications of Python object exposing the single-segment buffer interface (NumPy arrays, builtin bytes/string/array objects).

No known packages available

psom
pipeline system for Octave and Matlab
Responsible: NeuroDebian Team
License: MIT/Expat
Debian package not available
Language: Matlab/Octave

PSOM is a lightweight library to manage complex multi-stage data processing. A pipeline is a collection of jobs, i.e. Matlab or Octave code with a well identified set of options that are using files for inputs and outputs. To use PSOM, the only requirement is to generate a description of a pipeline in the form of a simple Matlab/Octave structure. PSOM then automatically offers the following services:

  • Run jobs in parallel using multiple CPUs or within a distributed computing environment.
  • Generate log files and keep track of the pipeline execution. These logs are detailed enough to fully reproduce the analysis.
  • Handle job failures: successful completion of jobs is checked and failed jobs can be restarted.
  • Handle updates of the pipeline: change options or add jobs and let PSOM figure out what to reprocess.
*Popularitycontest results: number of people who use this package regularly (number of people who upgraded this package recently) out of 247866