Debian Science Data Management packages

DataLad is a data management and distribution platform providing access to a wide range of data resources already available online. Using git-annex as its backend for data logistics it provides following facilities built-in or available through additional extensions

• command line and Python interfaces for manipulation of collections of datasets (install, uninstall, update, publish, save, etc.) and separate files/directories (add, get)
• extract, aggregate, and search through various sources of metadata (xmp, EXIF, etc; install datalad-neuroimaging for DICOM, BIDS, NIfTI support)
• crawl web sites to automatically prepare and update git-annex repositories with content from online websites, S3, etc (install datalad-crawler)

This extension enhances DataLad (http://datalad.org) for working with computational containers.

git-annex는 git에 파일 내용을 저장하지 않고, git으로 대용량 파일을 관리할 수 있습니다. 오프라인과 온라인에서 데이타를 동기화, 백업, 보관할 수 있습니다.체크섬과 암호화는 데이타를 안전하게 유지합니다. git-annex를 사용해서 git의 강력한 성능과 분산 특성을 대용량 파일에 적용할 수 있습니다.

로컬 하드 디스크에서 S3, WebDav, rsync등을 포함하는, 수많은 클라우드 스토리지 서비스까지 다양한 장소에 대용량 파일을 저장할 수 있으며, 플러그인을 통해 수십 개의 클라우드 스토리지 공급자를 사용할 수 있습니다. 파일은 gpg를 통해 암호화되므로 클라우드 스토리지 공급자는 파일의 내용을 볼 수 없습니다. git-annex는 각 파일이 저장되는 위치를 추적하여 사용 가능한 복사본 수를 파악하고 데이터를 보존할 수 있는 많은 기능을 제공합니다.

git-annex는 컴퓨터간 폴더를 싱크하고, 파일이 변경되면 이를 알아차려 자동으로 변경된 파일을 git에 커밋하고 변경된 파일을 다른 컴퓨터로 전송하는데 사용될 수 있습니다. git-annex webapp은 git-annex를 쉽게 사용할 수 있도록 합니다.

The external filter mechanism introduced with HDF5 1.8.12 allows applications to utilize custom filters not shipped by the HDF5 core library without recompiling your application. This package provides external filters for HDF5 for

• the LZ4 compression algorithm
• BZip2 compression

This package contains a filter for HDF5 that uses the Blosc compressor. By installing this filter, you can read and write HDF5 files with Blosc-compressed datasets.

H5Z-ZFP is a compression filter for HDF5 using the ZFP compression library, supporting lossy and lossless compression of floating point and integer data to meet bitrate, accuracy, and/or precision targets.

NeXus is a common data format for neutron, X-ray, and muon science. It is being developed as an international standard by scientists and programmers representing major scientific facilities in Europe, Asia, Australia, and North America in order to facilitate greater cooperation in the analysis and visualization of neutron, X-ray, and muon data.

This is the package containing some applications for reading and writing NeXus files.

The plfit software fits power-law distributions to empirical (discrete or continuous) data, according to the method of Clauset, Shalizi and Newman [SIAM Review 51, 661-703 (2009)].

This package provides two command line utilities, plfit and plgen.

The JData Specification (https://github.com/fangq/jdata/) defines a lightweight language-independent data annotation interface targeted at storing and sharing complex data structures across different programming languages such as MATLAB, JavaScript, python etc. Using JData formats, a complex python data structure can be encoded as a dict object that is easily serialized as a JSON/binary JSON file and share such data between programs of different languages.

Python data processing framework for building complex data processing software by combining widely used machine learning algorithms into pipelines and networks. Implemented algorithms include: Principal Component Analysis (PCA), Independent Component Analysis (ICA), Slow Feature Analysis (SFA), Independent Slow Feature Analysis (ISFA), Growing Neural Gas (GNG), Factor Analysis, Fisher Discriminant Analysis (FDA), and Gaussian Classifiers.

NeXus is a common data format for neutron, X-ray, and muon science. It is being developed as an international standard by scientists and programmers representing major scientific facilities in Europe, Asia, Australia, and North America in order to facilitate greater cooperation in the analysis and visualization of neutron, X-ray, and muon data.

This is the package containing the Python 3 bindings.

PyZoltan is as the name suggests, is a Python wrapper for the Zoltan data management library.

In PyZoltan, only specific routines and objects are wrapped. The following features of Zoltan are currently supported:

• Dynamic load balancing using geometric algorithms
• Unstructured point-to-point communication
• Distributed data directories

OpenLink Virtuoso는 고성능 객체관계형 SQL 데이타베이스입니다. 이는 트랜잭 션, 스마트 SQL 컴파일러, 핫 백업, SQL:1999 지원, 서버 사이드 JAVA 또는 .NET 을 지원하는 강력한 저장 프로시저 언어를 제공합니다. 이는 ODBC, JDBC, ADO.NET, 그리고 OLE/DB를 포함한 모든 주요 데이타 엑세스 인터페이스를 제공합니다.

Virtuoso는 데이타베이스에 저장된 RDF 데이타 쿼리를 위해 SQL에 내장된 SPARQL 을 지원합니다. SPARQL은 SPARQL 자체 타입 캐스팅 규칙과 전용 IRI 데이타 유형 같은 엔진 그 자체에 저수준 지원의 혜택이 있습니다.

Virtuoso OSE ("Open-Source Edition")를 구성한는 완전한 패키지 세트를 위해 이 메타 패키지를 설치합니다.

VisiData is a multipurpose terminal utility for exploring, cleaning, restructuring and analysing tabular data. Current supported sources are TSV, CSV, fixed-width text, JSON, SQLite, HTTP, HTML, .xls, and .xlsx (Microsoft Excel).

NeXus is a common data format for neutron, X-ray, and muon science. It is being developed as an international standard by scientists and programmers representing major scientific facilities in Europe, Asia, Australia, and North America in order to facilitate greater cooperation in the analysis and visualization of neutron, X-ray, and muon data.

This is the package containing the development libraries.

NeXus is a common data format for neutron, X-ray, and muon science. It is being developed as an international standard by scientists and programmers representing major scientific facilities in Europe, Asia, Australia, and North America in order to facilitate greater cooperation in the analysis and visualization of neutron, X-ray, and muon data.

This is the package containing the java libraries.

The plfit software fits power-law distributions to empirical (discrete or continuous) data, according to the method of Clauset, Shalizi and Newman [SIAM Review 51, 661-703 (2009)].

This package contains the header files, static libraries and symbolic links that developers using the plfit library will need.

Openpyxl is a pure Python 3 module to read/write Excel 2007 (OpenXML) xlsx/xlsm files.

This package contains the module itself.

Modular Python implementation of t-Distributed Stochasitc Neighbor Embedding (t-SNE), a popular dimensionality-reduction algorithm for visualizing high-dimensional data sets. openTSNE incorporates the latest improvements to the t-SNE algorithm, including the ability to add new data points to existing embeddings, massive speed improvements, enabling t-SNE to scale to millions of data points and various tricks to improve global alignment of the resulting visualizations.

The plfit software fits power-law distributions to empirical (discrete or continuous) data, according to the method of Clauset, Shalizi and Newman [SIAM Review 51, 661-703 (2009)].

This package provides a Python module.