Debian Science Linguistics packages

Um motor de tradução para máquinas de transferência superficial, o Apertium é inicialmente voltado a pares de idiomas relacionados.

Ele usa transdutores de estado finito para processamento léxico, modelos Markov ocultos para marcação parte de fala, e pedaços baseados em estado finito para transferência estrutural.

O sistema é em grande parte baseado em sistemas já desenvolvidos pelo grupo Transducers da Universitat d'Alacant, tais como interNOSTRUM (espanhol-catalão, http://www.internostrum.com/welcome.php) e Traductor Universia (espanhol-português, http://traductor.universia.net).

Será possível usar o Apertium para construir sistemas de tradução de máquinas para uma variedade de pares de idiomas relacionados simplesmente fornecendo os dados linguísticos necessários no formato correto.

This package contails Perl scripts to evaluate Apertium-based machine translation output against reference: WER, PER, TER, BLEU.

Module for compiling lexical selection rules and processing them in the pipeline.

Artha é um thesaurus do idioma inglês com recursos distintos como:

• tecla de atalho para consulta de palavra (selecione um texto em qualquer janela e pressione a tecla de atalho pré-configurado para fazer a consulta)
• pesquisa baseada em expressões regulares (pesquisa mais compreensiva usando curingas como *, ?, etc.)
• notificações passivas na área de trabalho (de definições de palavras para fluxo de trabalho ininterrupto)
• sugestões de ortografia (quando a forma exata é vaga/desconhecida)

Uma vez executado, ele monitora uma combinação pré-definida de teclas de atalho. Quando algum texto é selecionado em qualquer janela e a tecla de atalho é pressionada, uma janela instantânea é aberta com a palavra consultada. Se o usuário preferir notificações passivas, isto pode ser feito habilitando essa opção.

Quando o termo consultado é vago/desconhecido, a pesquisa pode ser expandida com o uso de expressões regulares (*, ?, etc.) na string de busca ou temos sugestões de ortografia quando um termo está incorreto.

Para que a pesquisa baseada em expressões regulares funcione, o pacote wordnet-sense-index é requerido.

Constraint Grammar compiler and applicator for the 3rd edition of CG that is developed and maintained by VISL SDU and GrammarSoft ApS.

CG-3 can be used for disambiguation of morphology, syntax, semantics, etc; dependency markup, target language lemma choice for MT, QA systems, and much more. The core idea is that you choose what to do based on the whole available context, as opposed to n-grams.

See https://visl.sdu.dk/cg3.html for more documentation

Collatinus can be used to lemmatise latin texts, i.e. extract words and make a lexicon which indicates for each word its canonic form, and how the form actually found in the text was derived from it, for instance by declining it. Example : rosam gives : rosa-rosae -- acc. sing. Collatinus provides a nice graphic front-end to each operation.

Collatinus-nouus (stands for Collatinus, new generation) replaces every previous version of Collatinus.

This package provides a documentation in HTML format.

Dimbl é um envoltório ao redor do mais próximo vizinho classificador em TiMBL, oferecendo classificação paralela em máquinas com múltiplos processadores. Ele divide o conjunto de formação original, constrói subconjuntos de formação por classificadores TiMBL separados e mescla seus conjuntos de vizinhos mais próximos através de instâncias classificadas.

Os recursos do Dimbl são:

• Envolve ordenadamente ao redor do TiMBL, mantendo todas as opções de linha de comando;

• Sabe o que fazer com seus dois, quatro ou múltiplos núcleos;

• Utiliza a especificação OpenMP para programação paralela;
• Pode atingir ganhos de velocidade superlineares comparado com o padrão TiMBL.

Dimbl é um produto do Grupo de Pesquisa ILK (Universidade de Tilburg, Holanda).

Se você realiza pesquisa científica em processamento de linguagem natural utilizando a técnica de aprendizagem baseada em memória, Dimbl provavelmente será de utilidade para você.

fastText is a library for efficient learning of word representations and sentence classification, which refers subword information to enrich word vectors.

Memory-Based Learning (MBL) is a machine-learning method applicable to a wide range of tasks in Natural Language Processing (NLP).

Frog is a modular system integrating a morphosyntactic tagger, lemmatizer, morphological analyzer, and dependency parser for natural languages. It is based upon it's predecessor TADPOLE (TAgger, Dependency Parser, and mOrphoLogical analyzEr). Using Memory-Based Learning techniques, frog tokenizes, tags, lemmatizes, and morphologically segments word tokens in incoming UTF-8 text files, and assigns a dependency graph to each sentence. Frog is particularly targeted at the increasing need for fast, automatic NLP systems applicable to very large (multi-million to billion word) document collections that are becoming available due to the progressive digitization of both new and old textual data. Up to now, frog has only been tested and used using corpora of Dutch natural language (see the frogdata package for samples).

Frog is a product of the Centre of Language and Speech Technology at Radboud University Nijmegen, it subsumes previous work by the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium). It is currently maintained at the KNAW Humanities Cluster.

If you do scientific research in NLP, Frog will likely be of use to you.

Data package providing Giellatekno language resources for North Saami.

The Helsinki Finite-State Transducer software is intended for the implementation of morphological analysers and other tools which are based on weighted and unweighted finite-state transducer technology.

Minimal HFST optimized lookup format based spell checker library and a demonstrational implementation of command line based spell checker.

The IRST Language Modeling Toolkit can be used to learn a language model from data. The generated n-gram models should be usable on any system supporting ARPA language model format.

This package provides the command line tools.

Detects over 80 languages in UTF-8 text, based largely on groups of four letters. Also tables for 160+ language version.

This is the development package.

Em Sleator, D. e Temperley, D., "Parsing English with a Link Grammar" (1991) (Verificando o Idioma Inglês com uma Gramática de Ligações), os autores definiram um novo sistema gramatical formal chamado de "gramática de ligação" (link grammar). Uma sequência de palavras está na linguagem de uma gramática de ligação se houver uma maneira de traçar ligações entre palavras de tal forma que os requisitos locais de cada palavra sejam satisfeitos, que os vínculos não se cruzem e que as palavras formem um gráfico conectado. Os autores codificaram uma gramática inglesa em um sistema e escreveram este programa para verificar * idioma inglês usando essa gramática.

link-grammar pode ser usado para verificação linguística em recuperação ou extração de informações de documentos em linguagem natural. Também pode ser usado como verificador de gramática.

Este pacote contém o binário executável pelo(a) usuário(a).

The lttoolbox contains the augmented letter transducer tools for natural language processing used by Apertium, a platform for building rule-based and hybrid machine translation systems. The software is also useful for making morphological analysers and generators for natural language processing applications.

MBT is a memory-based tagger-generator and tagger in one. The tagger-generator part can generate a sequence tagger on the basis of a training set of tagged sequences; the tagger part can tag new sequences. MBT can, for instance, be used to generate part-of-speech taggers or chunkers for natural language processing. Features:

• Tagger generation: tagged text in, tagger out,
• Optional feedback loop: feed previous tag decision back to input of next decision,
• Easily customizable feature representation; can incorporate user-provided features,
• Automatic generation of separate sub-taggers for known words and unknown words,
• Can make use of full algorithmic parameters of TiMBL.

MBT is a product of the Centre of Language and Speech Technology (Radboud University Nijmegen, The Netherlands), the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in natural language processing, MBT will likely be of use to you.

MbtServer extends Mbt with a server layer, running as a TCP server. Mbt is a memory-based tagger-generator and tagger for natural language processing. MbtServer provides the possibility to access a trained tagger from multiple sessions. It also allows one to run and access different taggers in parallel.

MbtServer is a product of the Centre for Language and Speech Technology (Radboud University, Nijmegen, The Netherlands), the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in natural language processing, MbtServer will likely be of use to you.

The Apache OpenNLP library is a machine learning based toolkit for the processing of natural language text. It supports the most common NLP tasks, such as tokenization, sentence segmentation, part-of-speech tagging, named entity extraction, chunking, parsing, and coreference resolution. These tasks are usually required to build more advanced text processing services. OpenNLP also included maximum entropy and perceptron based machine learning.

This package contains the command line wrapper.

PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language models. It also contains complex data types and algorithms. Moreover, it includes parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL) and clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).

This is the Python 3 version.

Thinc is the machine learning library powering spaCy https://spacy.io. It features a battle-tested linear model designed for large sparse learning problems, and a flexible neural network model under development for spaCy v2.0 https://spacy.io/usage/v2.

Thinc is a practical toolkit for implementing models that follow the "Embed, encode, attend, predict" architecture. It's designed to be easy to install, efficient for CPU usage and optimised for NLP and deep learning with text – in particular, hierarchically structured input and variable-length sequences.

An R implementation of the LexRank algorithm implementing stochastic graph-based method for computing relative importance of textual units for Natural Language Processing. The technique on the problem of Text Summarization (TS) is tested. Extractive TS relies on the concept of sentence salience to identify the most important sentences in a document or set of documents. Salience is typically defined in terms of the presence of particular important words or in terms of similarity to a centroid pseudo-sentence.

An R interface to the C libstemmer library that implements Porter's word stemming algorithm for collapsing words to a common root to aid comparison of vocabulary. Currently supported languages are Danish, Dutch, English, Finnish, French, German, Hungarian, Italian, Norwegian, Portuguese, Romanian, Russian, Spanish, Swedish and Turkish.

SentencePiece is an unsupervised text tokenizer/detokenizer mainly designed for Neural Network-based text generation systems where the vocabulary size is predetermined prior to the neural model training.

Memory-Based Learning (MBL) is a machine-learning method applicable to a wide range of tasks in Natural Language Processing (NLP).

The Tilburg Memory Based Learner, TiMBL, is a tool for NLP research, and for many other domains where classification tasks are learned from examples. It is an efficient implementation of k-nearest neighbor classifier.

TiMBL's features are:

• Fast, decision-tree-based implementation of k-nearest neighbor classification;

• Implementations of IB1 and IB2, IGTree, TRIBL, and TRIBL2 algorithms;

• Similarity metrics: Overlap, MVDM, Jeffrey Divergence, Dot product, Cosine;

• Feature weighting metrics: information gain, gain ratio, chi squared, shared variance;

• Distance weighting metrics: inverse, inverse linear, exponential decay;

• Extensive verbosity options to inspect nearest neighbor sets;
• Server functionality and extensive API;
• Fast leave-one-out testing and internal cross-validation;
• and Handles user-defined example weighting.

TiMBL is a product of the Centre of Language and Speech Technology (Radboud University, Nijmegen, The Netherlands), the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in NLP, timbl will likely be of use to you.

timblserver is a TiMBL wrapper; it adds server functionality to TiMBL. It allows TiMBL to run multiple experiments as a TCP server, optionally via HTTP.

The Tilburg Memory Based Learner, TiMBL, is a tool for Natural Language Processing research, and for many other domains where classification tasks are learned from examples.

TimblServer is a product of the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in NLP, TimblServer will likely be of use to you.

Ucto can tokenize UTF-8 encoded text files (i.e. separate words from punctuation, split sentences, generate n-grams), and offers several other basic preprocessing steps that make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation.

This package provides the command-line tool itself.

Ucto was written by Maarten van Gompel and Ko van der Sloot. Work on Ucto was funded by NWO, the Netherlands Organisation for Scientific Research, under the Implicit Linguistics project, the CLARIN-NL program, and the CLARIAH project.

Ucto is a product of the Centre of Language and Speech Technology (Radboud University Nijmegen), and previously the ILK Research Group (Tilburg University, The Netherlands).

If you are interested in machine parsing of UTF-8 encoded text files, e.g. to do scientific research in natural language processing, ucto will likely be of use to you.

Ucto can tokenize UTF-8 encoded text files (i.e. separate words from punctuation, split sentences, generate n-grams), and offers several other basic preprocessing steps that make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation.

This package provides necessary language-specific datafiles for running Ucto.

Ucto was written by Maarten van Gompel and Ko van der Sloot. Work on Ucto was funded by NWO, the Netherlands Organisation for Scientific Research, under the Implicit Linguistics project, the CLARIN-NL program, and the CLARIAH project.

Ucto is a product of the Centre of Language and Speech Technology (Radboud University Nijmegen), and previously the ILK Research Group (Tilburg University, The Netherlands).

WordNet(C) is an on-line lexical reference system whose design is inspired by current psycholinguistic theories of human lexical memory. English nouns, verbs, adjectives and adverbs are organized into synonym sets, each representing one underlying lexical concept. Different relations link the synonym sets.

WordNet was developed by the Cognitive Science Laboratory at Princeton University under the direction of Professor George A. Miller (Principal Investigator).

WordNet is considered to be the most important resource available to researchers in computational linguistics, text analysis, and many related areas.

Binary and manpages of WordNet as well as general manpages.

This is a transitional dummy package. It can safely be removed.

This package contains Apertium APY which is simple Apertium API written in Python 3 meant as a drop-in replacement for ScaleMT.

Data package providing Apertium language resources for Aragonese

Data package providing Apertium language resources for translating between the Aragonese and Catalan languages.

Data package providing Apertium language resources for Belarusian.

Data package providing Apertium language resources for translating between the Belarusian and Russian languages.

This is a linguistic package for the Apertium shallow-transfer machine translation system. The package can be used to translate between Breton and French.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for Catalan.

Data package providing Apertium language resources for translating between the Catalan and Sardinian languages.

Data package providing Apertium language resources for Crimean Tatar

Data package providing Apertium language resources for translating between the Crimean Tatar and Turkish languages.

Data package providing Apertium language resources for translating between the Welsh and English languages.

Data package providing Apertium language resources for Danish.

Data package providing Apertium language resources for translating from the Danish to the Norwegian Nynorsk/Norwegian Bokmål variants and from Danish to Norwegian Nynorsk.

This is a transitional dummy package. It can safely be removed.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for translating between the English and Galician languages.

Data package providing Apertium language resources for translating between the Esperanto and Catalan languages.

This is a linguistic package for the Apertium shallow-transfer machine translation system. The package can be used to translate between Esperanto and English.

Data package providing Apertium language resources for translating between the Esperanto and Spanish languages.

Data package providing Apertium language resources for translating between the Esperanto and French languages.

This is a transitional dummy package. It can safely be removed.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for translating between the Spanish and Galician languages.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for translating between the Spanish and Portuguese languages.

Data package providing Apertium language resources for translating between the Spanish and Romanian languages.

Data package providing Apertium language resources for translating between the Basque and English languages.

Data package providing Apertium language resources for translating between the Basque and Spanish languages.

Data package providing Apertium language resources for translating between the French and Spanish languages.

Data package providing Apertium language resources for French.

Data package providing Apertium language resources for translating between the French and Catalan languages.

Data package providing Apertium language resources for Serbo-Croatian.

Data package providing Apertium language resources for translating between the Serbo-Croatian and English languages.

Data package providing Apertium language resources for translating between the Serbo-Croatian and Macedonian languages.

Data package providing Apertium language resources for translating between the Serbo-Croatian and Slovenian languages.

Data package providing Apertium language resources for Hindi.

This is a transitional dummy package. It can safely be removed.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for Icelandic.

Data package providing Apertium language resources for translating between the Icelandic and English languages.

Data package providing Apertium language resources for Italian.

Data package providing Apertium language resources for Kazakh

Data package providing Apertium language resources for translating between the Kazakh and Tatar languages.

This is a transitional dummy package. It can safely be removed.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for translating between the Maltese and Arabic languages.

Data package providing Apertium language resources for Norwegian Nynorsk.

Data package providing Apertium language resources for translating between the Norwegian Nynorsk and Norwegian Bokmål languages.

Data package providing Apertium language resources for Norwegian Bokmål.

Data package providing Apertium language resources for translating between the Occitan and Catalan languages.

Data package providing Apertium language resources for translating between the Occitan and Spanish languages.

Data package providing Apertium language resources for Occitan.

Data package providing Apertium language resources for Polish.

Data package providing Apertium language resources for translating between the Polish and Silesian languages.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for translating between the Portuguese and Galician languages.

Data package providing Apertium language resources for Russian

Apertium module for reordering separable/discontiguous multiwords.

Data package providing Apertium language resources for translating between the Northern Sami and Norwegian Bokmål languages.