NAT::PTD - Module to handle PTD files in Dumper Format


  use NAT::PTD;
  $ptd = NAT::PTD->new( $ptd_file );


PTD files in Perl Dumper format are simple hashes references. But they use a specific structure, and this module provides a simple interface to manipulate it.


The new constructor returns a new NAT::PTD object. This constructor receives a PTD file in dumper format.

  $ptd = new NAT::PTD( $ptd_file );

If the filename matches with /dmp.bz2$/ (that is, ends in dmp.bz2) it is considered to be a bzip2 file and will be decompressed in the fly.

If it ends in <.sqlite>, then it is supposed to contain an SQLite file with the dictionary (with NATools standard schema!).


The dump method is used to write the PTD in its own format, but taking care to sort words lexicographically, and sorting translations by their probability (starting with higher probabilities).

The format is Perl code, and thus, can be used independetly of this module.

   $ptd -> dump;

Note that the dump method writes to the Standard Output stream.


The words method returns an array (not a reference) to the list of words of the dictionary: its domain.

   @words = $ptd -> words;


The trans method receives a word, and returns the list of its possible translations.

   @translations = $ptd -> trans( "dog" );


The transHash method receives a word, and returns an hash where keys are the its possible translations, and values the corresponding translation probabilities.

   %trans = $ptd -> transHash( "dog" );

Returns the empty hash if the word does not exist.


The prob method receives a word and a translation, and returns the probability of that word being translated that way.

   $probability = $ptd -> prob ( "cat", "gato");


Returns the total number of words from the source-corpus that originated the PTD. Basically, the sum of the count attribute for all words.

   $size = ptd -> size;


The count method receives a word and returns the occurrence count for that word.

   $count = $ptd -> count ("cat");

If no argument is supplied, returns the total dictionary count (sum of all words).


Computes a bunch of statistics about the PTD and returns them in an hash reference.


This method subtracts to the domain of a PTD, the elements present on a set of elements. This set can be defines as another PTD (domain is used), as a Perl array reference, as a Perl hash reference (domain is used) or as a Perl array (not reference). Returns the dictionary after domain subtraction takes place.

  # removes portuguese articles from the dictionary
  $ptd->subtractDomain( qw.o a os as. );
  # removes a set of stop words from the dictionary
  $ptd->subtractDomain( \@stopWords );
  # removes the words present on other_ptd from ptd
  $ptd->subtractDomain( $other_ptd );


This method recalculates all probabilities accordingly with the number of translations available.

For instance, if you have

    home => casa => 25%
         => lar  => 25%

The resulting dictionary will have

   home => casa => 50%
        => lar  => 50%

Note that this methods replaces the object.


This method intersects the current object with the supplied PTD. Note that this method replaces the object values.

Occurrences count in the final dictionary is the minimum occurrence value of the two dictionaries.

Only translations present on both dictionary are kept. The probability will be the minimum on the two dictionaries.


This method adds the current PTD with the supplied one. Note that this method replaces the object values.


This method iterates over a dictionary and calls the function supplied as argument. This function will receive, in each call, the word in the source language, the number of occurrences, and the hash of translations.

  $ptd -> downtr( sub { my ($w,$c,%t) = @_;
                        if ($w =~ /[^A-Za-z0-9]/) {
                            return undef;
                        } else {
                            return toentry($w,$c,%t);
               filter => 1);

Set the filter flag if your downtr function is replacing the original dictionary.


This function is exported by default and creates a dictionary entry given the word, word count, and hash of translations. Check downtr for an example.


Method to save a PTD in another format. First argument is the name of the format, second is the filename to be used. Supported formats are <dmp> for Perl Dump format, <bz2> for Bzipped Perl Dump format, <xz>, for Lzma xz Perl Dump format and <sqlite> for SQLite database file.

Return undef if the format is not known. Returns 0 if save failed. A true value in success.


NATools, perl(1)


Alberto Manuel Brandão Simões, <>


Copyright (C) 2008-2011 by Alberto Manuel Brandão Simões