NAME

Miniperl::Tokenizer - Miniperl6 tokenizer

SYNOPSIS

  use Miniperl::Tokenizer;
  my $tok = Miniperl::Tokenizer->new([$file_name]); # Optional file name
  $out_aref = $tok->tokenize([$string]); # Optional string

DESCRIPTION

The tokenizer takes either a text string (directly in tokenize()) or a file in the new() member function. This gets broken into perl6 tokens.

AUTHOR

Jeffrey Goff, drforr@hargray.com

SEE ALSO

parrot(1).