Pynie - A Python compiler for Parrot

This is a Pynie parser/compiler, an early version (no version numbers yet). It's still very early, only simple expressions and functions are available.

However, even though this is not a complete compiler yet, you can still see how programs are parsed, help us create test cases, and extend/improve the grammar and runtime to cover more of Python. Here's how the system currently works:

Compiling

Pynie lives in pynie.pbc. To create this file, simply issue the command

  $ make

To invoke pynie on a Python program from a shell prompt, use:

  $ parrot pynie.pbc foo.py

To run interactively, entering single-line statements:

  $ parrot pynie.pbc

Note that pynie's interactive mode is not yet up to spec with Python's interactive mode (it doesn't handle blocks yet).

To display the parse tree, add the "--target=parse" option:

  $ parrot pynie.pbc --target=parse foo.py

Or, to display the abstract syntax tree, the opcode syntax tree, or the generated PIR code, use "--target=PAST", "--target=POST", or "--target=PIR".

  $ parrot pynie.pbc

Files

The "top" file for the parser is pynie.pir which is used to create the pynie.pbc file. It initializes the overall parsing system and registers the parser as a Parrot "Pynie" compiler.

The other files needed for the compiler are in the src/ subdirectory.

The src/parser/Grammar.pg file defines the tokens, rules, and protos for Python. Much of this has been taken from the "Python Language Reference Manual (release 2.3)" by Guido van Rossum (Fred L. Drake, Jr., editor). PGE's "Perl6Grammar.pbc" compiler is then used to compile the two grammar files into src/parser/Grammar_gen.pir, which is included by pynie.pir. (See Synopsis 5 for more details on Perl 6 rules syntax, and compilers/pge/ for more details about PGE.)

The src/parse.pir file defines a few special-purpose rules needed to support parsing that are better written directly in PIR instead of using the rules or token syntax. Currently this file defines the <expression> rule, which just calls into the operator precedence parser.

The file src/parser/Actions.pm defines the actions that are invoked during the parse. These actions construct the Parrot AST (PAST) nodes. At the end of the parse, the whole PAST is constructed and handed off to the next compilation phases (PAST->POST->PIR)

The file src/PAST/Grammar.tg is a tree grammar that specifies how to convert the parse tree into the abstract syntax tree (PAST). This file is no longer used in the current implementation.

The PIR files in src/builtins are included as part of compiling pynie.pir to produce pynie.pbc.

The pynie.pbc file can also be used to compile Python code from PIR:

    load_bytecode 'pynie.pbc'
    $S0 = 'print "hello world"'          # source code to compile
    $P0 = compreg('Pynie')               # obtain the compiler
    $P1 = $P0($S0)                       # compile source code
    $P1()                                # execute

One can also provide the "target" option to the compiler:

    $P1 = $P0($S0, 'target'=>'parse')    # obtain parse tree
    $P1 = $P0($S0, 'target'=>'PAST')     # get AST
    $P1 = $P0($S0, 'target'=>'POST')     # get OST
    $P1 = $P0($S0, 'target'=>'PIR')      # get PIR

Submitting Bugs/Patches

Patches/bugs/suggestions can be sent to <http://launchpad.net/pynie>.

AUTHOR

Patrick Michaud <pmichaud@pobox.com> is the current author and maintainer.