[DRAFT] PDD10: Embedding and Extending

Abstract

What we believe people will do when embedding and extending Parrot, why they do it, and how.

{{ NOTE: some of this will later move into pdds 11 & 12, but for now just want to get the stub checked in. }}

Version

$Revision$

Description

Why embed:

• access to special features/libraries/languages Parrot provides
• need an interpreter for a DSL or existing language
• want to run Parrot on another platform or environment (dedicated hardware, in a web server, et cetera)

Why extend:

• need something NCI doesn't provide
• writing a custom PMC

Philosophical rules:

• only ever use opaque pointers
• should be able to communicate through PMCs
• minimize conversions to and from C data
• perhaps macros; Ruby does this fairly well and Perl 5 does this poorly
• minimize the number of necessary functions
• probably can follow core Parrot code to some extent, but beware the Perl 5 problem
• do not expose Parrot internals that may change
• minimize the number of headers used
• minimize the number of Parrot types exposed
• follow boundaries similar to those of PIR where possible
• probably includes vtable functions on PMCs

Gotchas:

• who handles signals?
• who owns file descriptors and other Unix resources?
• is there an exception boundary?
• namespace issues -- especially key related
• probably a continuation/control flow boundary
• packfiles and subroutines probably too much information for either
• do not let MMD and other implementation details escape
• okay to require some PBC/PIR/PASM for handling round-trip data
• Parrot should not spew errors to STDERR when embedded
• who allocates and deallocates resources passed through the boundary level?
• should be access to Parrot's event loop when embedded
• passing var args to Parrot subs likely painful
• perhaps macros/functions to add parameters to call
• build up a call signature somehow?
• some abstraction for a call frame?
• compiling code from a string should return the PMC Sub entry point (:main)
• are there still directory path, loading, and deployment issues?
• how do dynamic oplibs and custom PMCs interact?
• what's the best way to handle character sets and Unicode?

Definitions

Embedding - using libparrot from within another program, likely with a C/NCI/FFI interface

Extending - writing Parrot extensions, likely through C or another language

In practice, there is little difference between the two; mostly in terms of who has control. The necessary interfaces should stay the same.

Implementation

Implementation details.

Simplicity is the main goal; it should be almost trivial to embed Parrot in an existing application. It must be trivial to do the right thing; the APIs must make it so much easier to work correctly than to make mistakes. This means, in particular, that:

• it should never be possible to crash or corrupt the interpreter when following the interface as documented
• each API call or element should have a single purpose
• names must be consistent in the API documentation and the examples
• it should be possible to embed Parrot within Parrot through NCI, as a test both of the sanity of the external interface as well as NCI

Working with Interpreters

It is the external code's duty to create, manage, and destroy interpreters.

Parrot_new( NULL ) returns an opaque pointer to a new interpreter:

  Parrot_Interp Parrot_new(Parrot_Interp parent);

parent can be NULL for the first interpreter created. All subsequent calls to this function should pass an existing interpreter.

Note: it is not clear what happens if you fail to do so; is there a way to detect this in the interface and give a warning?

Parrot_destroy ( interp ) destroys an interpreter and frees its resources.

  void Parrot_destroy(Parrot_Interp);

Note: It is not clear what happens if this interpreter has active children.

Working with Source Code and PBC Files

Perhaps the most common case for working with code is loading it from an external file. This may often be PBC, but it must also be possible to load code with any registered compiler. This must be a single-stage operation:

  Parrot_PMC Parrot_load_bytecode( Parrot_Interp, const char *filepath );

  Parrot_PMC Parrot_load_hll_code( Parrot_Interp, const char *compiler,
                                                  const char *filepath );

The PMC returned will be the Sub PMC representing the entry point into the code. That is, it will be the PMC representing the :main subroutine, if one exists, or the first subroutine in the file.

If there is an error -- such that the file does not exist, the compiler is unknown, or there was a compilation or invalid bytecode error -- the PMC should be an Exception PMC instead.

Note: I suppose NULL would work as well; it might be more C-like. Continue considering.

Note also: the current Parrot_pbc_read() and Parrot_pbc_load() exposes the details of packfiles to the external API and uses two operations to perform a single logical operation.

Note: it may be worth reconsidering these names, if Parrot_load_bytecode() can load PBC, PIR, and PASM files without having a compiler named explicitly.

Compiling source code generated or read from the host application is also possible:

  Parrot_PMC Parrot_compile_string( Parrot_Interp, Parrot_String compiler,
                                                   const char *code,
                                                   Parrot_String error );

The potential return values are the same as for loading code from disk.

Note: this declaration should move from interpreter.h to embed.h.

Working with PMCs

TBD.

Calling Functions

TBD.

Calling Opcodes

TBD.

Language Notes

It should be possible to register a compiler for an HLL with an interpreter such that it is possible to load source code written in that language or pass source code to an interpreter successfully.

References

None.

Copyright © 2002-2010, Parrot Foundation.