NAME

ROADMAP - A plan for Parrot

PURPOSE

As of this writing, there is no one place in which all remaining parrot work is documented. There are various TODO files, the RT queue, comments in the source code and the documentation. This document is a first pass at unifying this information and providing a single point of reference for anyone interested in the current state of the parrot, whether they are developers looking for items to work on, or external users looking for an idea of when they can expect a release.

This document is not meant to contain all the metadata regarding particular tasks, but merely provide a summary of the tasks.

This document is now becoming a roadmap, because the Architect is deep in a weeklong mindmeld with the Pumpking and the results are being encoded in this document, among others.

For a high level overview of parrot milestones, please see: http://www.perlfoundation.org/gc/grants/2005-p6-proposal.html

CURRENT RELEASE (0.3.0)

Parrot 0.3.0 was released on Oct. 1st, 2005. For a list of new and fixed feature see NEWS.

OVERVIEW

The remaining tasks are broken down into the following categories, roughly listed in the order through which a task might progress.

METADATA

Each task has certain metadata associated with it which must be tracked. Eventually all metadata should be stored in RT at https://rt.perl.org/rt3/

DESIGN

These are pending design decisions which must be made before any code can be written to implement them. Note - DESIGN is a special kind of documentation task, but due to the nature of this kind of documentation, it deserves to be broken out into its own category.

TODO

Any item, whether visible to parrot users or hidden behind an API, that has already been designed (but not implemented) belongs here.

BUGFIX

Any items which have been implemented but don't work as expected.

PORTABILITY

Any items which are specific to a particular platform or require special attention for cross-platform compatibility.

DOCUMENTATION

Any items pertaining to source, API, or user level documentation.

HIGH LEVEL LANGUAGES

Parrot currently ships with several partial language implementations. Any tasks which are specific to those languages, or affect cross-language compatibility will be listed here.

ISSUES

METADATA

• To be included: docs/pmc, docs/resources, docs/*.pod, anything in SVN not in docs, RT. (That is, quite a bit.)

[[What does this item mean, exactly? --Chip]]

The intention was to comb through the various sources of things to be done and collate them in some fashion. At the time of that writing, none of the tasks documented in those directories were documented in a central area. Something a little nicer than grep -r 'XXX', but that would certainly do for a start. Chromatic's recent email is another step on this path. --Coke

DOCUMENTATION

• Eliminate dead documentation. Let historians dig through our subversion history. We look forward toward successful deployment.
• When docs and code disagree, they must be brought into agreement immediately. At the very least, appropriately mark docs that are apparently wrong but may eventually become right, e.g.: "[NOTE: this does not currently work. Why? --committer]".

PDDs are not exceptions to this rule! Fix them when they're wrong. Delete them when they're obsolete. (Never delete docs lightly, though. Check first with someone who should know.)

• Document project goals and priorities, based on Chip's talk to APW as found at http://feather.perl6.nl/~chip/Chip_APW.sxi or http://feather.perl6.nl/~chip/Chip_APW.pdf.
• memory_internals.pod has a question about whether or not string reuse is limited to a single user or multiple users.

DESIGN

• MMD Roundup 2, from the mailing list:

http://xrl.us/gvfw (Link to groups-beta.google.com)

Section 1 of that email needs a decision from the Architect.

• Variable-length register frames are an approved feature, but the design still needs fleshing out. Then there are the minor details of documentation and implementation. :-,
• Calling conventions are definitely changing for the better, but not in a way that need change much of substance for language implementors. Chip/Leo mind meld is in progress; stay tuned.

BTW: Don't panic! Life is getting better for you, the Parrot user. It had already become obvious recently to us that Parrot needed a nice opcode-based interface for automated collection and extraction of call parameters and return values. That's what you were going to write to anyway. How it works, precisely, shouldn't matter much. (You were writing to PIR instead of PASM, right?)

• Design the standard library to replace all those opcodes that should never have been opcodes in the first place, e.g. the IO and transcendental math features.

(Note: Whatever tech we use to implement transcendental math should at *worst* compile to function calls under JIT. And I'll consider making exceptions for the transcendantal math if someone can show me why.)

I expect PIR will still let you pretend they're opcodes, though, to avoid immediate howls of outrage. (But only silicon-based life forms write PASM, so we're not worried about that.)

• Do detailed design on new Chip/Leo-agreed-in-principle lexical system.

The key change learns a lesson that Perl 5 only half learned: Lexical values may be dynamic, but their names and scopes are known at compile time. It's just silly to add '$x' to a newly created pad at runtime, over and over and over, every time the same scope is entered. Silly.

So the names and scopes will be static data attached to the subroutine, while the data will be in dynamically created pad entries.

Think of a pseudohash: the static data are like the index hash, which can be shared and immutable; the storage slots are like the rest of the array, which can be mutable even when the index hash isn't.

• Finish event and IO specification
• Finish specification of return continuation and sub being called slots in interpreter, with introspection
• Spec out bytecode metadata system and introspection/modification system
• Specify what we need to serialize continuations
• Final namespace spec
• Solidify design for Truly Evil speed hack for fast regex/string matching
• Describe the behavior of init_pmc_props. PDD02
• Describe the behavior of the serialization functions. (freeze, thaw, thawfinish, visit). PDD02
• Describe the behavior of share. PDD02
• Describe the components of a struct PMC_EXT. PDD04
• Define what bits of the Parrot internals should be exposed as part of the Parrot API for extension writers.

TODO

 requester: leo

Ok. In the long run I expect a multi-stage config system. First is to get miniparrot built and running, then Parrot core. Extensions like gdbm could be configured at that point or even in a third step.

• IO: Add IO tables, right now IO's just float around.
• Finish PMC for IO object
• IO's "stdio" layer: buffering, seek/random access, shared streams, thread safe
• Copy-on-write IO stacks
• Make IO layer cache the vtable calls to the next layer.
• IO Filters
• IO mmap support
• Bottom level socket/pipe/process support in API
• UTF layers (or should we not bother and do as a filter?)
• Sample programs and test suite
• AIO - On many platforms we must fake async IO, are we doing a background thread/interp or an inline event queue function that is called every N ops?
• Get digest (probably SHA1) added to bytecode segments
• Speed up the OrderedHash implementation
• Rearchitect internals for the string specifications.
• To PerlClass or not to PerlClass, that is the question. The class hierarchy should be reworked (e.g. Boolean->PerlInt->Scalar).
• Configuration - missing tests for processor features
• Configuration - duplication, e.g. nvsize/numvalsize
• Tests - ~1/3 of opcodes are uncovered by tests
• IO - Async core isn't there yet, even though it is rumoured to be.
• Overflow detection/promotion to bigint/float types.
• Create opcode_process.pl. PDD05
• Fill in the remaining functions from the EDA, verify that the test suite still passes, integrate the library into the rest of Parrot, provide PMC types and suitable opcodes. Conversion to and from Parrot strings, conversion to and from floating point types, sprintf output of bignumbers. PDD14

[[What's an EDA? --Coke]]

• Implement the parrot equivalent of perl5's $0, based on recent (January 2005) mailing list activity.
• load_bytecode does not search the library path to load the file. "core.ops" in ops
• The debugger needs to clone the interpreter to "allow people to play into the debugger and then continue the normal execution of the program." "debug.ops" in ops
• parrot doesn't catch the exception thrown by the op cmod. "math.ops" in ops
• rx_compile op is documented but unimplemented. "rx.ops" in ops
• There is no way to do a case-insensitive match. "rx.ops" in ops

BUGFIX

• You can add parents to a class with children and objects, and attributes to a class with children or objects. This is silently allowed and quietly corrupts the child classes and instantiated objects.
• building miniparrot is broken
• GC issue (copying collector) - immobile strings/buffers get freed during GC.
• Memory leak - PIO subsystem leaks file handles (ParrotIO structures)
• Memory leak - regexen leak bitmaps
• Memory leak - string_to_cstring returns a malloced string, which isn't freed everywhere
• Memory Leaks - imcc is currently leaking Unit structures and possibly more
• Regular Expressions - are non reentrant due to interpreter global intstack
• JIT - 64 bit integers on 32 bit machines don't work
• JIT - parrot register inspection is incomplete (P-regs)
• This sequence:

   loop: dec I0
         inc I1       # or add, sub, ...
         if I0, loop

isn't handled correctly on JIT/i386.

• Parrot_destroy currently leaks most of the memory that it is supposed to free. embed.pod

PORTABILITY

• Portable fcntl() replacement for platforms without it
• See PORTABILITY. Test suite should have no failure on supported platforms.
• Tests - src/tests are not supported on all platforms
• ParrotIO uses integral file descriptors, which may not be portable. "io.ops" in ops
• conversion ops for machines without 1/2/4 byte integer data types
• 32 bit pointer/64 bit integer and vv.
• dynamic library loading

HIGH LEVEL LANGUAGES

• Items in languages/tcl/TODO must be completed.
• Get python bytecode translator done
• Pass the pie-thon test suite.

FUTURE

This document should eventually be obsoleted by use of the RT ticketing system, except to the extent that it documents things that show promise but have not yet been committed to. Every task that is actually ready to be done should be documented in RT, and reports generated based on that information.

BUGS (in this document)

As for the future development, it has been pointed out that RT is not the best system for handling such things (though it does have the advantage of being available). The main point is to have single point of contact to help volunteers check (and update) the status of issues important to them.