|parrotcode: Freeze/Thaw Design Notes|
|Contents | Documentation|
docs/dev/pmc_freeze.pod - Freeze/Thaw Design Notes
This document describes freeze/thaw internals version 0.1. This is not the final implementation.
Freezing or serializing arbitrary PMCs is an interesting problem. Aggregates can hold other aggregates and can be deeply nested, so so a recursive approach could easily blow the stack, especially on embedded systems. Also, aggregates can be self-referential -- they can hold pointers to themselves -- so that working on such structures could create infinite loops.
Although the file is named pmc_freeze.c it ultimately will deal with every kind of operation that deeply traverses an arbitrary data structures. For example:
freeze, possibly called from an exception handler or on resource shortage before interpreter shutdown, to save some data before dying. It must not consume any additional resources.
freeze: reconstruct all PMCs to generate an identical copy of the original frozen PMC. As with
freeze, can be called from user code.
dclone(p)is basically the same as
The basic scheme of operation looks like this:
info = init() push todo_list, pmc while (todo_list) current = shift todo_list current->visit(info) done.
This structure holds all necessary information and function pointers specific to the desired functionality. It gets passed on to all vtable methods and callback functions.
These are labeled visit_loop_*. There are currently two schemes to handle the todo_list.
This is done by a callback function inside the visit_info structure called visit_pmc_now. It gets called initially to put the first item on the list and is called thereafter from all PMCs for contained PMCs inside the visit vtable method.
There is another callback visit_pmc_later which adds PMCs to the todo list for later processing, but doesn't do any action on these immediately.
The general scheme above shows that this method is called for all items on the todo_list. visit has to call visit_pmc_now for all contained PMCs, which then get visited until all is done.
The basic operation is:
(seen, id) = was_already_seen(pmc) do_specific_action(pmc, seen, id) if (!seen) pmc->visit_action()
As stated in the introduction structures can be self-referential, they can contain (at an arbitrary depth) PMCs, that were already processed. Just following these PMCs would lead to endless loops. So already seen PMCs have to be remembered.
So after all we finally arrived at the point to actually perform the desired functionality. First the PMC-specific part is done inside pmc_freeze.c then the specific vtable method freeze, thaw, whatever, is called, again via a function pointer called visit_action.
As stated PMCs are currently processed inside the core, PMC-specific parts are done by calling the PMCs vtable method. This parts could of course be moved to default.pmc too, so that it's simpler to override the functionality.
During initialization the visit_infos image_io data pointer is filled with an object having vtable methods that remarkably look like a PMCs vtable. So io->vtable->push_integer spits out an INTVAL to the frozen image, while shift_integer gets an INTVAL from the frozen stream.
This simplifies final changes when image_io becomes just a PMC of some serializer class. There are currently two serializers:
PMC ids ranging from 1 to N-PMCs are shifted left by two, so that the 2 lo bits can serve as flags:
id + 0x1 ... PMC was seen id + 0x2 ... PMC has same type as previous PMC id + 0x3 ... escape flag
A PMCs image generally looks like:
The text representation of the array
P0 = [P1=666, P2=777, P0]
may look like:
0xdf4 30 3 0xdf8 33 666 0xdf2 777 0xdf5 0xdf4 ... PMC id (with "0x" in front for clarity) 30 ... enum_class_ResizablePMCArray 3 ... elements count 0xdf8 ... id of first element 33 ... enum_class_Integer 666 ... value 0xdf2 ... id of second element, same type as prev element 777 ... value 0xdf5 ... id of array itself with lo bit set
The escape flag marks places in the image, where additional data will follow. After the escape flag is an int defining the kind of the following data, passed on in extra_flags. During thaw the PMCs vtable is called again, to restore these data. So a PMCs thaw vtable has to check extra_flags if normal or extra data have to be shifted from the image.
This is e.g. needed for PMC properties or arrays containing sparse holes, to set the array index of the following data.
A Integer(666) with a property hash ("answer"=>42) thus looks like:
0xdfc 33 666 0xdff 2 0xdf4 32 1 answer 0xdf8 33 42
0xdff is the escape mark for the PMC 0xdfc followed by the constant EXTRA_IS_PROP_HASH.
[ To be continued ]