Building Tools - Beyond Lilly

For building tools with more complex functionality than that prescribed by Lilly, a discussion of how the user code interacts with Lilith, handling communications within the tree.


For cases where the simple Lilly actions of a repeated distribution and result collection on the entire tree will not suffice, more direct contact with the Lilith system is required. In such cases, Lilith still provides services of data distribution, result collection and communication, but the user will have to invoke these services in the desired order, rather than having the Lilly base classes handle the invocation and the ordering.

More direct use of Lilith is still made easy through defining simple interfaces and a single object through which the user interacts with the entire Lilith framework.

Lilith Architecture
In order to understand and use Lilith, it is not necessary to be conversant with the full Lilith Distributed Object System. This section, then covers only those objects with which the user has to be directly concerned.

Lilim are the Lilith Objects that carry user code within them. Lilim, as well as the core Lilith objects, export well known interfaces which allow the user code to interact with the Lilith environment. The Lilim run as threads under the Lilith Hosts. Lilith-based tools are created by construction of suitable Lilim. The Lilim need only be written with the goal of the tool in mind; details of code distribution and communication between nodes in the tree are handled by objects in the Lilith environment.

LilithHosts maintain the tree and, via lower level objects, comunicate with one another. The LilithHost object is responsible for protecting the computing resource on which it is running from other Lilith objects and for instantiating Lilith objects on that host. For purposes here, we define a host to be a system running under a single OS. If Lilith is being used simply as a platform for remote objects there may be more than one LilithHost per host.

Lilim interact with the Host Object and each other through the Lilim-Implementation-CommunicationsObject, a.k.a. LICO. Thus each node of the tree consists of a LilithHost with a Lilim running on it, and a LICO for communications between the two. The LICO provide a well defined set of methods by which Lilim can send data up and down to other Lilim in the tree. LICO pass arguments to the Lilim and gather up their return messages for passage back up the tree. By compartmentalizing the interactions of Lilim with Lilith in this way, not only is the entry-level knowledge needed to use Lilith small but also access to Lilith Objects by a potentially malicious user is contained. For instance, rogue Lilim cannot by direct call get illegal control of the lower level objects in the system which provide socket access. This restriction is enforced through Java Package assignments and checks on the sequence of classes in the execution stack whenever the SecurityManager is invoked.

Communications are handled through the sending of MessageObjects, a.k.a., MOs. The MO is a general purpose data rack that can hold a list of data objects.

Intra-tree Communications in Lilith

MO get(String) returns an MO from LICO with MOUUID equal to the String value
void put(MO) puts MO into LICO
void scatterToChildren(MO[]) scatters MO[] to children
MO[] gatherFromChildren(String) returns MOs from children corresponding to MOUUID equal to the String value
MO getArg(String) returns initial MO sent down with Lilim with MOUUID equal to the String value
The LICO provides a simple set of methods through which Lilim communicate with each other up and down the tree.

The methods get() and scatterToChildren() are used as a pair to get messages from the parent Lilim and send messages down to the child Lilim. (Although the terms "parent" and "child" are more accurately used in terms of the LilithHosts which maintain the tree, the extension of this terminology to the Lilim residing on those hosts is straightforward and unambiguous.) A Lilim sends a message to its children via LICO.scatterToChildren() using an MOUUID tagged MO as its argument. This method puts each MO into the LICO corresponding to each child. A Lilim gets an MO from its LICO via LICO.get() using the appropriate MOUUID tag as the argument. Messages can then be sent recursively down the tree by each Lilim first calling LICO.get(myTAG) and then calling scatterToChildren(myMO) where myMO has been tagged with the same tag.

The call getArg() is similar to that of get() and is used by a Lilim to get data, in the from of an MO, initially sent down with the Lilim bytecode. Sending this information down with the Lilim reduces the number of messages required.

Methods involved in sending data down the tree.

Processing the results up the tree is also performed recursively. In this case the methods put() and gatherFromChildren() are used. A Lilim calls gatherFromChildren() with an MO tag as its argument to receive an MO array containing all such tagged results from its children's LICOs. This call blocks execution in the Lilim until returns from all the children have come into the parent Lilim's LICO. That same Lilim then makes its own results available to tis own parent by calling LICO.put() with an argument of its own identically tagged MO.
Methods involved in returning data back up the tree.

Note that a child Lilim calls LICO.put() in anticipation of the parent calling gatherFromChildren() to collect that MO. Thus both the processes of sending messages down the tree and of gathering returns back up the tree are initiated by the parent. Communications both down and up the tree are only with the levels in the tree directly above or below the current level.

The recursive up and down calls and tagging are illustrated in an example using a distributed sort.

In this case the downward processing will consist of: each Lilim getting a list of numbers to sort, subdividing that list into pieces for itself and its children to sort, and then sending those pieces down to its children. After a Lilim has sorted its own piece, it then processes the results back up the tree by: gathering the children's sorted list, combining their results with its own via a merge sort, and then passing the combined sorted list up to its parent. The relevant pseudo-code is as follows:

public class Lsorter implements Lilim{
private LICO myLICO; /* field for LICO with which this Lilim communicates.*/


public void run(){
MO tmpMO = myLICO.get(TAG1); /* gets an MO from the LICO containing the list of numbers to be sorted */

... /* code here which:
1) Unpacks MO to get array of numbers to sort via calls to tmpMO.pullInt()
2) Divides array into subarrays for self and children
3) Packs arrays for children into MO[] kids_piecesMO via calls to kids_piecesMO[i].pushInt(int)
4) Set MOUUIDs on each MO to TAG1 via kids_piecesMO[i].setMOUUID(TAG1)

myLICO.scatterToChildren(kids_piecesMO); /* scatters MOs with arrays for children to sort */
sort(myArray); /* sort own piece using own sort method */
kids_piecesMO = myLICO.gatherFromChildren(TAG2); /* gather MOs containing sorted arrays from children */

... /* code here which unpacks sorted arrays from kids_piecesMO and places them into kidsArrays */

final array = mergeSort(myarray,kidsArrays); /* merge sort all sorted arrays */

... /* code here which:
1) Packs final array into tmpMO
2) Sets MOUUID tag on tmpMO to TAG2

myLICO.put(tmpMO); /* put MO containing final sorted array into LICO for parent to gather */

In the above example, the packing and unpacking of messages is not explicitly shown - these are straightforward calls to push/pullInt() and setMOUUID(). The key thing to note is the usage of the tags in the operation of the recursive calls. TAG1 is used to obtain the correct MO from the parent via get(), and is therfore also used to make the MO sent from the parent in scatterToChildren(); thus TAG1 is used for signaling in sending the messages down the tree. Similarly, TAG2 is used in put() and gatherFromChildren() on sending returns back up the tree.

Many tools can be written using this basic structure. The sequence of calls to scatter information down the tree and gather it back up, as well as the tagging, can be reused unchanged, with the action tailored to the specific tool. This is, in fact, the structure that Lilly is designed to. LillyLilim contains code that handles the scatterToChildren(), gatherFromChildren(), get(), and put() calls and takes care of the MOUUID matching. It invokes the methods specified by the Lilly Interface at the appropriate phases of initialization, data distribution, and result collection.