Building Tools - Using Lilly


Lilly - an easy way to build tools with Lilith.

Bookmarks:



Preface
The Lilly classes make tool building with Lilith easy. They define a simple interface and default actions for the client and the user code (Lilim) on the tree nodes for initialization, data distribution, and result collection.

The Lilly classes provide simple interface and default actions for the client and user code (Lilim) on the tree nodes for initialization, data distribution, and result collection.


Lilith-based tools are easy to develop since users are only responsible for writing their user tool code action on a single node, with Lilith handling the details of the user code (Lilim) distribution and result collection. In many cases, the user will want services from Lilith in a simple pattern: data distribution down the entire tree, execution of the tool action on each node of the tree, and result collection up the entire tree. Lilith makes writing tools with this simple behavior particularly easy by providing two classes specially designed for this case. A special Lilim, called LillyLilim, is instantiated at each node of the tree and handles the action of the tool on the node and any manipulation required on the distribution of data and returning of results. Class LillyClient handles interactions with the root of the tree. These classes have simple, well defined interfaces for methods involved in the data distribution, action on the node, and result collection, with default behaviors specified. The user extends these classes, overriding those methods when necessary. The Lilly classes provide a simple way to interact with the Lilith framework, and require no detailed knowledge of the workings of Lilith.

The LillyLilim Class

In Lilith in general, the user code, called Lilim, handles the action at each node of the tree. In the Lilly classes, a special Lilim, called LillyLilim, provides an easy way for the user code to obtain a special type of functionality from Lilith. The LillyLilim class is automatically instantiated on each node of the tree. It is responsible for:

  1. handling initializing events on the node
  2. manipulating data to be sent down the tree
  3. perfoming its major action on the node it is instantiated on
  4. manipulating results passed back up the tree
The LillyInterace class provides a few well-defined interfaces for defining these events. The LillyLilim class implements LillyInterface, providing default methods for the interfaces. The user extends the LillyLilim class, overriding the methods to suit his own purposes.

Unseen by the user, the LillyLilim class also ensures that the methods are called in the proper order and that data is passed around the tree properly. The user gains these actions automatically through the inheretance of LillyLilim; this requires no knowledge or intervention on the part of the user.

The LillyInterface and default behavior is described below.

Lilly Interface
Lilly Interface PURPOSE DEFAULT BEHAVIOR
void initOnTree(String[] args, MO fromClient) performs initial actions on the tree. Receives as arguments an array containing the command line parameters and an MO sent from the client (the return value from LillyClient.initOnClient())for any initialization to be done on that node. All nodes in the tree will receive the same paramters. nothing
MO[] distributeOnTree(MO m, int[] numDesc) processes data to be sent down the tree. Receives as arguments an MO from its parent in the tree and an integer array containing the value of the total number of descendents of each child of this node. Returns an array of MO's, the first for itself and the rest for its children. The first MO in this array will become the argument to onTree(), and the rest will be arguments to the childrens' distributeOnTree() methods. returns an MO array where each MO is a copy of the MO received in the arguments
MO onTree( MO m) performs action on this node. Receives the first MO in the return array from this node's distributeOnTree(). Returns an MO that will be the first argument in the array to this node's collateOnTree() returns the MO it received as an argument
MO collateOnTree(MO[] m) processes data being returned up the tree. Receives as arguments an MO array where the first element is that returned from this node's onTree() and the rest are the returns from the childrens' collateOnTree(). Returns an MO that will be in the argument MO array of the parent's collateOnTree(). returns an MO containing all the MOs in the argument array
Hashtable getSystemHandle() a utility method that returns a hashtable useful for storing infomation. This method is not meant to be overridden. By default, this hashtable contains infomation about this node's position in the tree. N/A

The user's class derived from LillyLilim is automatically distributed and instantiated on each node. Its methods are then called automatically. initOnTree() is called only once, prior to the real action to be performed to handle any initial actions to be performed on that node. The others are called in a looping sequence until the client stops the loop. Each time through the loop, distributeOnTree() manipulates and distributes data down the tree, onTree() performs action on the node, and collateOnTree() manipulates and returns data back up the tree. Data is sent to and returned from methods in the form of Message Objects, aka MOs. The MO is a general purpose data rack that can hold a list of data objects.

Use of the LillyLilim is illustrated in the case of the distributed sort, distributeOnTree() receives the list of numbers to be sorted, separates the list into pieces for itself and its children to sort, and returns these lists. onTree() is responsible for sorting the list of numbers for that node and returning the sorted list, and collateOnTree() takes the sorted lists from oneself and one's children and merge sorts them together. The relevant pseudo-code follows:

public class SorterLilly extends LillyLilim{
public MO[] distributeOnTree(MO tmpMO){
/* tmpMO contains the list of numbers to be sorted */

/* code here which:
1) Unpacks MO to get array of numbers to sort via calls to tmpMO.pullInt()
2) Divides array into subarrays for self and children
3) Packs arrays for self and children into MO[] all_piecesMO
via calls to all_piecesMO[i].pushInt(int)
*/

return all_piecesMO;
}

public MO OnTree(MO myMO){
/* myMO contains the list of numbers to be sorted on this node. It is the 1st element in the return from distributeToTree()*/

/* code here which:
1) Unpacks MO to get array of numbers to sort via calls to myMO.pullInt() and places them into myArray
2) sort(myArray); /* sort own piece using own sort method */
3) Packs sorted array from self into MO my_pieceMO
via calls to my_pieceMO.pushInt(int)
*/

return my_pieceMO;
}

public MO collateOnTree(MO[] allMO){
/* allMO contains my return from onTree() and my kids returns from collateOnTree*/

/* code here which:
1) Unpacks sorted arrays from allMO
allArrays
2)final array = mergeSort(allArrays); /* merge sort all sorted arrays */
3) Packs combined sorted array into MO combined_pieceMO
via calls to combined_pieceMO.pushInt(int)
*/

return combined_pieceMO;
}

}

In the distributed sort, initOnTree() is not used and performs its default action of nothing, but a possible example of its usage in another context would be to unpack a file send down from the client to be instantiated on each node.

Note that MOs for the node itself to handle are returned from distributeOnTree() and onTree() and passed as parameters to onTree() and collateOnTree(). This is in order to minimize the number of methods needing to be overriden in the case the default behavior is desired. For example, if the data to be distributed need not be manipulated down the tree, but needs to be saved for the action of onTree(), then distributeOnTree() need not be overridden to save the data because it is provided as an argument to onTree(). Similiarly, if the data is not going to be changed in onTree(), then onTree() need not be overridden to save the data since it is provided as an argument to collateOnTree().

The Lilly Client Class

The user starts up the LillyClient to interact with the root node of the tree. It is responsible for:

  1. handling initializing events on the client
  2. determining data to be distributed to the root
  3. handling the results from the root
  4. determining whether or not to continue looping through events 2 & 3
The LillyClientInterface class provides a few well-defined interfaces for defining these events. The LillyClient class implements LillyClientInterface, providing default methods for the interfaces. The user extends the LillyClient class, overriding the methods to suit his own purposes.

Unseen by the user, the LillyClient class also ensures that the methods are called in the proper order and that data is sent to and retrieved from the root properly. The user gains these actions automatically through the inheretance of LillyClient; this requires no knowledge or intervention on the part of the user.

The LillyClientInterface and default behavior is described below.

Lilly Client Interface
LillyClient Interface PURPOSE DEFAULT BEHAVIOR
MO initOnClient() performs initial actions on the client. Returns an MO containing the results of those actions. This MO will be used in an initial distribution of information down the tree and is provided to the tree nodes as an argument to LillyLilim.initOnTree(). returns an empty MO
MO distributeToTree(MO m) calculates an MO to be sent to the root node to be distributed to the tree. The return value is sent to the root node as the argument to LillyLilim.distributeOnTree(). (This MO may be further processed by LillyLilim.distributeOnTree() as it is sent down the tree.) The argument is an MO - in the first iteration it is an empty MO; in subsequent iterations it is the MO returned from the previous iteration via reapOnClient(). returns an empty MO
void reapOnClient(MO m) takes the resulting MO from the results collection on the tree (the return value from the root's distributeToTree() in the next iteration of the loop. displays the Objects in the MO as strings
boolean stopLoop(MO m) determines whether to stop iterating. Takes MO returned from reapOnClient() returns true, so default action is just one iteraction
boolean checkCommandLine(String[] argv) takes command line arguments to check validity. If it returns false, the client exits. returns true
String[] getParameterArgs() utility method to return the paramters from the command line arguments. not intended to be overridden. returns the parameters from the command line

The user starts LillyClient on the client. The methods are called automatically. checkCommandLine() is called to check the validity of the command line. If the method returns false, the client exits. Otherwise the rest of the the LillyClient's actions progrees. initOnClient() is called to handle any initial actions on the client. The return value of initOnClient(), the command line parameters, and the user's LillyLilim are automatically packaged up and sent to the tree. The other methods are called in a loop until the client decides to stop the loop. distributeToTree() calculates an MO to be sent to the root of the tree, reapOnClient() handles the return values from the collation on the tree, and stopLoop() is used to determine when to end the loop.

For example in the distributed sort, distributeToTree() calculates a set of random numbers to be sorted and returns this list. reapOnClient() displays the sorted list. The relevant pseudo-code follows:

public class SorterLillyClient extends LillyClient{
public MO distributeToTree(MO reapedMO){
/* reapedMO is the returnedMO from the previous iteration on the tree. it is unused */

/* code here which:
1) get total numbers of random numbers to sort from user specified parameter via call to getParameterArgs()
2) Generate random numbers
3) Packs numbers into MO all_numsMO
via calls to all_numsMO.pushInt(int)
*/

return all_numsMO;
}

public void reapOnClient(MO reapedMO){
/* reapedMO is the returnedMO from the root of the tree */

/* get sorted numbers and display them */
int nitems = ret.pullInt();
for (int i = 0; i < nitems; i++)
System.out.println(ret.pullInt());
}

}

In this case, initOnClient() is not used and performs its default action of nothing, but a possible example of its usage in another context would be to pack up a local file to be sent to each node of the tree only once, rather than each time through the loop, as would be the case if it were sent through distributeToTree().

Note how the LillyClient and LillyLilim methods work together. This is illustrated in color in the interfaces diagram. LillyClient.initOnClient() provides an argument for LillyLilim.initOnTree(). LillyClient.distributeToTree() provides the argument for the root LillyLilim.distributeToTree(). LillyClient.reapOnClient() handles the collected results from the tree provided by the return from the root LillyLilim.collateOnTree().

Data Passing in Lilith - the Message Object

MO API
MO Method ACTION
void pushXXX(yyy) adds an itemobject yyy of type XXX to the MO
XXX pullXXX() removes an itemobject of type XXX from the MO
XXX peekXXX(m) returns the value of the itemobject of type XXX at position m (integer) from the MO without removing it from the MO
Object hashedPut(String, yyy) adds object yyy to be specified in the hash table interface by the String value returning the previous object
Object hashedGet(String) returns a reference to the value of the object specified by the String value when placed in using the hash table interface, hashedPut(). This method does not remove the object from the MO.
Object hashedRemove(String) returns a reference to the value of the object specified by the String value when placed in using the hash table interface, hashedPut(). This method removes the object from the MO
boolean hashedIsEmpty() returns true if there are no hashed items in this MO
Enumeration hashedKeys() returns an object that allows the user to enumerate over all the keys in the hash table
void setMOUUID(String) sets the MOUUID to the String value
String getMOUUID() returns the MOUUID value of this MO
Valid types for XXX are all the primitive types, as well as String, ByteArray(for byte[]), and MO. Valid types for the hash table interface are all the wrapped types as well as String, ByteArray, and MO; these are returned as class Object references and must be cast to the desired class.


Communications are handled through the sending of Message Objects, MOs. The MO is a general purpose data rack that can hold a list of data objects. It is capable of marshaling this data into a byte stream, unmarshaling it from a byte stream, and recreating the data in a new MO. Each data item consists of a length, type, and the actual data. Data is placed into and removed from the MO through a well-defined set of calls pertaining to the primitive data types such as push/pull/peekInt(), push/pull/peekString(), as well as push/pull/peekMO(). Push places an object into the MO, pull removes it from the MO, and peek returns the value without removing it from the MO. There is also a hash table interface which allows the users to assign a label to each item. This interface can be used to provide random access to internal MO data structures. The methods are hashedPut(), hashedGet(), and hashedRemove(), and they support the Java wrapped types corresponding to the primitive types supported by MO. The total set of user-related calls for assembling/disassembling the MO is defined above.

MOs also carry with them an id field called an MOUUID used for identification of the MO. The MOUUID is in the form of a user-defined String. MOUUIDs are set/returned using set/getMOUUID.