JAsCo Artifact Loader Manual

Index

-What does the program do?
-What is the concern model?
-Concern model examples
    1) A java class element
    2) A relation element
    3) Intensional concern model element
-What is the concern model for Jasco?
    1) Elements
    2) Relations
-How does the loader work internally?
    1) Overview of packages
        jasco.tools.connectorparser
        jasco.tools.aspectparser
        jasco.artifacts.types and jasco.artifacts.types.artifacts
        jasco.artifacts
        jasco.artifacts.loaders   
    2) Overview of loading process
-Installation
-Relation User-interface problems
-Problems and solutions

What does the program do?

So the CME is plugin for eclipse and I've written two plugins for the CME. They are both pretty much the same.
There's a loader that loads a jasco aspect and there's a loader that loads a jasco connector.
Loading involves 2 phases:
1)making a general representation (concern model) off the jasco-specific parse tree.
2)building relations between various types of elements (like java elements, jasco element and
intensional queries for pointcut expressions). In a second phase relations other then parent-child hierarchy
are added between the various elements of model. The relations can crosscut the model, like connecting
all java-methods with jasco advices. Most work was in making the query-concern relations.
It's completed indepent of other jasco tools.

What is the concern model?

It's a very general model designed to represent any software artifact (UML, makefiles, source code in various
languages). The basic element uses the composite design pattern. The elements form a class hierarchy.
At the bottom there are elements such as unit and group. At a higer level there's a compound unit
element which is a both a group and a unit. Other elements are called method, field, aspect and concern.
A simplyfied version of a concern model is shown here:

interface Artifact
{
    //Name
    String getName();
    //Class
    String getClassifier();
    //Composite methods
    void addChild(Artifact artifact);
    Artifact[] getChildren();
    //Location
    Object Location();
    //Modifiers
    String[] getModifiers();
    //Attributes
    void putAttribute(String key,Object value);
    Object getAttribute(String key);
}

Concern model examples

1) A java class element:

-name is the name of the class.
-classifier is the pre-defined string "CLASS" (a "CLASS" is a subclass of the types unit, group and concern)
-children are Artifacts with classifiers "METHOD" or "FIELD"
-location is a String containing absolute filename
-modifiers can contains "private" or "abstract"
-attributes can be any object, like a pointer to a JDT parse tree
Note that each artifact is specialized and adds additional details. Also remark that types are encoded as Strings.
Details could be used by other plugins that use those details for weaving. But are not general.

2) A relation element:

Another important kind of element is a relation.
example: the "java-class extends java-class" relation
-name is the name of relation
-classifier is "RELATIONSHIP" (subclass of group)
-children are two java-class artifacts
-location is a filename and linenumber of the java-class source code where the extends relationship is specified
-modifiers and attributes not important.

3) Intensional concern model element:

The last kind of element is the intensional concern.
example: an element which represents 'all classes implementing the interface Observer'
-name is specified by user
-classifier is "CONCERN"
-children is a list of classes
-modifiers is "INTENSIONAL"
-location not important
-attribute: query= "parent(relationship implements(class *, interface Observer))"
Note that the list of children is updated by an associated query expressions described in
the puma language. More about this later.
As you may see this generic type-less approach allows for all kinds of artifacts to be represented. Not just
languages similar to AspectJ or java. One other language that is also supported is Ant (an xml
based scripting language used for compiling and packaging of java programs very similar to makefiles).

What is the concern model for Jasco?

Note that the concern model is a bit simple. Many details such as regular methods or
combination strategies have not been added. The most complex element however, namely
a query-concern is in a very evolved phase.

1)Elements:

-Jasco Aspect is an "ASPECT"
-Advice is "ADVICE"
-Package is "PACKAGE"
-Hook is "POINTCUT"
-Hook constructor is "METHOID"  ("METHOID" is used for artifacts for which the source code is important)
-Connector is "CONCERN"
-Hook instance in a connector is "UNIT"
-Advice called by connector is "METHOID"
-Hook isApplicable test is "METHOD"
-A concern specified by a query defining a cutpoint expression is "CONCERN"
-There is also a special unloaded element, which can represents hooks/aspects/classes that
are refered to by another element, but are not loaded because they are not in a CME project.
Note:Classifiers are not easy to add to the conman model. (The puma parser has to be adjusted)
So I re-used existing classfiers (instead of subclassing them) which is a bit ackward.

2)Relations:

-java-method "adviceexecution" advice
-advice-call "adviceexecution" advice
-hookinstance "connects" hook and query-concern
-aspect "extends" aspect
-hook "extends" hook
-aspect "refersTo" java-class    (if it's in an import statement)
-connector "refersTo" java-class

How does the loader work internally?

1) Overview of packages:

There are 5 packages in my project. See source code for more
details about individual classes. All work is done in the package jasco.artifacts.loaders.

jasco.tools.connectorparser:

   Modification of the jasco connectorparser to reflect my concern that this parser
   should only depend on connector source code, not on compiled files.
   Not interresting for anyone.

jasco.tools.aspectparser:

   -Defines 3 classes needed for tokenizing the hook constructor body, it
    also simplifies basic operations like execute and call.
   -an example of a hook constructor:
           ahook.ahook(method(..args),method2(String, int))
        {
               execute(method) && withincode(method2);
        
   -HookConstructorTokenizer.tokenize() then returns (using xml to make clearder):
     <tokeslist>
       <methodToken keyword = "execute", name = "method", type = "methodexec"/>
       <token name="&&" type = "booloperator">
       <methodToken keyword = "withincode", name = "method2", type = "methodexed)
     </tokenlist>

jasco.artifacts.types and jasco.artifacts.types.artifacts

   -most classes are trivial
   -See section above for concern model elements intro used by my loader.
   -See the paper on concern modeling (www.eclipse.org/cme) if you need
    more information.
   -class ClassifiersAndAttributes contains many constants, like
    the classifier for the different types of artifacts.
   -All artifacts have the same structure. See their base-class
    JascoArtifact for more info.
   -JascoQueryUnit uses observer design pattern (see GangOfFour book),
    and is still under development to fix problem (see section
    problems and solution problem 4 first point).

jasco.artifacts

   -DirUtils can find various directories needed, like dir with icons
    or root dir for project in workspace
   -LoaderUtil contains some copy-pasted stuff from ant-loader.
    (jasco loader design seriously inspired by ant loader solution)
    Some method create indexes in concernmodel (so that elements
    are found rapidly).
    Other methods do stub-loading(see next package) for java elements.
    Other methods retrieve the top-level element of the concern model, which
    are:
      -The world space = the workspace subset off all CME aware projects
      -The feature space = contains saved (concern-)queries like jasco pointcut's
      -The unloaded space (space for stub elements)
   -Logger is used for logging errors and debugging info

jasco.artifacts.loaders

   In this package all classes are defined that do the work.
   The two loaders are very similar. The aspect loader is more
   complicated because there are more aspect artifacts. The connector
   loader is more difficult because it handels the intensional (jasco pointcut)
   query element and relationship making. There's some copy-pasting
   code for the aspect and connector loader which further re-factoring
   should avoid.

2) Overview of loading process:

 Here are some different phases that a loader does
 -1)Initialize parser
 -2)Open aspect/connector source
 -3)Make parse tree containing aspect/connector details
 -4)Traverse the parse tree and build corresponding concern model
    elements (thus all types in types package and especially in types.artifacts
    package).
 -5)Make relations between artifacts of one source file. (easy)
 -6)Make relations between artifacts of different source files.
    Harder because stub-loading is necessary = making
    temporary 'stubs' for elements that aren't loaded yet.
    When the temporary elements are loaded later, the stub is augmented
    with all details. A stub element is en empty element with an
    unqualified name in the Unloaded space.
 -7)Make relations between artifacts of different source files,
    which require all elements to be fully loaded. Hard because this
    requires a more difficult version of Stub-loading. I used
    the command and observer design pattern to solve this problem.
        
 Different phases as done concretely(which class does what) by ConnectorLoader.    
 I only explain the basic of the ConnectorLoader, because AspectLoader
 has identical design. ConnectorLoader.load() is called with the filename of the connector and
 the concern element of the project containing the connector.
 The loader then starts each phase.
  -1->3)First it initialise the ConnectorParser with the correct options set
     and then calls parse() method to build the parse-tree for the
     connector.
  -4)Phase four is done by two classes ConnectorVisitor and ConnectorFactory.
     ConnectorVisitor traverses the parse-tree and creates concern model elements
     starting from root-node to leaf-node. The Visistor call the ConnectorFactory.makeUnit
     method which dispatches on the node type and returns a concern model
     element of corresponding type. The Visitor has two kinds of
     methods: makeConnectorRelations(XXX node) and traverse(XXX node).
     The traverse methods are self-explainatory. The makeConnectorRelations
     methods only builds (java) lists or map guide the ConnectorRelationMaker
     in making relations in step 5.
  -5)ConnectorRelationMaker makes all relations involving a connector.
     The relation advice-call "adviceexecution" advice is easy. It adds a relation
     between the hook 'instantiated' by the connector and the advice called by
     the connector.
  -6)ConnectorRelationMaker has to make two relations more to construct, namely:
       hookinstance "connects" hook and query-concern relation and the
       java-method "adviceexecution" advice relation.
       For this to work it searches for the corresponding hook. If it is not
       loaded already, this relation-making process will be delayed until the hook
       is loaded (done by using the innner class AddQRelationToHookStub).
       Once the hook and the hookinstance are both found and loaded, ConnectorRelationMaker
       has all data it needs to make the full cutpoint expression. It then makes
       a new concern-query. The concern-query is evaluated and to each method matching
       the query a new relation java-method "adviceexecution" advice is added.
       Note that as in future other java-classes are loaded the query-concern will have
       potentially new java methods mathing the expression. Thus new "adviceexecution" relations
       have to be added, this doesn't work at the moment but a listener for the query-Concern
       like the one implemented JascoQueryListener should be used for this problem.

Installation

-Download eclipse 2.1.3 www.eclipse.org
 The CME works most stable in version 2.1.3. But in the very near future only eclipse 3 will be supported.
-Download the CME plugin using the update perspective in eclipse www.eclipse.org/cme
-Download jasco loaders zip-file: "jasco.artifacts.jar" at home page for this project
 Unzip this file (open with winzip) into the plugins directory of eclipse in the folder jasco.artifacts.
 This file include includes the source and the jasco libs.
 
Important remarks:
-AspectJ and CME have problems when lot's of memory is required (e.g. larger projects loaded).
 Avoid this problem by buying lot's of ram and increase the java VM min heap size to 150 megs.
 Do this by starting eclipse in windows as follows: C:\len\eclipse\eclipse.exe -consoleLog -vmargs -Xmx150m.
-Program is only tested by me using windows XP pro, java 1.4.2 and eclipse 2.1.3!
 (Should work under Linux however, otherwise check DirUtils class)
-Start with a clean eclipse if you have problems with CME.
-For developpers: you can download the source of the CME from the CVS.

Relation User-interface problems

Relationships are not very well supported from an end-user GUI interface perspective.
I tried to work around these problems, but the result is still user unfriendly.
These are the problems with the concern explorer view concerning the display of relations:
-The concern explorer view in eclipse shows the concern model using a tree-view. The tree-view is
 fine for the elements, but is ackward for relations.
-Every element has an associated icon with relations even if no relations are defined for it.
 Furthermore the icon indicates that relations are indeed defined for the element even if they are not.
-For memory efficiency reasons pointers to the endpoints of a relation are not stored. Instead
 all relations are stored in one list and endpoints are calculated. As a side-effect of this
 approach relations are inherited by parents in the tree. Thus if a certain hook refersTo a java-class,
 then the hook will also referTo the package and project in which the class is defined. Similarly
 all hooks connect to the parent of all query-concerns, namely Features. Java-classes
 also inherit any "adviceexecution" relation from their children.
-Relations are not visible per default, change this by disabling the filtering on relations
 (press filter-icon in concern explorer view).

Problems and solutions

My loader as wel ass the CME are still in an evolving phase. Some irritating bugs are these:

1)Relations (see previous)

2)BUG: Concern explorer view refresh button disables. Due to some bizar reason
the concer explorer view doesn't allow you to press the refresh button even if you're
source code has changed. One stupid fix is to rebuild the project after adding some whitespace to a file
(After  which in the CME Event Trave windows a message appears indicating the discovery).

3)BUG: The CME saves the feature space (this is the space containing intensional query-concerns) in the feature.xml file.
This can cause serious problems if these cached queries become inconsistent with the workspace.
If the program crashes for this reason, the solution is to delete the file in
".../eclipse/workspace/.metadata/cme.uit/feature.xml". This is especially true if you get the folowing error:
[Fatal Error] :1:1319: The entity name must immediately follow the '&' in the entity reference.

3)BUG: I suspect there's a bug in ant, which manifests itself if you open the Unloaded space (default
  this space is hidden) and the concern explorer view goes blank.

4)Not properly implemented:
-Java-method "adviceexecution" advice: does not work if classes are loaded after jasco artifacts are.
-Concern-Queries:
   -call relation is shown several times. (no fix, just bug in CME)
   -cflow doesn't work cause it's a runtime query
   -error-handling in puma expression is non-existing! A puma expression containing errors,
    can return all elements or none.