Last year a customer asked me to modularize a rather critical legacy Java system. It has grown over the years and was based on a rather monolithic Ant based build process. The architecture of the system clearly defined layers and components, but this architecture was not supported by the project layout. Now, due to the monolithic layout the well-defined architecture became more and more corrupted, because nothing in the build process really enforced the rules and restrictions of the architecture. My customer did not feel comfortable with this situation and decided to invest some time and money in a major refactoring of the system. The goal of the refactoring was to break up the monolithic project structure. Instead, multiple smaller modules with clearly defined dependencies on each other should be constructed.
After some investigation we decided very early in the refactoring process to move the whole thing to Maven. This was not an easy decision. If you decide to use Maven, you have to adhere to the conventions and rules defined by the Maven build process. Projects that have their own, Ant-based build process typically do not follow any other rules than their own. So converting such a project to Maven can be a daunting task. After some prototyping I was sure that I would be able to accomplish it anyhow, and here is how I did it:
A very good starting point for the refactoring was the existing software architecture, which defined layers and components. The goal was to more or less define one separate module per component. Additionally, some infrastructure modules were necessary that capsuled for example the configuration files. However, even with this basic strategy in mind, the practical transformation of the one big project to about 50 modules is a complicated thing. The main problem are the dependencies between the various modules. In a Maven POM, you explicitly define all dependent modules. If this dependency set is incomplete, the resulting project won't build. If you only have 2 or 3 modules, you can figure out the dependencies by try and error. This won't work in a reasonable time frame for 50+ modules.
Another pitfall are cyclic dependencies. Maven won't build projects with cyclic dependencies between modules. A cyclic dependency occurs for example when Module A is dependent on B which again is dependent on A. The problem is that in most cases the cycle is not so easy as in this example. Typically, you have something like A->B->C->D->A. Since Maven resolves dependencies transitively, it will detect this cycle. And immediately cancel the build (which is perfectly all right IMO, because cyclic dependencies are a bad thing and should never occur in a well-structured project).
At short, I was faced with two problems. First, I needed a possibility to cut the big project in the "right" modules and to find out the dependencies between the modules. And second, I had to identify and break up any cyclic dependencies between the new modules. To work efficiently, all this must be done in a kind of simulation, before actually performing the refactoring and breaking up the project.
Luckily there are some tools out there that are able to support these kind of tasks. I chose SonarJ for this specific project. SonarJ allows you to define a target architecture for a given software system. In my case, I defined the planned 50+ modules as target architecture. Next I defined the target dependencies between the modules using a graphical modelling tool within SonarJ. The cool thing is that SonarJ is now able to identify all modules, packages and even classes that break these rules. Often the violation are caused by wrongly placed packages (someone placed the code in component A, but this is simply nonsense, it belongs to component B). Now, I was able to fix any errors by virtually moving the package to the right component. This movement occurs only in the architecture model of SonarJ, no source code is changed (yet).
Additionally, SonarJ has a very powerful tool set to identify and break up cyclic dependencies. In most cases, the break-up is also done by virtually moving classes or packages around in the model. Of course, some cycles are caused by real "bad" code that needs some thorough refactoring. In these cases, one can mark the culprit classes on a todo list with some additional notes for later, manual rework.
As soon as my SonarJ model was fully defined, all cycles resolved and all dependencies defined, I "only" needed to actually perform the refactoring:
For a project of this size one cannot perform these tasks manually. Since I am a big fan of Groovy, I decided to write some scripts that interpret the SonarJ model file (which is luckily a well-defined XML-file). The first set of scripts created the Maven module structure. To keep thing as simple as possible, I first created a set of simple Maven Archetypes that contained empty templates for all needed module types. The script itself simply iterated through the SonarJ file and invoked the "right" archetype with some parameters. The actual creation of the directories, POM files, etc. was performed by the Maven Archetype Plugin. A second set of scripts performed the copy tasks using AntBuilder. AntBuilder is incredibly powerful in copying things and hence the script fits on a single page.
Basically I wrote a generator that interpreted the SonarJ model to perform the refactoring. This approach allowed me to easily try out variants or fix problems in the model that showed up after the generation. After I finished everything I realized that this was the first time I used a model-driven approach successfully! I'm still not a fan of MDSD, but hey, in this case it really worked OK. Of course, it was not real software development but "just" a refactoring with some Groovy scripts, but I have the feeling that any other approach would have been much more painful.
A very good starting point for the refactoring was the existing software architecture, which defined layers and components. The goal was to more or less define one separate module per component. Additionally, some infrastructure modules were necessary that capsuled for example the configuration files. However, even with this basic strategy in mind, the practical transformation of the one big project to about 50 modules is a complicated thing. The main problem are the dependencies between the various modules. In a Maven POM, you explicitly define all dependent modules. If this dependency set is incomplete, the resulting project won't build. If you only have 2 or 3 modules, you can figure out the dependencies by try and error. This won't work in a reasonable time frame for 50+ modules.
Another pitfall are cyclic dependencies. Maven won't build projects with cyclic dependencies between modules. A cyclic dependency occurs for example when Module A is dependent on B which again is dependent on A. The problem is that in most cases the cycle is not so easy as in this example. Typically, you have something like A->B->C->D->A. Since Maven resolves dependencies transitively, it will detect this cycle. And immediately cancel the build (which is perfectly all right IMO, because cyclic dependencies are a bad thing and should never occur in a well-structured project).
At short, I was faced with two problems. First, I needed a possibility to cut the big project in the "right" modules and to find out the dependencies between the modules. And second, I had to identify and break up any cyclic dependencies between the new modules. To work efficiently, all this must be done in a kind of simulation, before actually performing the refactoring and breaking up the project.
Luckily there are some tools out there that are able to support these kind of tasks. I chose SonarJ for this specific project. SonarJ allows you to define a target architecture for a given software system. In my case, I defined the planned 50+ modules as target architecture. Next I defined the target dependencies between the modules using a graphical modelling tool within SonarJ. The cool thing is that SonarJ is now able to identify all modules, packages and even classes that break these rules. Often the violation are caused by wrongly placed packages (someone placed the code in component A, but this is simply nonsense, it belongs to component B). Now, I was able to fix any errors by virtually moving the package to the right component. This movement occurs only in the architecture model of SonarJ, no source code is changed (yet).
Additionally, SonarJ has a very powerful tool set to identify and break up cyclic dependencies. In most cases, the break-up is also done by virtually moving classes or packages around in the model. Of course, some cycles are caused by real "bad" code that needs some thorough refactoring. In these cases, one can mark the culprit classes on a todo list with some additional notes for later, manual rework.
As soon as my SonarJ model was fully defined, all cycles resolved and all dependencies defined, I "only" needed to actually perform the refactoring:
- Create new Maven modules for all modules defined in SonarJ. This task included the creation of the necessary parent modules that group modules of the same type. As a result of this task, a project structure with all Maven POMs was created, but still without any content (source code, resources, etc.). Of course, the newly created POMs included all dependencies defined in SonarJ.
- Copy all source-code files, resources, config-files, test cases etc. from the old monolithic Ant project to the correct new Maven module.
For a project of this size one cannot perform these tasks manually. Since I am a big fan of Groovy, I decided to write some scripts that interpret the SonarJ model file (which is luckily a well-defined XML-file). The first set of scripts created the Maven module structure. To keep thing as simple as possible, I first created a set of simple Maven Archetypes that contained empty templates for all needed module types. The script itself simply iterated through the SonarJ file and invoked the "right" archetype with some parameters. The actual creation of the directories, POM files, etc. was performed by the Maven Archetype Plugin. A second set of scripts performed the copy tasks using AntBuilder. AntBuilder is incredibly powerful in copying things and hence the script fits on a single page.
Basically I wrote a generator that interpreted the SonarJ model to perform the refactoring. This approach allowed me to easily try out variants or fix problems in the model that showed up after the generation. After I finished everything I realized that this was the first time I used a model-driven approach successfully! I'm still not a fan of MDSD, but hey, in this case it really worked OK. Of course, it was not real software development but "just" a refactoring with some Groovy scripts, but I have the feeling that any other approach would have been much more painful.
