Introduction
You can take two extremes to software development.
You can do a Big Design Up Front (BDUF) as captured in the waterfall model where you design you software in detail and then, in principle, implementation of the software is a mechanical process of implementing the specifications. The problem with the waterfall model is that it is impossible to capture all the details in the specification and as a result the implementation deviates from the specification. Further, as new requirements come in or get changed, the implementation tends to get updated, but the specification remains unchanged with the result that it gets out of date and worse than useless.
At the other extreme you can do no design. It’s unfair to say that agile methodologies have no design, but in principle in an agile methodology you start with a single test and then write code to make that test pass. You don’t worry about anything beyond implementing code to pass that single test. In practice you may have a software architecture in your head, but that it is not captured in any form that somebody else might be able to use. As a result agile methodologies sometimes find it hard to scale to bigger teams. Also, central to agile is the idea of refactoring as more functionality is added. This reworking is wasted effort that is ideally avoided.
The Specification Driven Development (SDD) methodology is an attempt at a middle road that draws on benefits from these two extremes. It allows for Design Up Front, but aims to implement that in an agile way.
There is a meme in some developer circles that “the software is the documentation.” SDD turns this on its head and says that “the documentation is the software,” or more specifically, “the specification is the software.” The principle is that, when the specification is finished, almost as a side effect, you will have running code. Thus, although when writing code your objective is to have code that implements the functionality you want, from a SDD methodology point of view running code simply validates that the specification is complete.
Principles
So how do we put a SDD methodology into practice? SDD takes the view that the planning up front part of the waterfall model isn’t the problem, it’s the fact that when we do planning up front the ‘design’ tool we reach for is a word processor. As such there is no connection between the specification and the implementation.
Like waterfall, SDD starts by developing a specification. But rather than creating the specification in something like Word, the SDD specification is written in the programming language that your final code will be written in. The reason for this is that as your specification becomes more refined, your specification will morph into your application – “the specification is the software.”
Just to emphasis that, when you start coding on a new project, you DO NOT start by creating a project named something like “My Application”. You start by creating a project called something like “My Application Specification”. Only towards the end of the implementation will you create a project called “My Application”, which should be a minimal component that ties together the various parts of the specification in a from that results in your application. (In practice you may end up creating 3 projects: “My Application”, “My Application Specification” and “My Application Code”. “My Application Core” contains code that is common to both “My Application” and “My Application Specification”. You would create “My Application Core” at the same time you created “My Application Specification”. “My Application” is added later.)
The reason for implementing the specification using the target programming language is to use the compiler to validate the specification at each stage of its development. Hence, right from day 1 the specification should be compilable and runnable.
Class Specifications
As new implementation classes are added to the specification (most likely into the “My Application Core” project) a sister set of classes are added that mirror the implementation classes. The sister set are the specification classes. There should be a one-to-one mapping between implementation classes and specification classes, and each public method within an implementation class should have a matching specification method in the specification class. The mapping between the implementation method and its specification method should be completely deterministic, firstly so that a developer looking at an implementation method knows where to find the specification, but also so that automated tools can check (perhaps as part of the nightly build regime) that each implementation method does in fact have a specification. (As a result, in C++, a class named “foo” will typically have a header file named “foo.h”, an implementation file named “foo.cpp” and a specification class named “foo-spec.cpp”, the latter of which contains the “foo_spec” class. “foo.cpp” will be part of the “My Application Core” project and “foo-spec.cpp” will be part of the “My Application Specification” project.)
The specifications within a specification method look the same as unit tests. However, to keep with the specification theme they are called unit specifications (which leads to the idea of Specification Driven Development or SDD). One difference between unit tests and unit specifications is that unit specifications allow for specifications to be defined as unimplemented. So in addition to unit test methods such as assertTrue(), unit specifications will have methods such as assertTrueTodo(). When such methods are run, rather than run the test and report pass or fail, they will increment a counter the records the number of unimplemented tests. It is analogous to a warning when performing normal code compilation – the target is to get 0 fails, 0 unimplemented specifications.
In principle, when implementing a methods functionality a developer removes the “Todo” part of the name from one of the unit specifications and attempts to make that test pass. Having made that test pass they convert another unimplemented test to a real test and repeat the process.
If a co-developer (or the developer themselves) needs functionality from a method that they believe is not captured by the current set of specifications they can add suitable “Todo” style specifications to the specification method (ideally having discussed it with the person that’s responsible for implementing it. The version control system can capture who added a particular specification and the implementer can seek out the requester if additional clarification is required. As such, the burdens on an independent change control system / bug reporting system are reduced.
Someone requesting functionality from a class that requires a new method could even add a new method to the specification class that is not mirrored in the implementation class. Nightly build tools would detect this discrepancy and highlight it as part of the daily report (ideally flagging it as a new item rather than just reporting that there is now, say, 15 specification methods without implementations).
Class Relationships
A key high level design activity is specify the relationship of classes. SDD does this in the implementation classes an uses the compiler to check that the relationships are valid. One problem with this is that it is sometimes difficult to get compilable code that has a relationship with other classes when neither class is fully implemented.
SDD introduces some helper constructs to get around this problem. In C++ they are defined similar to the following:
#ifndef SDD_H
#define SDD_H
#if defined( SDD_ALLOW ) || defined( SDD_SILENT )
template< class T >
class sdd_class
{
public:
typedef T Tuses;
sdd_class() : p( 0 ) {}
T * operator ->() { return p; }
private:
T * p;
};
// sdd_subsystem<T> is a documentation template to indicate
// that a class containing it is associated with a particular
// sub-system. A class is used to identify a sub-system.
// Typically the sub-system classes would be listed in a
// header file. Tools may also be able to use C++ namespaces
// to partition classes into sub-systems.
template< class T >
class sdd_subsystem
{};
#ifdef SDD_SILENT
#define SDD_SUBSYSTEM( the_subsystem )
#define SDD_USES_CLASS( the_class )
#define SDD_IMPLEMENTS( the_method_group )
#define SDD_USES( the_class, the_method_group )
#define SDD_CALLS_SPECIFIC( the_class, the_method )
#else
#define SDD_SUBSYSTEM( the_subsystem ) \
typedef sdd_subsystem< the_subsystem > \
sdd_subsystem_ ## the_subsystem;
#define SDD_USES_CLASS( the_class ) \
typedef sdd_class< the_class > sdd_ ## the_class;
#define SDD_IMPLEMENTS( the_method_group ) \
struct sdd_ ## the_method_group {};
#define SDD_USES( the_class, the_method_group ) \
{ typedef sdd_ ## the_class :: Tuses :: \
sdd_ ## the_method_group x; };
#define SDD_CALLS_SPECIFIC( the_class, the_method ) \
{ if( 0 ) { sdd_ ## the_class() -> the_method; } }
#endif
#else
// Uses undefined
#endif
#endif // SDD_H
An example of its usage, the class “foo” uses functionality from class “bar”. “foo.h” may look like this:
class wibble {}; // Actually defined elsewhere
class foo
{
private:
SDD_SUBSYSTEM( wibble )
SDD_USES_CLASS( bar )
public:
void action()
{
SDD_USES( bar, length )
//SDD_USES( bar, scale ) // Causes error
// A more specific statement of usage
SDD_CALLS_SPECIFIC( bar, length() )
SDD_CALLS_SPECIFIC( bar, length(2, 3) )
}
};
“bar.h” might look like:
class bar
{
public:
bar( int t ) {}
SDD_IMPLEMENTS( length )
size_t length() const;
size_t length(int, int) const;
};
“bar-spec.cpp” might look like:
class bar_spec
{
public:
void run()
{
length();
}
private:
void length()
{
// The length method return the length of the object
}
};
In the above, the “SDD_USES_CLASS” macro is used in the “foo” class to indicate that the “foo” class uses the “bar” class. In the “bar”class the “SDD_IMPLEMENTS” macro indicates that the class supports some functionality called “length”. It is intentionally a high level statement and purposefully omits whether the method takes any arguments and what it returns. In the “action” method in the “foo” class the “SDD_USES” macro indicates that it uses the “length” methods defined in “bar”. Again, this is a high level statement and no attempt is made to capture what sorts of arguments and return values are involved.
The “uses_class” template, and the “SDD_IMPLEMENTS” and “SDD_IMPLEMENTS” macros are intended to be sufficiently descriptive of the high level structure that UML diagrams can be automatically generated from the code in order to allow initial high level code reviews and walk throughs.
They also have sufficient mechanics to them to allow the compiler to check that “bar” does indeed exist for “foo” to be able to use it and the “bar” has declared that it does support some functionality named “length”.
Thus these provisions allow developers to sketch in the sort of functionality that “foo” and it’s methods require from “bar” without getting bogged down into having to prematurely implement functionality just to get the code to compile. This makes it much easier to iterate though a high level design that is validated by the compiler. Hopefully tools will also be able to detect whether a class offers functionality that is not actually used. Alternatively if a certain piece of functionality is suspected to not be required the relevant “SDD_IMPLEMENTS” statement could be commented out and the compiler will report errors if the hypothesis is incorrect.
Over time, as the software is developed, definitions like the “SDD_USES_CLASS” macro and the “SDD_IMPLEMENTS” and “SDD_USES” macros will be replaced by real instances or pointers to objects etc. As an intermediate step a developer could use something like the “uses_bar->length();” statement above. This states that the code is intended to use a specific instance of a method defined by “bar”. However, because it is access by the “uses_bar” object which is an instance of the “uses_class” template it indicates that this is not the final form.
The example also includes the “bar_spec” class which is the specification for the “bar” class. It shows that the specification for the “length” functionality, which in this case is undefined. The specification validation framework will report on specifications that are not implemented.
If a class has multiple overloaded methods with the same name, then each variant should have its own method and suitable tests. Some form of name mangling should be used to tie the implementation method to it’s specification function so that automated code can check if there are any missing specifications.
The “SDD_USES_CLASS” macro and the “SDD_IMPLEMENTS” and “SDD_USES” macros are included in a project by setting the “SDD_ALLOW” definition in the project settings. As various files are more and more defined the developers stop including the “SDD_ALLOW” definition in the project.
In this sense the “SDD_USES_CLASS” macro and the “SDD_IMPLEMENTS” and “SDD_USES” macros are like scaffolding that supports the construction of a building. Once the building is built the scaffolding is removed. The compiler can validate that all the scaffolding has been removed by not setting the “SDD_ALLOW” definition.
Part of the functionality of the daily reporting software could be how many instances of the various macros are in place and how they are changing.
Note that you should be able to supply a saleable product to a customer while still having high-level, unimplemented specifications in the code that are ready to be worked on in the next release cycle.
Conclusion
SDD is intended to be a light weight methodology that aims to maximise the amount of design information captured in the code base itself, such that as the code based specification is refined the specification yields the final code. It captures a number of the benefits of agile methodology such as writing tests to define functionality, but also allows up front design that can be validated by the compiler with the goal of reducing the amount of development rework required due refactoring.
Refactoring will still be part of a SDD project, but the refactoring can be done at a higher level of abstraction. As refactoring at a higher level means that less lines of code is thrown away during a refactoring exercise it is hoped that developers will be more inclined to refactor as they will be less wedded to the volume of code that they have already produced.
As the code is the specification and the specification is the code, there is no divergence between code and specification as requirements change over the life of a project.
While SDD does allow developers to do design up front, how much design up front is done is a question for each development team. It may just be enough to see them through the next sprint, or it may be sufficient that the developers feel that the structure will not change dramatically over the lifetime of the project.