The SDD software development methodology

February 3rd, 2012

Introduction

You can take two extremes to software development.

You can do a Big Design Up Front (BDUF) as captured in the waterfall model where you design you software in detail and then, in principle, implementation of the software is a mechanical process of implementing the specifications. The problem with the waterfall model is that it is impossible to capture all the details in the specification and as a result the implementation deviates from the specification. Further, as new requirements come in or get changed, the implementation tends to get updated, but the specification remains unchanged with the result that it gets out of date and worse than useless.

At the other extreme you can do no design. It’s unfair to say that agile methodologies have no design, but in principle in an agile methodology you start with a single test and then write code to make that test pass. You don’t worry about anything beyond implementing code to pass that single test. In practice you may have a software architecture in your head, but that it is not captured in any form that somebody else might be able to use. As a result agile methodologies sometimes find it hard to scale to bigger teams. Also, central to agile is the idea of refactoring as more functionality is added. This reworking is wasted effort that is ideally avoided.

The Specification Driven Development (SDD) methodology is an attempt at a middle road that draws on benefits from these two extremes. It allows for Design Up Front, but aims to implement that in an agile way.

There is a meme in some developer circles that “the software is the documentation.” SDD turns this on its head and says that “the documentation is the software,” or more specifically, “the specification is the software.” The principle is that, when the specification is finished, almost as a side effect, you will have running code. Thus, although when writing code your objective is to have code that implements the functionality you want, from a SDD methodology point of view running code simply validates that the specification is complete.

Principles

So how do we put a SDD methodology into practice? SDD takes the view that the planning up front part of the waterfall model isn’t the problem, it’s the fact that when we do planning up front the ‘design’ tool we reach for is a word processor. As such there is no connection between the specification and the implementation.

Like waterfall, SDD starts by developing a specification. But rather than creating the specification in something like Word, the SDD specification is written in the programming language that your final code will be written in. The reason for this is that as your specification becomes more refined, your specification will morph into your application – “the specification is the software.”

Just to emphasis that, when you start coding on a new project, you DO NOT start by creating a project named something like “My Application”. You start by creating a project called something like “My Application Specification”. Only towards the end of the implementation will you create a project called “My Application”, which should be a minimal component that ties together the various parts of the specification in a from that results in your application. (In practice you may end up creating 3 projects: “My Application”, “My Application Specification” and “My Application Code”. “My Application Core” contains code that is common to both “My Application” and “My Application Specification”. You would create “My Application Core” at the same time you created “My Application Specification”. “My Application” is added later.)

The reason for implementing the specification using the target programming language is to use the compiler to validate the specification at each stage of its development. Hence, right from day 1 the specification should be compilable and runnable.

Class Specifications

As new implementation classes are added to the specification (most likely into the “My Application Core” project) a sister set of classes are added that mirror the implementation classes. The sister set are the specification classes. There should be a one-to-one mapping between implementation classes and specification classes, and each public method within an implementation class should have a matching specification method in the specification class. The mapping between the implementation method and its specification method should be completely deterministic, firstly so that a developer looking at an implementation method knows where to find the specification, but also so that automated tools can check (perhaps as part of the nightly build regime) that each implementation method does in fact have a specification. (As a result, in C++, a class named “foo” will typically have a header file named “foo.h”, an implementation file named “foo.cpp” and a specification class named “foo-spec.cpp”, the latter of which contains the “foo_spec” class. “foo.cpp” will be part of the “My Application Core” project and “foo-spec.cpp” will be part of the “My Application Specification” project.)

The specifications within a specification method look the same as unit tests. However, to keep with the specification theme they are called unit specifications (which leads to the idea of Specification Driven Development or SDD). One difference between unit tests and unit specifications is that unit specifications allow for specifications to be defined as unimplemented. So in addition to unit test methods such as assertTrue(), unit specifications will have methods such as assertTrueTodo(). When such methods are run, rather than run the test and report pass or fail, they will increment a counter the records the number of unimplemented tests. It is analogous to a warning when performing normal code compilation – the target is to get 0 fails, 0 unimplemented specifications.

In principle, when implementing a methods functionality a developer removes the “Todo” part of the name from one of the unit specifications and attempts to make that test pass. Having made that test pass they convert another unimplemented test to a real test and repeat the process.

If a co-developer (or the developer themselves) needs functionality from a method that they believe is not captured by the current set of specifications they can add suitable “Todo” style specifications to the specification method (ideally having discussed it with the person that’s responsible for implementing it. The version control system can capture who added a particular specification and the implementer can seek out the requester if additional clarification is required. As such, the burdens on an independent change control system / bug reporting system are reduced.

Someone requesting functionality from a class that requires a new method could even add a new method to the specification class that is not mirrored in the implementation class. Nightly build tools would detect this discrepancy and highlight it as part of the daily report (ideally flagging it as a new item rather than just reporting that there is now, say, 15 specification methods without implementations).

Class Relationships

A key high level design activity is specify the relationship of classes. SDD does this in the implementation classes an uses the compiler to check that the relationships are valid. One problem with this is that it is sometimes difficult to get compilable code that has a relationship with other classes when neither class is fully implemented.

SDD introduces some helper constructs to get around this problem. In C++ they are defined similar to the following:


#ifndef SDD_H
#define SDD_H

#if defined( SDD_ALLOW ) || defined( SDD_SILENT )
    template< class T >
    class sdd_class
    {
    public:
        typedef T Tuses;
        sdd_class() : p( 0 ) {}
        T * operator ->() { return p; }
    private:
        T * p;
    };

    // sdd_subsystem<T> is a documentation template to indicate
    // that a class containing it is associated with a particular
    // sub-system. A class is used to identify a sub-system.
    // Typically the sub-system classes would be listed in a
    // header file. Tools may also be able to use C++ namespaces
    // to partition classes into sub-systems.
    template< class T >
    class sdd_subsystem
    {};

    #ifdef SDD_SILENT
        #define SDD_SUBSYSTEM( the_subsystem )
        #define SDD_USES_CLASS( the_class )
        #define SDD_IMPLEMENTS( the_method_group )
        #define SDD_USES( the_class, the_method_group )
        #define SDD_CALLS_SPECIFIC( the_class, the_method )
    #else
        #define SDD_SUBSYSTEM( the_subsystem ) \
                    typedef sdd_subsystem< the_subsystem > \
                                sdd_subsystem_ ## the_subsystem;
        #define SDD_USES_CLASS( the_class ) \
                    typedef sdd_class< the_class > sdd_ ## the_class;
        #define SDD_IMPLEMENTS( the_method_group ) \
                    struct sdd_ ## the_method_group {};
        #define SDD_USES( the_class, the_method_group ) \
                    { typedef sdd_ ## the_class :: Tuses :: \
                            sdd_ ## the_method_group x; };
        #define SDD_CALLS_SPECIFIC( the_class, the_method ) \
                    { if( 0 ) { sdd_ ## the_class() -> the_method; } }
    #endif
#else
    // Uses undefined
#endif

#endif // SDD_H

An example of its usage, the class “foo” uses functionality from class “bar”. “foo.h” may look like this:


class wibble {};    // Actually defined elsewhere

class foo
{
private:
    SDD_SUBSYSTEM( wibble )
    SDD_USES_CLASS( bar )

public:
    void action()
    {
        SDD_USES( bar, length )
        //SDD_USES( bar, scale )	// Causes error

        // A more specific statement of usage
        SDD_CALLS_SPECIFIC( bar, length() )
        SDD_CALLS_SPECIFIC( bar, length(2, 3) )
    }
};

“bar.h” might look like:


class bar
{
public:
    bar( int t ) {}

    SDD_IMPLEMENTS( length )
    size_t length() const;
    size_t length(int, int) const;
};

“bar-spec.cpp” might look like:


class bar_spec
{
public:
    void run()
    {
        length();
    }

private:
    void length()
    {
        // The length method return the length of the object
    }
};

In the above, the “SDD_USES_CLASS” macro is used in the “foo” class to indicate that the “foo” class uses the “bar” class. In the “bar”class the “SDD_IMPLEMENTS” macro indicates that the class supports some functionality called “length”. It is intentionally a high level statement and purposefully omits whether the method takes any arguments and what it returns. In the “action” method in the “foo” class the “SDD_USES” macro indicates that it uses the “length” methods defined in “bar”. Again, this is a high level statement and no attempt is made to capture what sorts of arguments and return values are involved.

The “uses_class” template, and the “SDD_IMPLEMENTS” and “SDD_IMPLEMENTS” macros are intended to be sufficiently descriptive of the high level structure that UML diagrams can be automatically generated from the code in order to allow initial high level code reviews and walk throughs.

They also have sufficient mechanics to them to allow the compiler to check that “bar” does indeed exist for “foo” to be able to use it and the “bar” has declared that it does support some functionality named “length”.

Thus these provisions allow developers to sketch in the sort of functionality that “foo” and it’s methods require from “bar” without getting bogged down into having to prematurely implement functionality just to get the code to compile. This makes it much easier to iterate though a high level design that is validated by the compiler. Hopefully tools will also be able to detect whether a class offers functionality that is not actually used. Alternatively if a certain piece of functionality is suspected to not be required the relevant “SDD_IMPLEMENTS” statement could be commented out and the compiler will report errors if the hypothesis is incorrect.

Over time, as the software is developed, definitions like the “SDD_USES_CLASS” macro and the “SDD_IMPLEMENTS” and “SDD_USES” macros will be replaced by real instances or pointers to objects etc. As an intermediate step a developer could use something like the “uses_bar->length();” statement above. This states that the code is intended to use a specific instance of a method defined by “bar”. However, because it is access by the “uses_bar” object which is an instance of the “uses_class” template it indicates that this is not the final form.

The example also includes the “bar_spec” class which is the specification for the “bar” class. It shows that the specification for the “length” functionality, which in this case is undefined. The specification validation framework will report on specifications that are not implemented.

If a class has multiple overloaded methods with the same name, then each variant should have its own method and suitable tests. Some form of name mangling should be used to tie the implementation method to it’s specification function so that automated code can check if there are any missing specifications.

The “SDD_USES_CLASS” macro and the “SDD_IMPLEMENTS” and “SDD_USES” macros are included in a project by setting the “SDD_ALLOW” definition in the project settings. As various files are more and more defined the developers stop including the “SDD_ALLOW” definition in the project.

In this sense the “SDD_USES_CLASS” macro and the “SDD_IMPLEMENTS” and “SDD_USES” macros are like scaffolding that supports the construction of a building. Once the building is built the scaffolding is removed. The compiler can validate that all the scaffolding has been removed by not setting the “SDD_ALLOW” definition.

Part of the functionality of the daily reporting software could be how many instances of the various macros are in place and how they are changing.

Note that you should be able to supply a saleable product to a customer while still having high-level, unimplemented specifications in the code that are ready to be worked on in the next release cycle.

Conclusion

SDD is intended to be a light weight methodology that aims to maximise the amount of design information captured in the code base itself, such that as the code based specification is refined the specification yields the final code. It captures a number of the benefits of agile methodology such as writing tests to define functionality, but also allows up front design that can be validated by the compiler with the goal of reducing the amount of development rework required due refactoring.

Refactoring will still be part of a SDD project, but the refactoring can be done at a higher level of abstraction. As refactoring at a higher level means that less lines of code is thrown away during a refactoring exercise it is hoped that developers will be more inclined to refactor as they will be less wedded to the volume of code that they have already produced.

As the code is the specification and the specification is the code, there is no divergence between code and specification as requirements change over the life of a project.

While SDD does allow developers to do design up front, how much design up front is done is a question for each development team. It may just be enough to see them through the next sprint, or it may be sufficient that the developers feel that the structure will not change dramatically over the lifetime of the project.

C++ hack to avoid new private methods causing large re-builds

January 27th, 2012

I blogged recently that one of the things that annoyed me about C++ was that private methods have to be declared in the class header file (http://codalogic.com/blogs/pete/?p=260). This means that all files that include that header file need to be re-compiled, even if there is no real impact on those files. This slows down development and as a result is a dis-incentive to break code functionality into small functions leading to code that isn’t as clean as it should be.

To see if there’s a way around this I came up with the following hack. The header file looks like this:


#ifndef FOO_PRIVATE_METHODS_UNDER_CONSTRUCTION
#define FOO_PRIVATE_METHODS_UNDER_CONSTRUCTION
#endif

class foo
{
public:
    foo( size_t n_times_in );
    void say_it() const;

private:
    size_t n_times;

    FOO_PRIVATE_METHODS_UNDER_CONSTRUCTION
};

And the .cpp file looks like this:


#define FOO_PRIVATE_METHODS_UNDER_CONSTRUCTION \
    const char * const hello_world() const;

#include "foo.h"

#include <iostream>

foo::foo( size_t n_times_in )
    : n_times( n_times_in )
{
}

void foo::say_it() const
{
    for( size_t i=0; i<n_times; ++i )
        std::cout << hello_world();
}

const char * const foo::hello_world() const
{
    return "Hello World\n";
}

The key bit is the FOO_PRIVATE_METHODS_UNDER_CONSTRUCTION macro. When the foo.cpp is compiled, this contains the declarations of the private methods (e.g. foo::hello_world()), and so the file compiles OK. When .cpp files other than foo.cpp are compiled, FOO_PRIVATE_METHODS_UNDER_CONSTRUCTION is not pre-defined and so the header creates its own empty instance of it.

The result is that private methods can be added to foo.cpp without requiring foo.h to be modified. Thus refactoring foo.cpp doesn’t cause other secondary files to need re-building, hence improving productivity.

That’s the up side. The down side is that it’s very hacky, not least because it uses the pre-processor.

Naturally you can move the private method declarations to the header file once you’ve stabilised on a suitable structure, so you could argue that it’s only used during “draft mode”.

So what do you think? Do you think the problem this attempts to solve is worth solving? Do the benefits out-weigh the ugliness of the hack? Is purity more important than productivity? Let me know.

Solving “the function’s difference is in the middle” problem using C++ templates

January 26th, 2012

It’s not uncommon for a bunch of functions to include a common start and common end, but differ in the middle. You can avoid this repetition using an if() or switch() block in the middle of a single function, or you can write a function that takes an interface class as an argument.

Here is a different solution to the problem using C++ templates. Here’s an example of the header file code:


class foo
{
public:
    void op_1();
    void op_2();

private:
    template< class T >
    void common( const T & );

    class diff_id_1 {};
    void difference( const diff_id_1 & );
    class diff_id_2 {};
    void difference( const diff_id_2 & );
};

And here’s an example of the .cpp file code:


#include "diff-in-middle.h"

#include 

template< class T >
void foo::common( const T & )
{
    std::cout << "Common start\n";
    difference( T() );
    std::cout << "Common end\n";
}

void foo::difference( const diff_id_1 & )
{
    std::cout << "Diff middle 1\n";
}

void foo::difference( const diff_id_2 & )
{
    std::cout << "Diff middle 2\n";
}

void foo::op_1()
{
    common( diff_id_1() );
}

void foo::op_2()
{
    common( diff_id_2() );
}

int main(int argc, char * argv[])
{
    foo f;
    f.op_1();
    f.op_2();
    f.op_1();
    return 0;
}

In the above, the common structure is coded in the foo::common() template method. The different ‘middles’ are coded in the variously overloaded foo::difference() methods.

The foo::common() template is templatised using class T, which is the type of the argument to foo::common(). By calling foo::common() with different types, the same common code gets called, but different versions of foo::difference() get called. In the above case the empty classes diff_id_1 and diff_id_2 (defined within the foo class) are used as keys to select the version of foo:difference() that is called.

To make the code more natural to use single line helper methods can be added to call foo::common() with the desired type, as demonstrated by foo::op_1() and foo::op_2().

The result is you get much DRYer code, and the foo::difference() methods have much more ready access to class state than would be the case if interface classes were used. I think it’s also more elegant than putting a if() or switch() block in the middle of the code.

Naturally there are a number of combinations you could use this with. For example, if the various foo::difference() methods required different arguments you could include data members in the respective diff_id_1 and diff_id_2 keying classes and initialise these appropriately in the foo::op_1() and foo::op_2() style helpers.

What do you think? What other solutions do you use to solve this problem?

Call to Deprecate C++’s “Declare Function Before Use” Rule

January 14th, 2012

In C and C++, as a minimum, a function must be declared before it is used. This often leads to you having to start reading code at the bottom of a file and then work up. This is unnatural and presents an small, but significant barrier to code comprehension.

An even bigger problem is that private methods in classes have to be declared in the class before they can be used. As the class definition is typically in a header file that may be a dependency of a large number of files this can result in significant re-compilation times if a new private method is introduced, even though the change has no effect of the ‘dependent’ files.

The result is that there is either a resistance to add a new private method, which again leads to less readable code, or people try to work around deficiencies in the language using idioms such as pImpl, which makes code more complex, affects performance and takes longer to write.

These problems could be significantly reduced if the “Declare Function Before Use” rule was deprecated. Doing this would align C++ with other languages such as C# and Java, and also with the way inline methods behave within a class definition.

Therefore, instead of requiring a function to be declared before use, an additional pass of a compilation unit (i.e. file) could be made to find out the prototypes of each contained function. Then, any function that was called within the same file as it is defined would not have to be declared before use.

I say “an additional pass,” but C++ has long since moved on since the early C days of 3 pass compilation and probably already includes a suitable pass, for example, to compile inline methods within a class body.

Even if an additional pass was required, a file is likely to be in RAM cache and so the extra pass would be essentially undetectable to a human user, and would certainly be far more time efficient than pointlessly re-building all the dependent files.

So what would code look like that exploited deprecating the “Declare Function Before Use” rule?

A simple procedural example might look like:


int main()
{
    std::cout << hello_world();
}

const char * const hello_world()
{
    return "Hello World\n";
}

This now allows for more natural top-to-bottom reading.

In the case of class based methods you could have:


// In foo.h
class foo
{
public:
    foo( size_t n_times_in );
    void say_it() const;

private:
    size_t n_times;
};

// In foo.cpp
foo::foo( size_t n_times_in )
    : n_times( n_times_in )
{
}

void foo::say_it() const
{
    for( size_t i=0; i<n_times; ++i )
        std::cout << hello_world();
}

const char * const foo::hello_world() const
{
    return "Hello World\n";
}

Notice that we still have to include the private data members within the class. But this is acceptable because the data members tend to change less frequently than the private methods. Also note that we wouldn’t be able to call foo::hello_world() from, say, foo2.cpp.

I believe this would make a major improvement in C++ programmer efficiency. It also makes the class interface cleaner. Is there any down-side to it? I can’t see one. Should I suggest it to the C++ technical committee?

Could local functions make for cleaner code?

December 6th, 2011

I’ve blogged a couple of times about using a class within a function to implement complex functionality (http://codalogic.com/blogs/pete/?p=206). This allows decomposing the implementation of a function while keeping all the details nicely grouped together.

Another benefit is that the sub-functions implementing the functionality can share state (in the form of the class variables). This is something that sibling functions can’t do and is something that is often a hinderance to decomposing functionality into multiple sibling functions that have to share state via function arguments.

Placing a class inside a function does however have the downside of resulting in quite a bit of clutter within the function – and clutter is bad. What I’m really looking for is functions inside functions and the ability of an inner function to share state of outer functions.

I have previously never seen the point of local functions, but here they fit the bill perfectly. For example, you could have code like:


// N.B. This is not real C++!
class my_class
{
    int class_var = 0;

    void my_func()
    {
        int my_func_var = 0;

        if( is_too_hot() )
            cool_down();
        else
            warm_up();

        bool is_too_hot()
        {
            // Local function can access variable in it's own
            // -scope, variables in it's ancestor's scope,
            // -class scope (and global scope).
            int is_too_hot_var = class_var + 10;
            return my_func_var < is_too_hot_var;
        }

        void cool_down()
        {
            //...
        }

        void warm_up()
        {
            //...
        }
    }
};

This results in very cohesive, clean code. Here’s hoping C++1x supports this in future!

“If” should not be part of a GUI spec

December 5th, 2011

I’m no GUI guru, but being a GUI user has lead me to believe that GUI behaviour should not include “if”.

For example, cases where “if” leads to a bad GUI experience include:

- IF the user has recently used a menu option… This means the menu layout changes each time you use it, slowing you down.

- IF an instance of a program is already open… This means there’s two ways to open a program and the users interaction is consequently more complicated.

- IF the user hasn’t done an action for a while… Means that something will be changed while the user is looking at it, meaning the user will have to start looking for what they want all over again.

Using “if” in your GUI spec means that the GUI is changing. Users don’t want to look at the whole of the GUI all of the time. They want to learn where each of the functions they want are located and just go to them on auto-pilot. Having “if” in your GUI spec prevents this. It’s a case of “smart is dumb”.

Why I prefer XP to Windows 7

December 5th, 2011

I use XP on my desktop system and Windows 7 on my laptop. While I like Windows 7, I still prefer XP. Here’s why:

- The start button doesn’t support classic mode. In XP classic mode program selection can take up the entire screen if it needs to, whereas in Windows 7 you are hemmed in to a small area that sometimes can’t even display the entire program name.

- When you use the start menu, the recently used programs display just long enough for you to work out that your desired program is not there. Then it automatically moves to the main programs list, just as you click the button to do the same thing; with the result that you end up moving straight back to the recently used program list!

- Win 7 seems to be more verbose. When you exit a program without saving XP will display something like “Save? [yes] [no].” Win 7 is far more verbose and displays something the like “The file has changed. If you don’t save it your changes will be lost. Do you wish to save it? [Save] [Don't save].” The Win 7 unmount USB stick dialog also displays roughly twice as much text as XP. Win 7 is just cluttered.

- I prefer the quick launch feature on the XP toolbar. Win 7’s pinning means that, IF I don’t have an instance of what I want open, then I do a single left mouse click on the icon, whereas IF I have a instance open, then I right mouse click, move up and then left click. The Win 7 approach is worse because of the IFs involved – and it’s more clicks. Additional, if you’re opening a new instance under Win 7, the close window command is very near the new window command. Not so good if you accidentally hit your touchpad too heavily by mistake!

- Even if you’re opening an instance for the first time on Win 7, the place you have to click changes depending on what other applications are open. That means I have to look for what I want rather than just going by reflex.

- The inactive pinned applications take up more space than the XP quick launch toolbar.

- Double clicking on a file in Windows 7’s Explorer’s list view that is on the right hand side doesn’t work. The selected files moves on the first click meaning your double click ends up hitting the wrong thing! XP doesn’t change the layout on the first click, giving you the result you expect.

- When you create a new file in Windows Explorer on Win 7 and then rename it, it will place the new file in its alphabetically sorted position. On the face of it this seems like the right thing to do, but the consequence of this is that you then have to visually search for the file you just created. While XP’s approach of placing the new file at the end of the list seems less sophisticated, it’s actually easier to use. If you want things in order under XP just press F5.

- Moving a window towards the top of the screen causes the window to maximize in Windows 7. I can press the maximize button or, even easier, double click on the title bar to achieve this. Why do I need another way? Especially a way that prevents me moving un-maximized windows towards the top of the screen.

No doubt people will tell me I’m using Win 7 the wrong way. Maybe I prefer XP just because I’ve learnt to cope with its foibles and I don’t notice them anymore. There are things I like about Windows 7; such as display mini-images of a application window when you mouse-over an icon on the taskbar and that it displays the date along with the time. But ultimately, one of the premises for upgrading from XP to Windows 7 is that it will make me more productive. And I have to say that that just does not hold true for me.

std::ivector - A new container type for C++?

November 3rd, 2011

A while back someone asked if anyone had any suggestions for additions to the C++ STL library.  Belatedly I thinking what would be handy is a container that had std::vector-like semantics, but stored it’s values in separately allocated objects that were pointed to by the values stored in the internal vector container.  Let’s call it an ivector for “indirect vector”.

Thus, if you had template< class T > class ivector…, it would (conceptually) contain an instance of:

std::vector< T * > pointers;

Or more likely:

std::vector< std::unique_ptr< T > > pointers;

But the at() method would be declared as:

T & at( size_t i ) { return *pointers.at(i); }

While you could easily implement this yourself, the benefit of having it as a standard library feature is that you can easily flip between vector and ivector without changing any of your other code.

What we need is an end to stocks and shares

October 21st, 2011

The Occupy XYZ movement has made me think…

We don’t need an end to capitalism. What we need is an end to stocks and shares.

Generally the Western world has done very well by capitalism. But trading in stocks and shares allows institutions to devisively make money without contributing anything. If an alternative to financing companies could be found the world would be a better place.

For example, profit sharing corporate bonds.  A company could issue bonds saying it would pay 10% of profits to the bearer over the next 10 years.

This would remove the boom and bust of over inflated corporate valuations.  It would also remove the need for continuous increases in short term share holder value.

Sure, there might a few less billionaires at IPOs or in banks, but generally the world would be a more stable, equitable place and 99% of us would be better off.

C++ IDEs should implement speculative compilation

October 17th, 2011

One of the problems with C++ is that a minor change to a header file can require a large number of files that use that file to need re-compilation. This is particularly annoying if the addition to the header file is something like an additional private method prototype that has no impact on the other files. This feature can affect the way code is written, meaning that sub-optimal code can be written just to accommodation the limitations of the development environment.

It’s therefore proposed that IDEs should continuously, speculatively compile files in the background whose dependencies have changed. For example, the compiler could continuously attempt to compile the least recently edited file whose dependencies have been changed. The results of such compilations would be stored in the background and only displayed when the developer explicitly asks for that file to be compiled. In that scenario, if an up to date speculative compilation result exists, then when a developer asks for a compilation to be performed, all that is required is for the results to be shown, with the result that the compilation appears to be instantaneous.

IDEs should also have a deeper understanding of what a dependency on an included file means. For example, if a file previously compiled OK and the only change to a header file is the addition of a new method prototype then there should not be a need to re-compile the file, and the file can be marked as compiled OK. While the assessment of whether a re-compilation is required may be non-trival, such a process should be much quicker than performing a full re-compile with optimisations - that yields the same result as the previous compile.

For example, the IDE could maintain an inventory of what resources each compilation unit references to in its header files. This may include the sizes of class member variables in referenced classes, the values of constants (and #defines!) and the prototypes that the file links to (taking into account implicit compiler conversions). From that, the compiler can make a decision about whether a full compile is required, thus only requiring a re-compile if a change is significant. (Some compilers currently do something like this, but as I understand it, they only look for things like white-space changes. This scheme would look deeper.)

As a result of this, compilation would be faster and developers would be less likely to drift off and look at the news and Twitter while their code compiled, dramatically increasing developer productivity. IDE developers, please sort it out! Thanks.