• To begin with, there are a lot of them.
• They are represented using a mix of ints, longs and strings. Even the same id may be represented using different data types in different places.
• We’ve had bugs where an object with multiple ids has been put into a map using one of its ids as key and someone else tries to get it out using another id as key.
• Their names are sometimes confusingly similar.
• The same id may have multiple names, used intermixed.

Therefore, we’ve discussed various ways of getting a grip on the situation. One of the alternatives, which I think sounds rather good, is to use a separate class to represent each type of id.

For example, a User class doesn’t have a int userId field, but a UserId userId field. How UserId internally represents the id (perhaps as a long or string), is up to that class.

Some of the pros of this solution, from a Java perspective, are:

• When the id is used “out of context”, you don’t have to rely on variable naming to convey its meaning, you will get static type checking to help you.
• If you want to change the internal representation (int, long, etc), you only need to change a rather limited number of places instead of all over the system.
• You get an opportunity to create a suitable toString(), hashCode(), and or equals() implementation.
• You get a natural place for common convenience methods regarding the id, reducing the need for various “util classes” (which really are more procedural than object oriented).

The primary drawback that I can think of is that it generates a lot more small objects which may increase memory usage. But until it has been proven to be an actual problem in a specific case, I think that is a reasonable cost for the benefits above.

If you would like to read through the description and leave a comment or two I would be very grateful! Feel free to just add a comment or two in this document.

I’m using a fictional weather service as example. It does something along the lines of fetching the current weather data from a web service, parse it and extract relevant data, fetch some information about the current geographical location, and finally analyze them to provide the user with some interesting information about the weather.

Design A

The system is designed with the majority of the code as a set of completely independent modules which are joined together with a small amount of glue code.

Classes and modules tend to send primitive data or value objects to each other rather than intelligent classes. Modules tend to have none or very few dependencies. Re-use of modules is simplified because they can be used independently of each other.

Modules tend to be stateless with any state kept in the caller/glue code. External dependencies are kept as shallow as possible in the call/dependency tree.

The sequence of calls needed to perform something is encoded in the glue code rather than spread out in the system. It is up to the caller needs to know in which order to call modules. It delegates the execution of each step to the modules, but keeps control. The call stack will always remain rather short.

Each module can be unit tested in isolation and internal classes inside the modules do not have separate tests. The glue code is tested through integration tests that run end-to-end to verify that the system works. Unit tests often become easier to write because there is no need for mocking dependencies.

This approach is more “functional” in nature as modules tend to be stateless, get all required data as input and return the result as output.

Design B

The system is designed as a set of interacting, often stateful, modules which are joined together in a graph of dependencies.

One module often calls another module without providing all information necessary to complete a task, secure in the knowledge that the other module can obtain it. This is typically performed through injection of dependencies. Modules therefore often have multiple dependencies.

Modules are often stateful or depends on objects which are. External dependencies are often pushed as deep down the dependency tree as possible.

The sequence of calls needed to perform something is often spread out over a set of modules and partially encoded in the dependency setup. The caller can often call one or a few modules knowing they will in turn call others. The caller delegates not only the execution of tasks but also control to a large degree. The call stack will in the middle of execution be rather deep.

Each module is unit tested, typically with any dependencies replaced with stub/mock/fake instances. Integration tests ensure that the wiring of the modules is correct and that the system works together. There is little fundamental difference between tests on different levels.

This approach is perhaps more “object oriented” in nature where the system is divided into a set of stateful, intelligent objects that cooperate in generating the output.

Questions

I would appreciate any comments or input, such as your thoughts about:

• Which alternative you prefer? Which one do you most often use? Perhaps a hybrid?
• Under which circumstances is one better than the other?
• Are they good at different levels? E.g. method, class, component, or system level.
• Is there a “OO vs functional” difference, as I suggest?
• Is either of them easier to maintain than the other?
• Can they be described more clearly? What are they called?
• Have I gotten anything or everything completely wrong?

Feel free to enter any feedback or comments below.

Example

In the project I’m working on, I had a class called SelectedInstalledBase which represented the subset of products in a customer’s “installed base” selected by the user. (An “installed base” is this domain is all the products a customer has bought.)

That was fine until I wanted an inverted version of the selection, i.e. the installed base not selected (the “remaining installed base”). Then, it suddenly didn’t make much sense to create a remaining installed base of type SelectedInstalledBase. It also didn’t make sense to create a clone of SelectedInstalledBase named RemainingInstalledBase as they would be identical apart from the name.

The solution the problem was renaming SelectedInstalledBase to InstalledBaseIdentifier (because it represented the ids of the selection made by the user) and naming the variables to reflect the different cases, such as InstalledBaseIdentifier selectedInstalledBase and InstalledBaseIdentifier remainingInstalledBase.

In some cases I might have rewritten, changed, extended or simply misunderstood what he was saying, but these were my notes from the course, nevertheless.

• Doing TDD is like losing weight. You like having lost some weight, but not actually losing it.
• Taking really, really small TDD steps is useful when you’re unsure/confused about the code you are writing. That way you are at least moving forward.
• What kills productivity isn’t doing something slowly, but not doing it. I.e. when you switch to reading email, surfing the web, etc instead of working.
• Implementing a piece of code inside a test keeps your options open if you’re not sure about the future interface.
• Start writing the test which you expect to learn the most from.
• Most important, according to Kent: Writing a test is really telling a story about the code. Having that mindset helps you work out many other problems with testing.
• Deleting tests comes down to story telling – which tests do we need to tell the story about this code. (There is also a technical/coverage aspect, but it is of lower importance).
• When unsure about whether to clean something up or not – think about the amount of leverage it will give you. Worth it?
• When coding exploratory (without TDD), start TDD:ing when/if you don’t expect the code to die anymore, or when you think you’ve learned the most important lessons.
• In many cases: The more discussion, the less important the decision. Everyone agrees about the really important ones.
• Write code in a functional style, as it is more suitable for testing.
• Consider not using dependency injection as it makes TDD harder by encouraging a design where dependencies are spread out over the system. Instead, design many stand-alone modules which are unit tested individually without the need for mocked dependencies, then join them together in other classes which are tested through integration tests.
• Avoid abstact superclasses for unit test classes as they make the individual test classes harder to read.
• Duplication is often more okay in tests than in the production code because readability of tests is so important.
• For infrastructure-related setup code, @Rule can be used instead of inheritance in many cases.
• Write tests on the high and low level, but be weary of too many tests on the mid level. They tend to be more of a problem than help when refactoring.
• Kent: “I try not to have more tests than I need”
• Use one test class per fixture (i.e. set of test data). Name the classes after the fixture. That is how JUnit is designed and intended to be used.

I ended up with a set of these methods, most of them accepting the same parameter. In this case, the common object was a Set. Seeing the pattern emerge, I created a new object called ProductSet and moved all the private static methods there. In this class, they would instead become public non-static methods. The code immediately felt better. As I continued working, I found even more code that really belonged on this ProductSet object and it became one of the main abstractions in the code.

So, to summarize, my insight of the day:

A sign that the Move Method refactoring might be appropriate is that you have private static methods in a class (or private methods that could as well be static). If it doesn’t use any of the class’ fields, then maybe it belongs elsewhere.

Today in the programming industry we have a similar problem, where developers and others involved in software development often underestimate the risks resulting from the absence of an automated test suite. Alternatively, they feel that the pressure to deliver quickly is too large. As a result many projects, even very large and complex ones, have no automated tests that can provide a warning signal when serious errors are introduced.

Maybe we simply need to demand that basic automated tests should be available to all code that is written. The larger and more complex the system, the higher the demands, in the same way that fire safety requirements of a skyscraper is several magnitudes higher than those for a cottage. Maybe it would be a bit overkill to write these requirements into a country’s laws. It might be a better idea to have them be built into the culture of our industry. That companies who does not follow the rules get a bad reputation that clients would not want to do business with, just as you wouldn’t want to use a contractor who ignores fire safety.

When committing changes to a Cayenne context, you get an exception like the following.

org.apache.cayenne.CayenneRuntimeException:
(v.3.0.1 Sep 06 2010 15:09:38)
Temporary ID hasn't been replaced on commit

Cause

You are trying to commit an object for which Cayenne does not know how to generate a permanent id. This in turn typically happens when the underlying table has no primary key defined.

Solution

To solve the problem, do one of the following.

• Add a primary key to the underlying table, either a surrogate key (new generated id column) or a natural key (combination of existing columns).
• Set your own ObjectID on the object(s) in question using the PersistentObject.setObjectID() method.

When committing changes to a Cayenne context, you get an exception like the following.

org.apache.cayenne.CayenneRuntimeException:
(v.3.0.1 Sep 06 2010 15:09:38)
Error generating PK : entity not supported: SOME_TABLE

Cause

A column that is part of the primary key of SOME_TABLE has no value. Therefore Cayenne tries to generate one, but does not know how to.

Solution

To solve the problem, do one of the following.

• If the value is supposed to be auto-generated, tell Cayenne how to generate the value.
• If it is not supposed to be generated, provide a value before committing changes in the context.

One type of interface, which we might call separation interface, aims to separate different components. In this case, a component only exposes interfaces for its functionality but not the actual classes to make sure that clients aren’t too tightly coupled with the implementation. In this scenario, there is typically ever only a single implementation of the interface and both the interface and the implementation have the same author(s). Whenever you see a Interface/InterfaceImpl pair, you are most likely looking a this kind of interface use. This pattern is considered a best practice between components and systems, but rarely useful inside cohesive modules where classes naturally are more tightly coupled.

The other type of interface, let’s call it extension interface, is intended to represent an ability than an object can have. Here, multiple implementations of the interface exists and the client can choose the implementation that best suites his or her needs. A component often defines an interface on which it depends in order to allow implementations to be added at a later time. Often, the author of the interface is not even the same as the ones who create the implementations. This type of interfaces (together with regular inheritance) are the foundation of polymorphism. Iterable or Comparator are typical examples.

Quite possibly, there are other types of interfaces as well. Feel free to comment on the issue.

They also created a new pedestrian crossing next to the tram stop. And there was one thing about it that I noticed. One side of the crossing was connected to the tram stop and the curb there was just rebuilt. The other side of the street had a perfectly fine curb, and it would have been no problems to just paint lines on the street and be done with the crossing. But they didn’t.

Instead, they ripped up the street, curb and part of the sidewalk on that side too. Why? The reason is that they wanted to improve that side of the crossing too. They’re making the sidewalk about half a meter wider and thereby narrowing the street. That makes it a little bit clearer that it is a crossing and therefore makes the crossing safer. They’ll lower the curb for a meter or so which makes crossing by bike or wheel-chair easier. That’s about it, as far as I know.

While arguably better, it didn’t really enable anything that wasn’t possible before. And it did cost money. Money that could have been spent elsewhere. And they had already spent a lot of money on rebuilding the tram stop. So why did they do it? I’m not sure, but I think whatever the reasons were, the same reasons could be used to motivate refactoring of software.

(Alright, that was possibly a crappy metaphor… but it’s my blog, and I’ll post what I want. )

You can finally get number of how much that “technical debt” really is. How much an average bug cost? How long does it on average take until a bug is discovered? Until it is fixed? How much did that quick and dirty patch a year ago really cost? Add to that all the technical value you could get out of such an analysis. What modules/classes have had the most bugs, historically? What classes are unstable and has to be changed all the time?

Think of it as a Freakonomics of version control.

Multitasking doesn’t work,
but it would be so nice if it did
that we try anyway.

That’s about the best summary of (human) multitasking I’ve heard.

This picture is pretty good too, though.

“With great power comes great responsibility”

So, if you can’t do it all yourself, you’re forced to either:

1. generalize and set broad rules, or
2. delegate control and responsibility.

The first one is tempting, as you can control people and ensure that they don’t do anything which they should not. The problem is, if you do this, you lower the “consciousness” of yourself and your organization. According to the Dreyfus model of skill acquisition, not taking context into consideration is one of the traits of the novice. And that is just what’s happening here.

On the other hand, if you delegate control to the people who actually deal with the everyday problems, each person can adjust their decisions to the situation at hand. If you don’t trust the people to handle that situation, either learn to trust them or don’t hire them in the first place!

I very much liked the idea, that a set of very low level instructions, carefully chosen and carefully followed, could create something so complex and beautiful. Jurgen Appelo has expressed similar ideas regarding complexity theory, and how ant hills or bee hives are complex structures created without a master plan through individual bees’ actions. Yet another connection would be local search algorithms, which find maxima in a landscape through looking only at their immediate surroundings.

Next, my brain started linking these ideas to software development. For example, Test-Driven Development. It is in its essence, a set of very simple rules. Write a test, write needed implementation, refactor. These three simple (at least in theory) steps is all that there is to it, when properly adhered to, it can give great results. Designs tend to be much more easy to use, contain less bloat, be more testable (duh!), and so on. Refactorings are another great example. A set of rules or recipes for how to transform a code base into a better code base.

Obviously, choosing the right set of rules to follow is the important part, as a bad set of rules will lead to crappy code just as an unwanted mutation in a human body may lead to e.g. cancer. But I guess the point I was trying to make is that there is immense power in follow simple rules.

My lunch mates felt that the decision to go with this vendor was not necessarily in the best interest of their company and the technical platform they were developing. Instead, they felt that there were other ways forward which would be better. Some of these ideas included common-sense things such as choosing quality over quantity, working on getting the current code into decent shape, and so on. They also felt that they had been trying to get this message across to management.

Why then did these suggestions not catch on with management, and why did the vendor’s salesmen manage to convince company management to choose them? I don’t have the answers, but I will not let that stop me from sharing my thoughts. I also don’t have all the details on this specific case, so I’ll be a bit more general.

I think that developers and architects have many good ideas on how to build systems, but that we don’t always do a great job of describing these ideas to business people. I think we also sometimes adopt ideas on the basis of liking them, rather than because the idea has been proven over and over again.

Failing that, we need to get a very well-paid management consultant to tell them the same thing. Because in the world of business, the greatness of an idea is proportional to the hourly rate of the person presenting it.

Some people who use test-driven development, also use tools that allow the to peek into classes and read private variables. That is an efficient way to ensure that some method altered the state of the object in the expected way.

However, as Michael pointed out, by doing so they miss the point of test-driven development. Test-driven development adds constraints. It does that in order to force developers to design their code better. (TDD is about design, not testing — remember?) By peeking into classes, one effectively removes one constraint added by TDD, and thereby removing much of the benefit of TDD.

Wow. That a blog post in itself—and a great warning to all us who are tempted to “go pro.”

So, consider yourselves warned, everyone. (Now, that would have worked way better if I actually had a lot of subscribers…)

His post made me think of how many bloggers that become popular behave. At first, they write good high-quality material, the kind of material that people want to read, and they get a lot of subscribers. As they become famous, they start doing talking engagements, they write books, they quit their jobs and starts making a livings as bloggers. And before you know it, most of what they post are “spam” such as info about their upcoming talk, book or business deal. And the posts which made me interested in the first place become rarer.

Now, that is only logical. They wrote about the subject in the first place because they were interested and worked with it everyday. When their life situation changes, and they become pro-bloggers, naturally their interests and day-to-day activities change as well (to some degree, at least). So they still write about what they are interested in and work with everyday. Same, same, but different.

Another commenter mentioned that he only followed people on Twitter with a moderate amount of followers and not the ones with millions, because the latter typically just sent broadcast-style marketing messages. Maybe that’s an expression of the same thing?

Constraints are good.

To me, that statement sounded rather odd. Constraints is a negative word to me, so intuitively I want to remove constraints. Thus, I asked him to clarify. And here’s what he said (or rather, my interpretation of it).

A big advantage of software development is that we can build virtually anything. However, that same fact also represents one of the biggest problems. “Anything” includes all the great things that we envision, but also all the crappy things which we typically produce.

Since the possibilities are endless, in theory nothing prevents us from creating something truly great. But — and here’s the thing — by adding wisely chosen constraints, we can eliminate some of the undesirable outcomes. That means, that even if we screw up a bit, we’re more likely to create something which is at least not half-bad because we’ve eliminated the really bad outcomes. If we stay within the constraints, that is…

Another way to see it is that just as any society needs some laws and rules in order to function smoothly, a software development team does too. Whereas an example in the first case may be “do not kill”, an example in the second could be “do not commit code which breaks the build”.

The rule

• Make a good estimate of the work (in any time unit).
• Multiply that number by itself in months plus one.
• What you get is a reasonable estimate for release date (in the original time unit).

Examples

Some examples:

• Work estimated to one week will take one week.
• Work estimated to two weeks you can expect to be ready in close to a month.
• Work expected to take two months weeks will take about six months.
• Anything estimated to half a year or more won’t be finished. The scope will be changed too much before the project is finished.

While not perfectly scientific, it is actually more or less empirically derived from my own company’s history.

A software project’s bus factor is a measurement of concentration of information in a single person, or very few people. The bus factor is the total number of key developers who would if incapacitated, as by getting hit by a bus, send the project into such disarray that it would not be able to proceed.

How would your company do without you if you became sick for a month? Six months? Quit completely? Or put the other way, if you want to quit, can you get out without leaving a complete mess behind you?

If you’re an entrepreneur, this is even more important. Especially if you’re looking into one day selling your company. Nobody wants to buy a company which is worthless without the seller.

What is the bus factor on your project or company?

A summary of the series:

1. Throwing Exceptions
2. Using Assertions
3. Catching Exceptions
4. Logging Exceptions (this one)

General guidelines

We’ll start off with a few general guidelines for logging.

Always include the exception message in the log

Never lose a stack trace!

Debug levels

A summary of the typical severity levels used when logging and guidelines on when to use them.

Info, Debug, or Trace

Warnings

Errors

Fatal errors

I hope this quick four part series on sensible exception handling has been interesting. Be sure to read the previous installments (see summary at top) and please feel free to add your comments below! Any and all comments are very welcome!

To quickly summarize, the series looks as follows:

1. Throwing Exceptions
2. Using Assertions
3. Catching Exceptions (this one)
4. Logging Exceptions

The information in this post is divided into two parts: when to catch, and how to catch.

When to catch

Here are some guidelines on when to catch exceptions and when to let them propagate up through the call stack.

Catch only exceptions you know how to handle, and be specific

Provide a general catch-all

How to catch

When you catch an exception, make sure to do it the right way.

Do not suppress or ignore exceptions

Always clean up after yourself

Declarative exception handling

Next week, the final episode of the series will be published, so be sure to check back. If you have any comments, please provide them below!

A quick summary of posts in this series:

1. Throwing Exceptions
2. Using Assertions (this one)
3. Catching Exceptions
4. Logging Exceptions

Guidelines for assertions

An assertion is code that’s used during development that allows a program to check itself as it runs. They are used much in the same spirit as unit tests, but spread out in the actual program code. It is a statement placed to indicate that the developer thinks that some predicate is always true at that place.

Use assertions for conditions that should never occur

Use a barricades to distinguish between assertions and exceptions

Don’t put executable code into assertions.

For highly robust code, assert and then handle the error any way

Assertions in Java

For those unfamiliar with assertions, I’ll give a quick summary. Java provides the keyword assert since version 1.4. It throws a java.lang.AssertionError when it fails, and has the following syntax:

assert expression : "error message shown if expression is false";

Where expression is any boolean expression which should always be true at the point of the assertion. For example, this variable should always be greater than 0, or that collection should always be empty.

To enable assertions when running a program, use the Java VM -ea (or -enableassertions) switch. Assertions are always compiled in, but without this switch, the assertions are ignored by the virtual machine.

Tune in next week for guidelines for catching exceptions! And please do write any comments or questions you might have below.

The series will cover what I think is a good and sound strategy for handling exceptions and errors in your application.

It is written with Java in mind, but is applicable to most languages which feature exceptions.

A quick summary of the coming posts:

1. Throwing Exceptions (this one)
2. Using Assertions
3. Catching Exceptions
4. Logging Exceptions

Let’s get started!

General

First, we’ll go through some things which apply to all types of exceptions.

Always preserve the stack trace

Never use exceptions for flow control

Provide enough context

Next, we’ll discuss checked and unchecked exceptions, and when to use them.

Checked exceptions

A checked exception is a Java concept which means an exception which is specified in a method’s signature. It is a way for the designer to tell the client of a method what might go wrong, so that the client can prepare accordingly.

Throw a checked exception if a method can’t fulfill it’s contract

Make your exceptions part of your contract

Never let implementation-specific exceptions escalate to the higher layers

Unchecked exceptions

Unchecked exceptions are all other exceptions, which are “unforeseen”.

Prefer unchecked exceptions for all programming errors

If the client cannot do anything useful, then make the exception unchecked

Feel free to provide any comments or thoughts below, and check back in a week for the next episode!

I originally intended to just comment that piece, but the sheer amount of comments made me realize that no-one would actually read it anyway. So I might as well write a response in my own blog which about as many will read but which makes me feel better.

I believe the answer is “yes”. I think that a competent programmer needs to be “mathematically inclined”. Not because the average programmer’s typical work includes much math such as applications in 3D, compression, and image manipulation. In those cases it is fairly obvious that a firm grasp of math is required. I believe the answer is “yes” because in my mind “mathematically inclined” also means “good at seeing patterns”, “good at identifying duplication”, “understands basic complexity theory”, or simply “good at groking thinks”. And while a good understanding of math in general is valuable, I think a solid understanding of discrete math and logic is invaluable.

Finally, I liked what the unnamed author mentioned in Jeff’s post wrote enough to reprint it here.

I run a small (4 people) web dev shop and I’m finding that younger coders haven’t had the pleasure of writing assembler or managing without library functions. I’ve always found strong math skills to be one of the most useful skills for coding, and when one has Google and a massive library of functions, one doesn’t have to be good at math to get things working, until it either breaks, has edge cases, or brings out OS or library bugs.

Some quick examples: simplifying tricky equations to determine array indicies or memory offsets; trigonometry to help with physical calculations; mental hex/bin/dec conversion; logic equalities such as DeMorgan’s theorem.