Unintentional Error Dismissal in Scala

Note: This article is unedited

Scala is a functional language and as such it supports many functional paradigms we expect from a functinal language such as Monads. While I will not over what a monad is in this post, I will be using them to a degree in as examples. Primarily the Future and Either monads.

Problems at Hand

Monads are all about composition. Having a single type leands it self very well to composing values such as:

val teamTotal = for {
  aliceTotal <- Future.unit(100)
  bobTotal <- Future.unit(50)
} yield aliceTotal + bobTotal
// res: Future(150)

This looks rather innocous, and for primitive values it is. We however can start to get into trouble when we start putting together more complex operations:

val teamTotal = for {
  aliceTotal <- getTotalFromDB("alice")
  bobTotal <- getTotalFromDB("bob")
} yield aliceTotal + bobTotal
// teamTotal: Future(150)

While the result in our example looks the same, there exists the possibility of failure within the computation that we have not considered. In order to properly handle the error we need to use either pattern match or use the recovery mechanism provided by our chosen monad. In the case of Future we can use recover.

teamTotal.recover {
  case NonFatal(error) => 0
}

In this case we default to zero if there is a non-fatal exception that occurs in either look up for Alice or Bob. Note that we are otherwise ignoring the error in the recover phase. In production system it may be prudent to log the exception if it’s something that “should not happen” or should only fail in adverse circumstances.

If we had not handled the error case, it would have eventually bubbled up as an exception when we attempted to extract the value inside of the Future.

Capturing the exception anywhere deeper than this and returning a wrapped value like an Either would prevent this error mechanism from working as expected.

Let’s say that getTotalFromDB function returns a Either[Throwable, Int] rather than a plain Int. The function now expresses to us that it will capture a particular set of exceptions.

def getTotalFromDB(key: String): Future[Either[Throwable, Int]]

The code to look up the team total remains similar, but with some changes that are worth noting:

val teamTotal = for {
  aliceTotal <- getTotalFromDB("alice")
  bobTotal <- getTotalFromDB("bob")
} yield aliceTotal.flatMap(a => bobTotal.flatMap(b => a + b))
// teamTotal: Future[Either[Throwable, Int]]

The teamTotal needs to combine the inner Either types, so that if either of the look ups fails it will be bubbled up. This has two effects. First effect is that it lessens the possiblity that we require setting a recover callback on the future, but we’ve only traded that for the requirement to handle an Either result.

Await.result(teamTotal, Duration.Infinite) match {
  case Right(total) => total 
  case Left(error) => 0
}

In the case we recieve a Right we can know the calculation completed successfully. In the case we get a Left we know at least one of the two look ups failed, and we can fallback just as we had with recover.

Both of these methods are proper ways to handle error recovery with unwrapped and wrapped nested values. There is one scenario that may seem problematic, let’s start by presenting the code. In this case the code will be executing an update for Alice and Bob.

// def setTotalToDB(key: String, value: Int): Future[Unit]
val updateTeamTotal: Future[Unit] = for {
  _ <- setTotalToDB("alice", 1000)
  _ <- setTotalToDB("bob", 500)
} yield ()

This code is pretty innocous, and any exception will bubble up through the Future just fine. Since the result of the function is Unit we really don’t care to store the result, and so we’ve throw it away by not binding it to a name with _.

If we decided that all set functions should get an updated signature with a nested type similar to before like:

def setTotalToDB(key: String, value: Int): Future[Either[Throwable, Unit]]

Even with this updated function signature, our original for-comprehension continues to compile without modification.

And this is the insidious bit. As code gets refactored, effectful function typicaly not bound to a name can cause errors and exceptions to be swallowed.

After updating the function setTotalToDB to Future[Either[Throwable, Unit]] we know that the function will capture some subset of exceptions and place them in a Left. However in the for comprehension we are disregarding the value inside the Future by not binding it to a name or inspecting it further. This means throwing away both Right which contains an uninteresting value, and Left which contains our failure.

The solution is to update our for-comprehension to inspect the return value of setTotalToDB:

val updateTeamTotal: Future[Either[Throwable, Unit]] = for {
  aliceSetResult <- setTotalToDB("alice", 1000)
  bobSetResult <- setTotalToDB("bob", 500)
} yield aliceSetResult.flatMap(_ => bobSetResult) 

Now the possiblity of failure is expressed and accounted for in the type of updateTeamTotal. We are now forced to handle the possibility of error in an update.

Conclusion

We’ve seen how using nesting monads can lead to some errors especially in code that is updated from one type signature to another. One way to defend against these types of operator errors is to ensure unit tests are also checking error pathways. Another solution is to use monad-transformers which aid in threading the inner-monad through for comprehension, reducing the need for manually composing the inner monad.

Additional Reading

• Monad Transformer
• Cats: EitherT