Content tagged scala

After working with Scala for a while now, I thought it would be good to write down a couple of notes on my current testing setup, in particular with regards to which libraries I've settled on and which style of testing I've ended up using.

Tests end up in the same package as the code that's tested. A group of tests are always in a class with the Tests suffix, e.g. FooTests. If it's about a particular class Foo the same applies.

scalatest is used as the testing framework, using AnyWordSpec, that means we're using the should / in pattern.

For mocking the only addition is MockitoSugar to make things more Scala-ish.

How does it look like?

package com.example.foo

import org.mockito.MockitoSugar
import org.scalatest.wordspec.AnyWordSpec

class FooTests extends AnyWordSpec with MockitoSugar {
  "Foo" should {
    "do something" in {
      val bar = mock[Bar]
      val foo = new Foo(bar)
      foo.baz(42L)
      verify(bar).qux(42L)
    }
  }
}

Easy enough. There's also some more syntactic sugar for other Mockito features, meaning ArgumentMatchersSugar should also be imported when needed. Same as scalatest has a number of additional helpers for particular types like Option or Either, e.g. OptionValues and EitherValues.

class BarTests extends AnyWordSpec with Matchers with EitherValues with OptionValues {
  "Bar" should {
    "do something else" in {
      val bar = new Bar
      bar.qux(42L).left.value should be(empty)
      bar.quux().value shouldBe "a value"
    }
  }
}

This can be done to the extreme, but usually it looks easier to me to simply assign highly nested values to a variable and continue with matchers on that variable instead.

Since sbt is often used, the two test dependencies would look like this:

libraryDependencies ++= Seq(
  "org.scalatest" %% "scalatest"               % "3.1.1"  % Test,
  "org.mockito"   %% "mockito-scala-scalatest" % "1.13.0" % Test,
)

Did you know Scala has macros? Coming from Common Lisp they serve pretty much the same purpose, doing things that the (plethora of) other language features don't support and to shortcut the resulting boilerplate code. And even the S-expressions can be had when macro debugging is turned on, though the pretty-printed Scala code is arguably much more useful here.

Why would you realistically use them then? Turns out I had to deal with some auto-generated code dealing with Protobuf messages. The generated classes for any message look something like this (in Java syntax since that's what the generated code is):

public interface ExampleResponseOrBuilder
  extends com.google.protobuf.MessageOrBuilder;

public static final class ExampleResponse
  extends com.google.protobuf.GeneratedMessageV3
  implements ExampleReponseOrBuilder {

  public static Builder newBuilder();

  public static final class Builder
    extends com.google.protobuf.GeneratedMessageV3.Builder<Builder>
    implements ExampleResponseOrBuilder;
}

That is, we have one interface, two classes, one of them conveniently gives you a builder for new objects of the class. That's used like this (back to Scala here):

val builder: ExampleResponse.Builder = Example.newBuilder()
builder.mergeFrom(stream)
val result: ExampleResponse = builder.build()

If you try and make a generic builder here, you'll quickly notice that this is rather hard as the generic types don't really express the relationship between ExampleResponse and ExampleResponse.Builder well.

As an aside, you want to have a generic builder parametrised on the return type to be able to write something like this:

val result = build[ExampleResponse](stream)

Without ever having to pass through the type as a value. Better even if you just specify the result type and the type parameter for build is then automatically derived.

These builders look something like this then:

trait ProtobufBuilder[T <: Message] {
  def underlying(): Message.Builder

  def build(string: String)(implicit parser: JsonFormat.Parser): T = {
    val builder = underlying()
    parser.merge(string, builder)
    builder.build().asInstanceOf[T]
  }
}

class ExampleResponseBuilder() extends ProtobufBuilder[ExampleResponse] {
  override def underlying(): ExampleResponse.Builder =
    ExampleResponse.newBuilder()
}

This then allows us to use some implicit magic to pass these through to the decoder (sttp's framework in this case) to correctly decode the incoming data.

But, we've to 1. write one class for each type, 2. instantiate it. This is roughly five lines of code per type depending on the formatting.

Macros to the rescue!

Inspired by the circe derivation API I finally got all the pieces together to create such a macro:

def deriveProtobufBuilder[T <: Message]: ProtobufBuilder[T] = macro deriveProtobufBuilder_impl[T]

def deriveProtobufBuilder_impl[T <: Message: c.WeakTypeTag](
    c: blackbox.Context): c.Expr[ProtobufBuilder[T]] = {
  import c.universe._

  val messageType   = weakTypeOf[T]
  val companionType = messageType.typeSymbol.companion

  c.Expr[ProtobufBuilder[T]](q"""
    new ProtobufBuilder[$messageType] {
      override def underlying(): $companionType.Builder = $companionType.newBuilder()
    }
   """)
}

Used then like this:

private implicit val exampleResponseBuilder: ProtobufBuilder[ExampleResponse] = deriveProtobufBuilder

That's one or two lines and the types are only mentioned once (the variable name can be changed). Unfortunately getting rid of the variable name doesn't seem to be possible.

Easy, wasn't it? Unfortunately all of this is hampered by the rather undocumented APIs, you really have to search for existing code or Stackoverflow questions to figure this out.

One thing that helped immensly was the -Ymacro-debug-lite option, which prints the expanded macro when used in sbt via compile.