Looking at IntelliJ - not the best start...

In a recent blog post, I wrote that IntelliJ IDEA is a popular choice for Scala developers. I wanted to try for myself, but had no luck - just as many others. While there are people who successfully code in Scala using IDEA 12, the most recent version 14 seems to have major problems in this regard. The bug I encountered was this one - unfortunately a show stopper; I can't even properly add the Scala SDK to IntelliJ. Others, who probably upgraded their install from a previous version of IDEA, seem to come a little further, and then run into other problems, like Gradle integration - which I want as well.

So, no trying IntelliJ with Scala for now. I still want to give it a try, however. With Laterna being Scala-only now, it will stay with Eclipse for a while though.

I think I'll try it with Android, as the official IDE for Android was recently migrated from Eclipse to IntelliJ. So it can't be just bad ;)

Replacement effects with partial functions

This is a very crude mockup of how replacement effects could look in Laterna Magica. Instead of any impressive effect, I'm just modifying a variable that represents a player's life total, but it does quite a lot for only being about 30 lines of code:

//a replaceable event
trait Event { def execute(): Unit }
//an effect that replaces an event
type ReplacementEffect = PartialFunction[Event, Event]

//applies all effects to the event. If a replacement doesn't match, the
//event remains unchanged
def replace(event: Event, effects: ReplacementEffect*) =
  effects.foldLeft(event) { case (event, effect) => effect.applyOrElse(event, identity[Event] _) }

var activeEffects: Seq[ReplacementEffect] = Nil
def execute(event: Event) = replace(event, activeEffects: _*).execute()

//example: A player's life points
var _life = 20
def life = _life

def gainLife(life: Int) = execute(GainLife(life))

case class GainLife(life: Int) extends Event {
  def execute() = _life += life
}

//execution:

//gain three life
gainLife(3)
println(life) //23

//gain twice as much life
activeEffects = List({ case GainLife(x) => GainLife(x * 2) })

//gain six life
gainLife(3)
println(life) //29

The trait Event is used to represent replaceable events, and effects are simply "partial functions": functions that can only be applied to some of the values that its parameter type would allow. For example, at the bottom there's the partial function:

{ case GainLife(x) => GainLife(x * 2) }

While ReplacementEffect is defined for all Events, this one only accepts GainLife instances. Using PartialFunction.applyOrElse, I handle the cases where a replacement effect does not apply to an event.

The example is self-contained, so you should be able to run it yourself. In any case, the output is next to the print statements: The first time, you only get 3 life, but after registering the effect, the amount is indeed doubled. In reality, execute (and replace?) needs to be a bit smarter to recognize the right effects to apply, but otherwise this could stay almost as it is.

Scala revisited: so much shorter!

I didn't even realize that I wrote about Scala before, until I logged into my blog and saw that it's apparently one of my most popular posts. My earlier flirt with Scala ebbed out rather quickly, and I think the reasons were twofold.

Firstly, back in 2010, IDE integration for Scala wasn't that great. While there's still much room for improvement, the situation is much better now. I only tried the Eclipse Scala IDE, and from what I heard, IntelliJ integration is nice as well.

The second reason is simply my lack of understanding of functional programming. I would say that I had a good concept of how functional programming is supposed to work back then, but I was surely lacking practice. My interest recently led me to a FP/Haskell class at university, and while it was only introductory level, I think it helped a lot.

As far as I know, the language itself has evolved quite a bit as well. There was a big cleanup of the collection API, and much more - it has been five years, after all. How time flies!

Anyway, I'm in love with that language! In the last three weeks, I converted Harmonic (and other projects) from Java to Scala - and from Maven to Gradle, by the way - and got from 1027 to 870 lines of code, about 15% less. Granted, that's not exclusively the language switch but also a partial redesign - hence the three weeks of work - but I don't think that lessens the message. It gets more impressive when I add that the Scala code includes about 300 lines of freshly written unit tests!

Scala gets rid of a lot of boilerplate code that bloats the line count of Java. For example, a typical property takes 7 lines of Java:

private int property;
public void setProperty(int property) {
    this.property = property;
}
public int getProperty() {
    return this.property;
}

Converted to Scala idioms, the code would look like this:

private var _property: Int = _
def property = _property
def property_=(property: Int) = _property = property

Well, that is a little shorter. Also note how def property omits the parentheses, and that methods ending in _= are called as if they were assignments. Now compare incrementing that property:

//Java
x.setProperty(x.getProperty() + 1);
//Scala
x.property = x.property + 1


That's just one example, I could go on: Instead of static members, Scala has objects, but they are actually singletons and can extend other classes and traits. Traits are like interfaces, can have method implementations like in Java 8, but are actually inherited as mixins. Case classes allow pattern matching, comparing that to switch is almost insulting given how much more flexible pattern matching is. Implicits allow the compiler to do even more of the work for you: implicit conversions are like user-defined autoboxing (another insult!), implicit parameters save you typing when a context variable is needed over and over in method calls. Need to pass a Game to every other method? Make it implicit!

I bet there are many more gems hidden in Scala, but I only had three weeks to look so far :P

Okay, one more thing. I won't even describe it, just show you code. This is how unit tests can look in Scala:

class EngineSpec extends FlatSpec with Matchers with GivenWhenThen {
  behavior of "An engine"

  it should "handle branches properly" in {
    Given("an engine")
    implicit val engine = new Engine()

    And("a PolybufIO for a custom action")
    engine.addIO(MyAction)

    {
      When("creating a branch at HEAD")
      val head = engine.head
      val branch = engine.Branches.createBranchHere("branch1")

      Then("the branch's tip should be the head")
      branch.tip should be(head)

      When("executing an action")
      engine.execute(new MyAction())

      Then("the branch's tip should still be the old head")
      branch.tip should be(head)
    }

    {
      When("creating another branch at HEAD")
      val head = engine.head
      val branch = engine.Branches.createBranchHere("branch2")

      Then("the branch's tip should be the head")
      branch.tip should be(head)

      When("making that branch current")
      engine.currentBranch = branch

      And("executing an action")
      engine.execute(new MyAction())

      Then("the branch's tip should be the new head")
      branch.tip should be(engine.head)

      When("moving that branch's tip")
      branch.tip = head

      Then("the new head should be the branch's tip")
      engine.head should be(branch.tip)
    }
  }
}

Scala: Functional programming & DSL

I first learned about Scala in forge's comment to this post, and the tutorial he mentioned is really great. Just for experimenting, I extended the Calculator example for Variables, Functions and exponentiation. The result is really extensible and took me just 140 lines of code, which really impressed me. So I'm showing you the result:

AST.scala (Scala doesn't require Files to represent the class names, and AST (abstract syntax tree) fits the classes in this file)
package net.slightlymagic.calc {
  abstract class Expr
  case class Variable(name: String) extends Expr {
    override def toString = name
  }
  case class Number(value: Double) extends Expr {
    override def toString = "" + value
  }
  case class UnaryOp(name: String, arg: Expr) extends Expr {
    override def toString = name + arg
  }
  case class BinaryOp(name: String, left: Expr, right: Expr) extends Expr {
    override def toString = "(" + left + name + right + ")"
  }
  case class Function(name: String, args: List[Expr]) extends Expr {
    override def toString = name + args.mkString("(", ", ", ")")
  }
}

Calc.scala

package net.slightlymagic.calc {
  object Calc {
    def simplify(e: Expr): Expr = {
      def simpArgs(e: Expr) = e match {
        case BinaryOp(op, left, right) => BinaryOp(op, simplify(left), simplify(right))
        case UnaryOp(op, operand) => UnaryOp(op, simplify(operand))
        case Function(op, operands) => Function(op, operands.map((x) => simplify(x)))
        case x => x
      }
     
      def simpTop(e: Expr) = e match {
        case UnaryOp("-", UnaryOp("-", x)) => x
        case BinaryOp("-", x, Number(0)) => x
        case BinaryOp("-", Number(0), x) => UnaryOp("-", x)
        case BinaryOp("-", x1, x2) if x1 == x2 => Number(0)
       
        case UnaryOp("+", x) => x
        case BinaryOp("+", x, Number(0)) => x
        case BinaryOp("+", Number(0), x) => x
       
        case BinaryOp("*", x, Number(1)) => x
        case BinaryOp("*", Number(1), x) => x
       
        case BinaryOp("*", x, Number(0)) => Number(0)
        case BinaryOp("*", Number(0), x) => Number(0)
       
        case BinaryOp("/", x, Number(1)) => x
        case BinaryOp("/", x1, x2) if x1 == x2 => Number(1)
       
        case BinaryOp("^", x, Number(1)) => x
        case BinaryOp("^", x, Number(0)) if x != Number(0) => Number(1)
        case BinaryOp("^", x1, UnaryOp("-", x2)) => BinaryOp("/", Number(1), BinaryOp("^", x1, x2))
        case e => e
      }
     
      simpTop(simpArgs(e))
    }
   
    def evaluate(expr: Expr, variables: Map[String, Double], functions: Map[String, (List[Double]) => Double]): Double = {
      expr match {
        case Number(x) => x
        case UnaryOp("-", x) => -evaluate(x, variables, functions)
        case BinaryOp("+", x1, x2) => evaluate(x1, variables, functions) + evaluate(x2, variables, functions)
        case BinaryOp("-", x1, x2) => evaluate(x1, variables, functions) - evaluate(x2, variables, functions)
        case BinaryOp("*", x1, x2) => evaluate(x1, variables, functions) * evaluate(x2, variables, functions)
        case BinaryOp("/", x1, x2) => evaluate(x1, variables, functions) / evaluate(x2, variables, functions)
        case BinaryOp("^", x1, x2) => Math.pow(evaluate(x1, variables, functions), evaluate(x2, variables, functions))
        case Variable(x) => variables(x)
        case Function(x, args) => functions(x)(args.map((x) => evaluate(x, variables, functions)))
      }
    }
   
    def parse(text : String) = CalcParser.parse(text).get
   
    val standardVars =
      Map(
        "E"  -> Math.E,
        "Pi" -> Math.Pi
      )
   
    val standardFuns =
      Map(
        "sin"  -> {x:List[Double] => Math.sin(x(0))},
        "cos"  -> {x:List[Double] => Math.cos(x(0))},
        "tan"  -> {x:List[Double] => Math.tan(x(0))},
        "cot"  -> {x:List[Double] => 1/Math.tan(x(0))},
        "asin" -> {x:List[Double] => Math.asin(x(0))},
        "acos" -> {x:List[Double] => Math.acos(x(0))},
        "atan" -> {x:List[Double] => Math.atan(x(0))},
        "acot" -> {x:List[Double] => Math.atan(1/x(0))},
        "exp"  -> {x:List[Double] => Math.exp(x(0))},
        "log"  -> {x:List[Double] => Math.log(x(0))},
        "min"  -> {x:List[Double] => Math.min(x(0), x(1))},
        "max"  -> {x:List[Double] => Math.max(x(0), x(1))}
      )
   
    def evaluate(expr: String, variables: Map[String, Double], functions: Map[String, (List[Double]) => Double]): Double = {
      evaluate(
        simplify(parse(expr)),
        if(variables == null) standardVars else standardVars ++ variables,
        if(functions == null) standardFuns else standardFuns ++ functions
      )
    }
   
   
    import scala.util.parsing.combinator._
   
    object CalcParser extends JavaTokenParsers {
     
      def expr: Parser[Expr] =
        (term ~ "+" ~ term) ^^ { case lhs~plus~rhs => BinaryOp("+", lhs, rhs) } |
        (term ~ "-" ~ term) ^^ { case lhs~minus~rhs => BinaryOp("-", lhs, rhs) } |
        term
 
      def term: Parser[Expr] =
        (factor ~ "*" ~ factor) ^^ { case lhs~times~rhs => BinaryOp("*", lhs, rhs) } |
        (factor ~ "/" ~ factor) ^^ { case lhs~div~rhs => BinaryOp("/", lhs, rhs) } |
        factor
 
      def factor : Parser[Expr] =
        (exp ~ "^" ~ exp) ^^ { case lhs~exp~rhs => BinaryOp("^", lhs, rhs) } |
        exp
 
      def exp : Parser[Expr] =
        ("+" ~ expr) ^^ { case plus~rhs => UnaryOp("+", rhs) } |
        ("-" ~ expr) ^^ { case minus~rhs => UnaryOp("-", rhs) } |
        "(" ~> expr <~ ")" |
        ident ~ params ^^ {case name~params => Function(name, params)} |
        ident ^^ {case name => Variable(name)} |
        floatingPointNumber ^^ {x => Number(x.toFloat) }
     
      def params : Parser[List[Expr]] =
        "(" ~> repsep(expr, ",") <~ ")"
     
      def parse(text : String) = parseAll(expr, text)
    }
  }
}

Wow! I don't want to repeat the tutorial, which is really great, but I want to mention a few features that really improved the way this code is written.

First of all, Scala is a mixture of functional and object oriented: Map[String, (List[Double]) => Double] (Scala uses square brackets for generic types) means a Map of Strings and Functions, which take a list of Doubles as a parameter and yields a Double as the result. To get the same result as standardFuns in the Calc object... Yes, object. This basically means that all members are static. On the opposite, normal classes can't have static members, which doesn't really matter because you can have class and object of the same name. To get the same result as standardFuns using Java, you'd have to define a common interface and several anonymous classes, which means a lot of duplication. Using anonymous functions and the fact that a function can be passed as a parameter, something you don't get from nonfuntional languages, you avoid most of the redundant code. No interface definition, just a single line per anonymous function.

The second thing that you can write method calls in an "unusual way": a.plus(b) is the same as a plus b. Together with the fact that you can name methods, e.g. "+", you basically have overriding operators. This means not much more than you can write short, concise code. But that is very important; just look at the preceding paragraph. The parser package of Scala uses such "Operators" to write short, expressive text parsing code.

Lastly, pattern matching. This is a great feature that is really hard to imitate using Java. It uses case classes, as seen in AST.scala, and the match/case construct. For example:

expr match {

  case BinaryOp(x1, op, x2) => doSomething()

  case BinaryOp(Number(x1), "+", Number(x2) => doSomethingElse()

}

doSomething() is only executed if the case matches, and at the same time assigns the three variables x1, op, x2. You can build more complex constructs, as the second example shows, including nesting Objects and using constant expressions to narrow down matching cases. This works as long as you use case classes.

And now comes another important part: Scala compiles to Java byte code, which means that you can code a part in scala. For example, the simple Calculator DSL was way easier to write in Scala than it would be in Java. Once it's packed as a jar, you don't even notice that the code was not written in traditional Java, and integrates nicely into the other parts. (Except you'd have a hard time to call "operator methods" or use API that uses Functions as parameters. Consider that for your public, plain-old-java accessible API)
The only downside is that Eclipse integration is not great so far. Maven does a good task in still compiling your Scala code, but the Scala plugin is not very stable, so I use the plain text editor. It works for small things like the Calculator, but I miss obvious features like import organizing, content assist, automatic building on every change, syntax & error highlighting, parentheses matching, folding...