Interoperability

Looking back at my earlier posts, I'm surprised how utterly Java-centric everything I did was. Never mind things like bytecode instrumentation, of course that's JVM specific, and it's cool stuff!

But for other things, I think I would try to achieve more interoperability these days. Specifically, I'm looking at JGroups. It has some nice features in terms of consensus, making sure that all known nodes have the same view of the system, but looking at it purely as a networking library, it shockingly limits you to implement all your nodes in a JVM language.

I think there are generally three layers of interoperability. The JGroups example is one extreme, basically no interoperability. It has one Java implementation and its network protocols are specific to that implementation. I don't know the specifics of JGroups, but the protocol used may well be tailored to transmit very specific manipulations of JGroups nodes' states. In other word, there are lots of assumptions that make JGroups easy to implement in Java, but very, very hard to port to another language.

One kind of interoperability comes through dynamic linking. For example, through native methods, Java can call code written in C or another language that is compiled into machine code. As long as the JVM uses the ABI and calling conventions of the native library, access is no problem. The native code is executed in the calling process, so this kind of interoperability is strictly for libraries, not for client/server communication.

So if it was CGroups instead - written in C - then there could simply be a Java binding and a .Net binding etc., and all platforms using these bindings could cooperate. Unfortunately the reverse isn't true: the JVM is, quite obviously, a virtual machine, and you can't simply call code targeted at a VM. That VM first has to run, and then you have to interface with the VM before finally getting access to the library. This is true for all languages requiring a runtime environment (i.e. interpreted languages), but it gets worse the more complicated the runtime gets.

Which leads me to the third kind of interoperability: wire protocols, which solve the issue of having a protocol that is full of assumptions from the original implementation. Instead, the first step to designing the application or library is to design a binary or text based encoding and to clearly states what kinds of communication are allowed, in what order, at what time, etc. The protocol has to be simple and explicit enough to write down, so it should also be possible to implement in different languages. That doesn't allow you to use one library in different contexts, but it makes it feasible to port the library for different platforms.

If it weren't for that last kind of interoperability, the Internet would be a very different place. Imagine if websites used a browser-specific protocol instead of HTTP! Microservice architectures also rely on well-defined protocols to connect otherwise incompatible systems: of course, a service written in Java can expose some JGroups-derived functionality via, say, HTTP, and then any other application can access that service.

Blogging on a Time Budget

I've never been regular in writing blog posts, and in the past years I never found the time necessary to do serious Magic programming. I enjoy thinking about general ways to model the Magic rules in a program, but without a serious amount of time, it will always remain tinkering. It's an interest, but not serious enough to resolve to put out thousands of lines of code to make it work.

However, I'm still a passionate programmer and learner. While most of my posts so far have had a Magic context, they were always about programming. And this is where I want to shift the focus: non-MtG posts aren't off-topic, and that should encourage me to write more about what I'm currently doing. I know I won't start spewing out post after post, but as long as I find the time, I shouldn't run out of topics.

So what have I learned about in the three years since my last blog post?

I have shortly continued with Scala, but got demotivated by the compile times. A lot of time was spent working with Python, and it's currently my most fluent language. I have written web applications with Django, learned about generators and async/await, and constructed protocols and servers using ZeroMQ.

There has also been a considerable amount of JavaScript and some TypeScript, mostly using Vue.js to create reactive frontends for my web applications. I have also looked at web-technologies, such as Webpack and, although briefly, Websockets, PWAs (particularly service workers) and WebAssembly.

Recently, I re-discovered Rust after playing with it years ago. In terms of languages-to-use-if-you-want-to-get-things-done-in-a-reasonable-amount-of-time, Rust is about as far away from Python and JS as you can get (yes, there's also C and C++, but I don't think they're farther away, just slightly worse at getting things done... just my opinion though).

I intend to write about most of these things, if they're still relevant when I get around to them. If there be still readers here, let me know with what to start.

Oh, and I have watched tons of programming videos on YouTube! There's lots of great content. But there are also videos that suffer from what I call "boring tutorial syndrome": they try to teach good programming style (or something), but actually don't go far beyond how to write an idiomatic Hello World program (or whatever the basics of the respective topic are). The problem is that by starting from scratch, these tutorials don't manage to reach a reasonable depth. So whatever I write, don't expect start-at-zero tutorials, expect food for thought and self-study!

PS: if you're looking for videos that don't suffer from the boring-tutorial-syndrome, try LiveOverflow

Looking at IntelliJ - not the best start...

In a recent blog post, I wrote that IntelliJ IDEA is a popular choice for Scala developers. I wanted to try for myself, but had no luck - just as many others. While there are people who successfully code in Scala using IDEA 12, the most recent version 14 seems to have major problems in this regard. The bug I encountered was this one - unfortunately a show stopper; I can't even properly add the Scala SDK to IntelliJ. Others, who probably upgraded their install from a previous version of IDEA, seem to come a little further, and then run into other problems, like Gradle integration - which I want as well.

So, no trying IntelliJ with Scala for now. I still want to give it a try, however. With Laterna being Scala-only now, it will stay with Eclipse for a while though.

I think I'll try it with Android, as the official IDE for Android was recently migrated from Eclipse to IntelliJ. So it can't be just bad ;)

What is a card?

This post is about ambiguity, but this post is not about Ambiguity. (Don't forget to look at the card topsy turvy as well. And don't forget to look at the card Topsy Turvy as well)

Okay, enough with the puns... at least if I don't find any others while writing... what I really want to write about is how overloaded the term "Card" is in Magic. Let's see how many different meanings I can come up with.

Cards are those things that Wizards designs. For example, Grizzly Bears is a card, and Llanowar Elves is as well.

What about two Grizzly Bears? A card can also mean the physical object, sold in boosters and put in decks. But you can't even simply say that the physical card is "a Grizzly Bears", because there's a few different ones. And there's foil as well, and probably other things I'm forgetting. So the physical card is actually not an instance of what Wizards R&D designed, but of one translation of one printing of such a card, either in foil or not.

Getting to the actual game, cards are a kind of object, in contrast with tokens and copies of cards, which are not cards but serve almost the same purpose.

Noncard objects exist in a few different forms: emblems are always in the command zone; tokens can only exist on the battlefield; ability objects exist on the stack; copies of spells exist on the stack; casting a copy of a card creates that copy in the original card's zone, which in then put onto the stack as a spell while casting.

Permanents are objects that can be either cards or tokens, so another thing a card can be. Compared to other objects, permanents have state:  tapped/untapped, flipped/unflipped, face up/face down, and phased in/phased out.

The comprehensive rules on spells are worded a little strangely: "A spell is a card on the stack", "A copy of a spell is also a spell", "if the player does [cast a copy of a card], that copy is a spell as well". It sounds contradictory at first, but it's manageable.

So what have we got here:

• What Oracle says a card is, identified by the English name of the card.
• A printing of such a card, identified by English name together with an expansion. This extends the Oracle card by rarity, artist, illustration, collector number, card frame style, guild symbols and similar in the text box, ...
• A translation of a printing, identified by a multiverse ID, or alternately by English name, expansion and language; I wouldn't trust that the translated name is unique. The translation adds obviously anything language specific about a card. This includes printed, non-oracle wording including reminder text, as well as flavor text.
• A piece of cardboard that has a Magic card printed on it. To my understanding, this interpretation of "card" can be uniquely identified by a function defined on an interval of the timeline that maps every instant in that interval to the volume of space that is occupied by the card at that time. Or, a card is a card.
• A digital representation of such can also be considered a card in that sense.
• A card is an object that can be in a zone in a game of Magic.
• Some permanents are (represented by) cards.
• Some spells are (represented by) cards.

You can imagine that all this ambiguity makes it quite hard to come up with a proper software model for a game of Magic!

Ordering of replacement effects

Handling of multiple replacement effects is intricate, but well described in the comprehensive rules, so I'll link you to my favorite online version: rule 616

The code I presented in monday doesn't handle any of that, but it wouldn't be too hard to do. In my quick mockup, effects are stored in a list and applied in that order. For example:

Boon reflection: If you would gain life, you gain twice that much life instead.
Nefarious Lich: If you would gain life, draw that many cards instead. (and other abilities)

Clearly a life gain could be replaced by either, but only one of the resulting events is a life gain in turn. So, if I gained two life, depending on the order, I'd draw either two or four cards, and it should be my choice.

So how would that be written? Almost exactly like the comprehensive rules are formulated: Starting with the set of all replacement effects,

1. filter the set for those applicable to the event
2. if the set is empty, we're done
3. filter further
• for self-replacement effects
• if the result is empty, for "under whose control an object would enter the battlefield" replacement effects instead
• if the result is empty, for "cause an object to become a copy of another object as it enters the battlefield" replacement effects instead
• if the result is empty, don't filter at all
4. if there's more than one effect, let the applicable player choose one effect
5. replace the event and remove the effect from the unfiltered set
6. repeat

a partial function already has isDefinedAt, and filtering sets is easy in scala, e.g. effects.filter { _.isDefinedAt(event) }. I can even use groupBy to partition the effects into their four types in one line. What needs thought is how to ask the players for ordering effects.

 Edit: here is my new vesion for replace:

//applies all effects to the event.
@tailrec
def replace(event: Event, effects: Set[ReplacementEffect]): Event = {
  val filtered = effects.filter { _.isDefinedAt(event) }
  //TODO here I need to know what type an effect is
  //0... self-replacement, 1... control, 2... copy, 3... regular
  val byType = filtered.groupBy { effect => 3 }
  val sorted = byType.toSeq.sortBy { case (k, _) => k }
  val choices = sorted.collectFirst { case (_, v) if !v.isEmpty => v }
  choices match {
    case None => event
    case Some(effects) =>
      val effect =
        if (effects.size == 1) effects.head
        else {
          //TODO let player choose one effect
          effects.head
        }
      //only applicable effects here, so use apply directly
      replace(effect.apply(event), effects - effect)
  }
}

The only change to my description is that I don't do step 2, but instead do it in step 4. The collectFirst call returns the first nonempty collection, or None if they're all empty, so I get that check here for free anyway. Also note that I don't use a loop, but instead use recursion, which is preferred in functional programming. Some types of recursion, like this, can be optimized into a loop, so here it's just a matter of style. In fact, by adding the @tailrec annotation, the compiler would issue an error if this couldn't be optimized into a loop.

Replacement effects with partial functions

This is a very crude mockup of how replacement effects could look in Laterna Magica. Instead of any impressive effect, I'm just modifying a variable that represents a player's life total, but it does quite a lot for only being about 30 lines of code:

//a replaceable event
trait Event { def execute(): Unit }
//an effect that replaces an event
type ReplacementEffect = PartialFunction[Event, Event]

//applies all effects to the event. If a replacement doesn't match, the
//event remains unchanged
def replace(event: Event, effects: ReplacementEffect*) =
  effects.foldLeft(event) { case (event, effect) => effect.applyOrElse(event, identity[Event] _) }

var activeEffects: Seq[ReplacementEffect] = Nil
def execute(event: Event) = replace(event, activeEffects: _*).execute()

//example: A player's life points
var _life = 20
def life = _life

def gainLife(life: Int) = execute(GainLife(life))

case class GainLife(life: Int) extends Event {
  def execute() = _life += life
}

//execution:

//gain three life
gainLife(3)
println(life) //23

//gain twice as much life
activeEffects = List({ case GainLife(x) => GainLife(x * 2) })

//gain six life
gainLife(3)
println(life) //29

The trait Event is used to represent replaceable events, and effects are simply "partial functions": functions that can only be applied to some of the values that its parameter type would allow. For example, at the bottom there's the partial function:

{ case GainLife(x) => GainLife(x * 2) }

While ReplacementEffect is defined for all Events, this one only accepts GainLife instances. Using PartialFunction.applyOrElse, I handle the cases where a replacement effect does not apply to an event.

The example is self-contained, so you should be able to run it yourself. In any case, the output is next to the print statements: The first time, you only get 3 life, but after registering the effect, the amount is indeed doubled. In reality, execute (and replace?) needs to be a bit smarter to recognize the right effects to apply, but otherwise this could stay almost as it is.

Scala revisited: so much shorter!

I didn't even realize that I wrote about Scala before, until I logged into my blog and saw that it's apparently one of my most popular posts. My earlier flirt with Scala ebbed out rather quickly, and I think the reasons were twofold.

Firstly, back in 2010, IDE integration for Scala wasn't that great. While there's still much room for improvement, the situation is much better now. I only tried the Eclipse Scala IDE, and from what I heard, IntelliJ integration is nice as well.

The second reason is simply my lack of understanding of functional programming. I would say that I had a good concept of how functional programming is supposed to work back then, but I was surely lacking practice. My interest recently led me to a FP/Haskell class at university, and while it was only introductory level, I think it helped a lot.

As far as I know, the language itself has evolved quite a bit as well. There was a big cleanup of the collection API, and much more - it has been five years, after all. How time flies!

Anyway, I'm in love with that language! In the last three weeks, I converted Harmonic (and other projects) from Java to Scala - and from Maven to Gradle, by the way - and got from 1027 to 870 lines of code, about 15% less. Granted, that's not exclusively the language switch but also a partial redesign - hence the three weeks of work - but I don't think that lessens the message. It gets more impressive when I add that the Scala code includes about 300 lines of freshly written unit tests!

Scala gets rid of a lot of boilerplate code that bloats the line count of Java. For example, a typical property takes 7 lines of Java:

private int property;
public void setProperty(int property) {
    this.property = property;
}
public int getProperty() {
    return this.property;
}

Converted to Scala idioms, the code would look like this:

private var _property: Int = _
def property = _property
def property_=(property: Int) = _property = property

Well, that is a little shorter. Also note how def property omits the parentheses, and that methods ending in _= are called as if they were assignments. Now compare incrementing that property:

//Java
x.setProperty(x.getProperty() + 1);
//Scala
x.property = x.property + 1


That's just one example, I could go on: Instead of static members, Scala has objects, but they are actually singletons and can extend other classes and traits. Traits are like interfaces, can have method implementations like in Java 8, but are actually inherited as mixins. Case classes allow pattern matching, comparing that to switch is almost insulting given how much more flexible pattern matching is. Implicits allow the compiler to do even more of the work for you: implicit conversions are like user-defined autoboxing (another insult!), implicit parameters save you typing when a context variable is needed over and over in method calls. Need to pass a Game to every other method? Make it implicit!

I bet there are many more gems hidden in Scala, but I only had three weeks to look so far :P

Okay, one more thing. I won't even describe it, just show you code. This is how unit tests can look in Scala:

class EngineSpec extends FlatSpec with Matchers with GivenWhenThen {
  behavior of "An engine"

  it should "handle branches properly" in {
    Given("an engine")
    implicit val engine = new Engine()

    And("a PolybufIO for a custom action")
    engine.addIO(MyAction)

    {
      When("creating a branch at HEAD")
      val head = engine.head
      val branch = engine.Branches.createBranchHere("branch1")

      Then("the branch's tip should be the head")
      branch.tip should be(head)

      When("executing an action")
      engine.execute(new MyAction())

      Then("the branch's tip should still be the old head")
      branch.tip should be(head)
    }

    {
      When("creating another branch at HEAD")
      val head = engine.head
      val branch = engine.Branches.createBranchHere("branch2")

      Then("the branch's tip should be the head")
      branch.tip should be(head)

      When("making that branch current")
      engine.currentBranch = branch

      And("executing an action")
      engine.execute(new MyAction())

      Then("the branch's tip should be the new head")
      branch.tip should be(engine.head)

      When("moving that branch's tip")
      branch.tip = head

      Then("the new head should be the branch's tip")
      engine.head should be(branch.tip)
    }
  }
}