<h1>Keyboard layout</h1>
10th January 2015
Here is one very quick suggestion for all fellow Finnish programmers:
change your keyboard layout. The default keyboard layout for Finnish
is absolutely terrible for programming.
Adjusting your keyboard layout is not hard, and it will make a
difference in your day-to-day programming activities, especially
if you're using Apple's computers, where the braces are behind two
modifier keys. No amount of apologism will make up for that.
Customizing your tools to fit you better is something we all should
constantly strive for, but even most professional programmers I've
dealt with use the system default keyboard layout. Most of the others
have changed to the US layout completely, but that's a little extreme
for me: I don't like the idea of putting ä and ö behind modifiers.
What I do envy is the placement of brackets and braces, conveniently
on the side. Standard Finnish keyboard layout maps å and diaeresis
¨ to those valuable keys!
I suggest a hybrid between the Finnish and US keyboard layouts, with
Finnish keyboard layout acting as the basis for these modifications:
<ul>
<li>å → [</li>
<li>Å → {</li>
<li>¨ → ]</li>
<li>^ → }</li>
<li>AltGr + ^ → ^</li>
<li>AltGr + o → å</li>
<li>AltGr + O → Å</li>
<li>AltGr + u → ü</li>
<li>AltGr + U → Ü</li>
</ul>
While at it, I changed tilde and circumflex from dead keys to live
ones, since practically every time I type those characters I just want
that character.
With these tiny modifications, you get the greatest benefits of using
US keyboard layout without the downsides. I've now been using this for
a few years on all my computers; it took no time at all to get used to
and comfortable with.

Using Qwerty is terrible for typing in general, but learning a whole
new layout like Dvorak or Colemak is a much bigger proposition than
switching a few keys around on the layout you already know.

Let's see how to do it!
<h2>Mac OS X</h2>
If you're happy with what I suggested above, I've already done the
work for you. Just download <a href="https://old.eagleflow.fi/Finnish%20%28programming%29.bundle.zip">this</a>.
Otherwise, download <a href="http://scripts.sil.org/ukelele">Ukelele</a> [sic]
and create a new key map based on your current keyboard layout. Click
on the keys you wish to modify, and after you're done save the new
keyboard layout as a bundle.
Place the bundle in <code>~/Library/Keyboard Layouts/</code> and add a new input
source from Preferences &gt; Keyboard &gt; Input sources.
(On OS X you can't remove the default keyboard layout from your input
sources for some reason.)
<h2>Linux</h2>
This is a bit harder in Linux. I'm running Ubuntu 14.10 on the desktop
for time being, so these instructions will reflect that. The first
file you want to edit is <code>/usr/share/X11/xkb/symbols/&lt;language&gt;.</code> I
appended this to the bottom of the file:
<pre><code>partial alphanumeric_keys
xkb_symbols &quot;programmer&quot; {
 include &quot;fi(fi)&quot;
 name[Group1]=&quot;Finnish (programmer)&quot;;

 key &lt;AD07&gt; { [ u, U, udiaeresis, Udiaeresis ] };
 key &lt;AD09&gt; { [ o, O, aring, Aring ] };
 key &lt;AD11&gt; { [ bracketleft, braceleft, oe, OE ] };
 key &lt;AD12&gt; { [ bracketright, braceright, asciitilde, asciicircum ] };
};
</code></pre>
Then I tested the changes with <code>setxkbmap 'fi(programmer)'</code>. Once happy
with everything, I edited <code>/usr/share/X11/xkb/rules/evdev.xml</code> and
added the following:
<pre><code>&lt;variant&gt;
 &lt;configItem&gt;
 &lt;name&gt;programmer&lt;/name&gt;
 &lt;description&gt;Finnish (programmer)&lt;/description&gt;
 &lt;/configItem&gt;
&lt;/variant&gt;
</code></pre>
After that, I added this one line to <code>/usr/share/X11/xkb/rules/evdev.lst</code>:
<pre><code>programmer fi: Finnish (programmer)
</code></pre>
These changes were necessary to add the Finnish (programmer) option to
the Gnome Keyboard Settings. Select Finnish (programmer) as your input
source and you're done.

Keyboard layout

<h1>Hearthstone Card List</h1>
19 July 2014
Ever since the <a href="http://eu.battle.net/hearthstone/en/">Hearthstone</a>
beta was opened to everyone, I've been playing this latest game from
Blizzard. At first I was pretty skeptic: as an old Magic: The
Gathering fan, I had my doubts about dumbing down the format and
shallower gameplay experience. Now, a few months after the release, I
have come around: Hearthstone's accesibility, pretty graphics and fun
casual arena runs have won me over.
As a budding hobbyist game developer, I was interested in how
Hearthstone was made. It turns out Hearthstone has been created with
<a href="http://unity3d.com/">Unity</a>! There is even an impressive case story
about it <a href="http://unity3d.com/showcase/case-stories/hearthstone">here</a>.
I'm not sure why this surprised me, since studios of all sizes have
been flocking towards Unity in the last few years. This also explains
how they were so fast with the iOS client.
If I were to work on any Hearthstone-related side projects in my spare
time, the first thing I would need is a list of cards and their data.
This list is exactly what we are going to create in this post.
The list could be created manually, but it would take a long time and
errors would inevitably creep in. Even worse, a manually compiled list
would get stale as new expansions and cards are released. In fact, as
I'm writing this the first expansion for the game, Curse of Naxxramas,
is just around the corner. It will contain 30 new cards, so all the
various lists found around the web will require updates in a few days.
It would be a better approach to take a look at the game's data files
and see how the cards are represented there. This will also give us
clues as to what Hearthstone's internal data structures look like.
Let's take a little dive into the installation to see what's there.
Here's the top level folder:
<pre><code>$ du -cks /Applications/Hearthstone/* | sort -rn
1060124 total
525920 /Applications/Hearthstone/Data
348596 /Applications/Hearthstone/base-OSX.MPQ
115676 /Applications/Hearthstone/Hearthstone.app
53268 /Applications/Hearthstone/Updates
9992 /Applications/Hearthstone/Hearthstone Beta Launcher.app
3312 /Applications/Hearthstone/SetupOSX.mpq
2900 /Applications/Hearthstone/Strings
232 /Applications/Hearthstone/DBF
172 /Applications/Hearthstone/Hearthstone.tfil
24 /Applications/Hearthstone/Logs
16 /Applications/Hearthstone/manifest-cards.csv
4 /Applications/Hearthstone/client.config
4 /Applications/Hearthstone/Launcher.db
4 /Applications/Hearthstone/Hearthstone.mfil
4 /Applications/Hearthstone/ConnectLog.txt
0 /Applications/Hearthstone/bnet_client.log
</code></pre>
So what are we looking at here? First of all, the total size of
Hearthstone installation is around one gigabyte. That's surprisingly
small for a modern game! (Of course it is a card game.)
Years ago I wrote <a href="https://github.com/eagleflo/mpyq">mpyq</a> to help me
in my data mining quests, but it's not needed this time around:
Blizzard has opted to simply extract the MPQs straight to disk when
installing the game. <code>base-OSX.MPQ</code> contains most of the other files
inside this directory, including the actual OS X .app bundle. It's
pretty curious that they've opted to keep these archive files around
after extraction.
So what's inside the largest subdirectory, Data/?
<pre><code>$ du -cks /Applications/Hearthstone/Data/OSX/* | sort -rn
525920 total
55528 /Applications/Hearthstone/Data/OSX/cardtextures0.unity3d
51996 /Applications/Hearthstone/Data/OSX/shared1.unity3d
45448 /Applications/Hearthstone/Data/OSX/cardtextures1.unity3d
44232 /Applications/Hearthstone/Data/OSX/movies0.unity3d
43820 /Applications/Hearthstone/Data/OSX/shared0.unity3d
39668 /Applications/Hearthstone/Data/OSX/gameobjects0.unity3d
32280 /Applications/Hearthstone/Data/OSX/spells0.unity3d
32124 /Applications/Hearthstone/Data/OSX/premiummaterials0.unity3d
30992 /Applications/Hearthstone/Data/OSX/uiscreens0.unity3d
30272 /Applications/Hearthstone/Data/OSX/spells1.unity3d
24020 /Applications/Hearthstone/Data/OSX/boards0.unity3d
23600 /Applications/Hearthstone/Data/OSX/actors0.unity3d
18516 /Applications/Hearthstone/Data/OSX/sounds1.unity3d
18312 /Applications/Hearthstone/Data/OSX/sounds0.unity3d
15424 /Applications/Hearthstone/Data/OSX/actors1.unity3d
13284 /Applications/Hearthstone/Data/OSX/textures0.unity3d
3288 /Applications/Hearthstone/Data/OSX/cardbacks0.unity3d
2160 /Applications/Hearthstone/Data/OSX/cardxml0.unity3d
320 /Applications/Hearthstone/Data/OSX/cards1.unity3d
316 /Applications/Hearthstone/Data/OSX/cards2.unity3d
312 /Applications/Hearthstone/Data/OSX/cards0.unity3d
8 /Applications/Hearthstone/Data/OSX/fonts0.unity3d
</code></pre>
Lots of .unity3d files! I'm assuming this is pretty much all of the
game's assets. There is an especially interesting file named
<code>cardxml0.unity3d</code> here. The name alone rewards a closer look at this
file.
I haven't worked with Unity3D files before, but based on this preamble
it seems like these files are in a custom binary format. I'm guessing
the numbers after <code>UnityRaw</code> are version numbers.
<pre><code>$ hexdump -Cn 64 cardxml0.unity3d | cut -c 11-
55 6e 69 74 79 52 61 77 00 00 00 00 03 33 2e 78 |UnityRaw.....3.x|
2e 78 00 34 2e 32 2e 32 66 31 00 00 21 b9 e8 00 |.x.4.2.2f1..!...|
00 00 3c 00 00 00 01 00 00 00 01 00 21 b9 ac 00 |..&lt;.........!...|
21 b9 ac 00 21 b9 e8 00 00 00 1c 00 00 00 00 01 |!...!...........|
</code></pre>
Taking a longer hexdump reveals that the file consists mostly of XML
fragments. I assume that the file begins with Unity3D metadata and
some other information before listing each card in somewhat garbled
XML format. Each card seems to be its own XML document. Here is an
example from the end of the file:
<pre><code>&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;
&lt;Entity version=&quot;2&quot; CardID=&quot;DS1_188&quot;&gt;
 &lt;Tag name=&quot;CardName&quot; enumID=&quot;185&quot; type=&quot;String&quot;&gt;
 &lt;enUS&gt;Gladiator's Longbow&lt;/enUS&gt;
 &lt;frFR&gt;Arc long du gladiateur&lt;/frFR&gt;
 &lt;zhTW&gt;鬥士長弓&lt;/zhTW&gt;
 &lt;zhCN&gt;角斗士的长弓&lt;/zhCN&gt;
 &lt;ruRU&gt;Длинный лук&lt;/ruRU&gt;
 &lt;ptBR&gt;Arco Longo do Gladiador&lt;/ptBR&gt;
 &lt;plPL&gt;Długi Łuk Gladiatora&lt;/plPL&gt;
 &lt;koKR&gt;검투사의 장궁&lt;/koKR&gt;
 &lt;itIT&gt;Arco del Gladiatore&lt;/itIT&gt;
 &lt;esMX&gt;Arco largo de Gladiador&lt;/esMX&gt;
 &lt;esES&gt;Arco largo de Gladiador&lt;/esES&gt;
 &lt;deDE&gt;Langbogen des Gladiators&lt;/deDE&gt;
 &lt;/Tag&gt;
 &lt;Tag name=&quot;CardSet&quot; enumID=&quot;183&quot; type=&quot;CardSet&quot; value=&quot;3&quot; /&gt;
 &lt;Tag name=&quot;CardType&quot; enumID=&quot;202&quot; type=&quot;CardType&quot; value=&quot;7&quot; /&gt;
 &lt;Tag name=&quot;Faction&quot; enumID=&quot;201&quot; type=&quot;Faction&quot; value=&quot;3&quot; /&gt;
 &lt;Tag name=&quot;Class&quot; enumID=&quot;199&quot; type=&quot;Class&quot; value=&quot;3&quot; /&gt;
 &lt;Tag name=&quot;Rarity&quot; enumID=&quot;203&quot; type=&quot;Rarity&quot; value=&quot;4&quot; /&gt;
 &lt;Tag name=&quot;Cost&quot; enumID=&quot;48&quot; type=&quot;Number&quot; value=&quot;7&quot; /&gt;
 &lt;Tag name=&quot;Atk&quot; enumID=&quot;47&quot; type=&quot;Number&quot; value=&quot;5&quot; /&gt;
 &lt;Tag name=&quot;Durability&quot; enumID=&quot;187&quot; type=&quot;Number&quot; value=&quot;2&quot; /&gt;
 &lt;Tag name=&quot;AttackVisualType&quot; enumID=&quot;251&quot; type=&quot;AttackVisualType&quot; value=&quot;8&quot; /&gt;
 &lt;Tag name=&quot;Collectible&quot; enumID=&quot;321&quot; type=&quot;Bool&quot; value=&quot;1&quot; /&gt;
 &lt;Tag name=&quot;CardTextInHand&quot; enumID=&quot;184&quot; type=&quot;String&quot;&gt;
 &lt;enUS&gt;Your hero is &amp;lt;b&amp;gt;Immune&amp;lt;/b&amp;gt; while attacking.&lt;/enUS&gt;
 &lt;frFR&gt;Votre héros est &amp;lt;b&amp;gt;Insensible&amp;lt;/b&amp;gt; quand il attaque.&lt;/frFR&gt;
 &lt;zhTW&gt;你的英雄在攻擊時&amp;lt;b&amp;gt;免疫&amp;lt;/b&amp;gt;&lt;/zhTW&gt;
 &lt;zhCN&gt;你的英雄在攻击时具有&amp;lt;b&amp;gt;免疫&amp;lt;/b&amp;gt;。&lt;/zhCN&gt;
 &lt;ruRU&gt;Атакуя, ваш герой не получает урона.&lt;/ruRU&gt;
 &lt;ptBR&gt;Seu herói fica &amp;lt;b&amp;gt;Imune&amp;lt;/b&amp;gt; enquanto ataca.&lt;/ptBR&gt;
 &lt;plPL&gt;Twój bohater posiada &amp;lt;b&amp;gt;Niewrażliwość&amp;lt;/b&amp;gt; podczas ataku.&lt;/plPL&gt;
 &lt;koKR&gt;내 영웅이 공격할 때 &amp;lt;b&amp;gt;피해 면역&amp;lt;/b&amp;gt; 상태가 됩니다.&lt;/koKR&gt;
 &lt;itIT&gt;Il tuo eroe è &amp;lt;b&amp;gt;Immune&amp;lt;/b&amp;gt; quando attacca.&lt;/itIT&gt;
 &lt;esMX&gt;Tu héroe es &amp;lt;b&amp;gt;Inmune&amp;lt;/b&amp;gt; mientras ataca.&lt;/esMX&gt;
 &lt;esES&gt;Tu héroe es &amp;lt;b&amp;gt;inmune&amp;lt;/b&amp;gt; al atacar.&lt;/esES&gt;
 &lt;deDE&gt;Euer Held ist &amp;lt;b&amp;gt;immun&amp;lt;/b&amp;gt;, während er angreift.&lt;/deDE&gt;
 &lt;/Tag&gt;
 &lt;Tag name=&quot;EnchantmentBirthVisual&quot; enumID=&quot;330&quot; type=&quot;EnchantmentVisualType&quot; value=&quot;0&quot; /&gt;
 &lt;Tag name=&quot;EnchantmentIdleVisual&quot; enumID=&quot;331&quot; type=&quot;EnchantmentVisualType&quot; value=&quot;0&quot; /&gt;
 &lt;Tag name=&quot;ArtistName&quot; enumID=&quot;342&quot; type=&quot;String&quot;&gt;
 &lt;enUS&gt;Peter C. Lee&lt;/enUS&gt;
 &lt;/Tag&gt;
 &lt;Tag name=&quot;FlavorText&quot; enumID=&quot;351&quot; type=&quot;String&quot;&gt;
 &lt;enUS&gt;The longbow allows shots to be fired from farther away and is useful for firing on particularly odorous targets.&lt;/enUS&gt;
 &lt;zhTW&gt;長弓能從更遠的地方發射，對於特別臭的目標格外好用。&lt;/zhTW&gt;
 &lt;zhCN&gt;弓弦很长，这使得射手能够射得更远，对付那些难闻的目标尤其有效。&lt;/zhCN&gt;
 &lt;ruRU&gt;Длинный лук позволяет стрелять намного дальше, что особенно полезно при стрельбе по дурно пахнущим целям.&lt;/ruRU&gt;
 &lt;ptBR&gt;O arco longo permite atirar flechas mais de longe e é útil para disparar contra alvos especialmente fedorentos.&lt;/ptBR&gt;
 &lt;plPL&gt;Ten długi łuk pozwala na ostrzał z naprawdę daleka i przydaje się do likwidowania szczególnie cuchnących celów.&lt;/plPL&gt;
 &lt;koKR&gt;멀리서 쏠 수 있기 때문에 냄새가 지독한 적을 상대할 때 유용합니다.&lt;/koKR&gt;
 &lt;itIT&gt;Gli archi lunghi possono colpire da molto lontano, il che è utile quando bisogna colpire bersagli particolarmente maleodoranti.&lt;/itIT&gt;
 &lt;frFR&gt;Cet arc long permet de tirer depuis une certaine distance, ce qui est particulièrement utile contre les cibles qui... sentent.&lt;/frFR&gt;
 &lt;esMX&gt;El arco largo te permite disparar desde más lejos y es muy útil cuando los enemigos son especialmente apestosos.&lt;/esMX&gt;
 &lt;esES&gt;El arco largo permite disparar desde mayores distancias y es muy útil para eliminar a objetivos particularmente apestosos.&lt;/esES&gt;
 &lt;deDE&gt;Da dieser Langbogen sich hervorragend dazu eignet, Feinde aus großer Entfernung zu erledigen, wird er für die Jagd auf stark riechende Ziele empfohlen.&lt;/deDE&gt;
 &lt;/Tag&gt;
 &lt;ReferencedTag name=&quot;Cant Be Damaged&quot; enumID=&quot;240&quot; type=&quot;Bool&quot; value=&quot;1&quot; /&gt;
 &lt;Power definition=&quot;d96807ae-a9fb-42f9-9c53-36e0f4da9446&quot; /&gt;
&lt;/Entity&gt;
</code></pre>
That's a whole lot of information about the card. Each separate card
appears to be demarcated by <code>&lt;Entity&gt;</code> tags. Each property of the card
is inside a <code>&lt;Tag&gt;</code>. Generating a full cardlist based on this file
should not be too difficult. Here's a Python script I came up with:
<pre><code class="language-python">#!/usr/bin/env python
import re

cardxml = open('cardxml0.unity3d', 'r').read()
cards = re.findall(r'\&lt;Entity.*?Entity\&gt;', cardxml, re.DOTALL)
with open('cards.xml', 'w') as f:
 f.write('&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;')
 for card in cards:
 f.write(card)
 f.write('\n')
</code></pre>
Now we have the raw data in our hands in an easy to access format. A
good next step would be to clean up the data by removing all the
duplicate cards and cards that are not obtainable in the game. We
should also prune all the properties we don't care about.
Some of the properties are pretty self-explanatory, but there are some
where the numeric values don't mean much by themselves. Here's a table
for some of the properties.
<pre><code>Value Class Rarity Card Type Card Set Race
1 - Common - - -
2 Druid Free - Basic -
3 Hunter Rare Hero Expert -
4 Mage Epic Minion Reward -
5 Paladin Legendary Spell Missions -
6 Priest - - - -
7 Rogue - Equipment - -
8 Shaman - - Debug -
9 Warlock - - - -
10 Warrior - Hero Power - -
11 - - - Promo -
14 - - - - Murloc
15 - - - - Demon
16 - - - Credit -
20 - - - - Beast
21 - - - - Totem
23 - - - - Pirate
24 - - - - Dragon
</code></pre>
There are also certain boolean properties that only show up if the
card has them, like Battlecry, Charge, Deathrattle, Divine Shield,
Stealth and Taunt. There is also a property called &quot;Poisonous&quot; for
minions like Emperor Cobra who kill everything they touch.
That's all I had time for today. Drop me a line in Twitter if you
enjoyed reading this.

Hearthstone Card List

<h1>gorb 0.1</h1>
13 September 2010
I just laid the finishing touches to
<a href="http://rubygems.org/gems/gorb">gorb</a> version 0.1. In the last post I
promised support for keeping track of captured stones and scoring in
the next update. I added these features and while doing them I also
added support for reading simple board diagrams.
First I tackled informal diagrams that are ideal for quickly setting
up some tsumego problems. You can feed the Board object a diagram like
this and it will set the board up. Here is an example of such a
diagram, in a custom format.
<pre><code>&gt; require 'gorb'
&gt; diagram = &lt;&lt;-END
 ---------------------
 | . . . . X O . . . |
 | . . . . X O . . . |
 | . . . . X O . . . |
 | . . . . X O . . . |
 | . . . . X O . . . |
 | . . . . X O . . . |
 | . . . . X O . . . |
 | . . . . X O . . . |
 | . . . . X O . . . |
 ---------------------
 END
&gt; board = Board.new
&gt; board.read(diagram)
</code></pre>
<code>Board#read</code> works by stripping off everything but dots, Xs and Os. It
then assumes that the diagram starts from the upper left corner and
starts putting stones in their correct places, switching rows when it
encounters a newline. In this way you can actually feed it only
partial diagrams, again excellent for tsumego which often require less
than the full board. As you can imagine, graphs like this also come in
handy when you need to set up board state for tests!
I also wanted to support output produced by
<a href="http://www.gnu.org/software/gnugo/">gnugo</a>. I haven't actually
researched if the output it produces is in any way canonical, but I'd
imagine it's a pretty good representative of a slightly more complex
diagram. Here is an example.
<pre><code>White (O) has captured 5 pieces
Black (X) has captured 3 pieces

 A B C D E F G H J K L M N O P Q R S T
19 . . . . . . X O O O O O . O X . . . . 19
18 . . . . X X X X O X X O . O X X . . . 18
17 . . . . X X X O O X X O . . O X X X X 17
16 . X X X O X O O X X X X O O O X O X O 16
15 X X O O O X O O O X O O . . . O O(O)O 15
14 O O . O . O X O X X O . . . . . O O O 14
13 O O O O . O X X . X X O . O . O . . O 13
12 X O X O . O X . . X O . O X O O . O X 12
11 X X X O O X O X . X O . O X X X O O X 11
10 X O X X X X O X . X O O X . . X X X X 10
 9 . . . . . X O O X X O X X . . . . . . 9
 8 . . . . X X X O O X X . . . . . . . . 8
 7 . . . X O O O . O O X X . . . . X . . 7
 6 . . . X O . . . . . O O X X X . . . . 6
 5 . . . X O . . . . . . . O O O X X . X 5
 4 . . . X X O . . . O . . O . O O X X O 4
 3 . . . . X O . . . . . . . . . . O O O 3
 2 . . X . X O . . . . . . . . . . . . . 2
 1 . . . X O O . . . . . . . . . . . . . 1
 A B C D E F G H J K L M N O P Q R S T
</code></pre>
This format required some <a href="http://xkcd.com/208/">magical regexp powers</a>
to tame.
I bumped the version up to 0.1 and pushed the latest version into
<a href="http://rubygems.org/">RubyGems</a>. I think I should add support for
reading and writing SGF files next.

gorb 0.1

<h1>Go, the ancient board game</h1>
1 September 2010
In year 2002 I encountered
<a href="http://en.wikipedia.org/wiki/Go_%28game%29">Go</a>, the ancient oriental
board game, for the first time. Back then I was studying Japanese and
reading a lot of manga when I came across one about an old board game.
I was pretty intrigued from the outset: how could you possibly tell an
interesting story about a board game? I was pleasantly surprised,
along with many others: ヒカルの碁 (Hikaru no Go) took the
manga-reading world by storm and introduced this ancient game to a lot
of people.
After reading about playing go I decided to learn how to play it
myself. I found <a href="http://gokgs.com/">KGS</a>, an online Go server pretty
quickly and started playing there regularly. After the initial giant
leaps of progress I felt like I hit a wall and stopped playing for
almost a year. Ever since I've been playing it in an “on again, off
again” manner.
Over the years I thought about making my own virtual Go board for fun
and exercise, and sometime in 2008 I finally decided to go ahead and
do it. The result was <a href="http://github.com/eagleflo/goban">goban</a>,
the first project I published on Github. Goban is written in Python
and comes with a <a href="http://pygame.org/">PyGame</a> -based GUI to provide
immediate feedback. I'm still pretty proud of how it came into shape:
I started with the GUI and thus could immediately test the real
library as it was being written. However, PyGame has been pretty
inactive during these years and only recently got support for
antialiased circles, which are absolutely essential for drawing the
stones. I also omitted tests because I used the GUI to test manually,
something that really bit me in the ass later on when I encountered
trouble with capturing by filling dame and the ko rule. After I
stopped playing go again I promptly forgot about goban for over a
year.
Some time last year after getting more familiar with the new
possibilities HTML5 offered it dawned on me: how cool would it be to
create a HTML5 based virtual go board? That would solve a major
problem that plagued my last attempt: nobody wants to install Python
and PyGame to try it out. But everyone has a browser, and by hiding
the go engine behind the server it would be trivial for anyone to try
it. I had heard great things about this new minimal Ruby web framework
called <a href="http://sinatrarb.com/">Sinatra</a> and together with HTML5,
canvas and JavaScript it turned out to be a really nice way to
prototype web apps. <a href="http://github.com/eagleflo/w2go">w2go</a> (way
to go), a browser-based go board is my first Sinatra app. Using canvas
to draw the board worked beautifully. I was astonished how easy it was
to develop a fully working go front end with these technologies. Sadly
w2go was lacking the engine itself: pretty GUI will be nothing but a
useless toy without some logic behind it. Goban could not be used
directly because it was written in Python and Sinatra apps are
supposed to be written in Ruby.
Now that I'm about to start playing again I'm ready to unveil
something I've been tinkering with in my spare time for a while:
<a href="http://github.com/eagleflo/gorb">gorb</a>. Gorb is in many ways an
improved version of goban that has been translated to Ruby. Learning
from past mistakes, I started by writing an extensive suite of unit
tests to really get the rules down, including all the corner cases.
Gorb finally passes this entire test suite and I now feel comfortable
enough with it to publish its initial version on Github. It supports
the Japanese rules and can handle both handicaps and non-standard komi
with ease.
Gorb will be the heart of w2go, and both of these two projects are
under heavy further development. Expect to see support for solving
sets of tsumego problems, scoring and reading
<a href="http://en.wikipedia.org/wiki/Smart_Game_Format">SGF</a> files soon. I've
also been planning to create some sort of interactive introduction to
go with w2go. In the meanwhile, go ahead and take a look at these
projects!

Go, the ancient board game

<h1>StarCraft II, mpyq and adjutant</h1>
30 August 2010
StarCraft II was easily the most anticipated game in 2010 for me. I
had to pre-order it from US to get into the beta and I have probably
sunk a couple of hundred hours into playing it already. The game is
excellent and definitely a worthy successor to the venerable
StarCraft: Brood War.
The idea of creating some kind of library to help me analyze the heaps
of replay files I'd generate through playing on ladder crossed my mind
pretty early in the beta. Ideally it would later form the backbone of
some sort of web application with pretty graphs and everything, but
the library would have to exist first — so I started writing it.
I chose Python for this project because it's the language I'm most
familiar with and I was eager to get fast results.
I think Blizzard made a mistake in not making StarCraft II replay
format completely public from the get-go. Writing both of these tools
required considerable reverse engineering efforts to get anywhere.
There are still a LOT of unknown details in the replay files
— I feel I've barely scratched the surface so far. Making the
replay format open and documenting it would help tremendously.
<h2>mpyq</h2>
The first issue I encountered was MPQ: Blizzard has been using MPQ
(MoPaQ) as a fast-to-read binary archive format for their game assets
for ages, and StarCraft II replays are no exception. After briefly
getting familiar with the various C-based MPQ libraries I decided to
roll my own — I did not want a C-based external dependency to my
project and I wanted to get more familiar with the MPQ format myself.
Luckily I found a couple of
<a href="http://www.zezula.net/en/mpq/mpqformat.html">great</a>
<a href="http://wiki.devklog.net/index.php?title=The_MoPaQ_Archive_Format">resources</a>
of the inner workings of MPQ and wrote a preliminary version of my MPQ
library in pure Python over the following weekend.
The result is <a href="http://github.com/eagleflo/mpyq/">mpyq</a>, a Python
library and command line tool for extracting MPQ archives. I was
intrigued by the idea of having an &quot;executable library&quot; and to my
delight this approach worked nicely.
The library is not the fastest MPQ library around by any means and it
only handles a couple of the various compression schemes MPQ supports,
so its general purpose utility is kind of limited at this point.
However, it works great for smaller MPQ archives like replays. It is
also great for taking a peek inside the larger MPQ archives —
full extraction is not required to get a decent idea of what's inside,
and with mpyq you can easily extract only the files you are interested
in. It's great for exploring Blizzard's game assets.
In the future I'm going to add support for more compression schemes
for increased backwards compatibility, add a decent test suite and
turn the project into a &quot;real&quot; Python project indexed in PyPI and
installable as an egg.
<h2>adjutant</h2>
That brings me to the actual StarCraft II replay parser,
<a href="http://github.com/eagleflo/adjutant/">adjutant</a>. After the
initial version of mpyq was done, using it to parse the key details of
replay files was pretty easy. I chose to separate the MPQ library from
StarCraft II specific code pretty early on to keep the design clean
and to make it easier to create other MPQ-related tools. The first
independent version of my replay analyzer came together slightly
before StarCraft II was released and it was able to gather map name,
game duration, client version and players with races and colors from
the replay file.
After playing a couple of games and wanting to review them I noticed
the lack of sophistication in the automatically generated replay
names: first only a timestamp and later only the map name. I thought
that this minor annoyance would be fixed before the game was released,
but sadly Blizzard did not deliver on this front. Many players want to
study and archive their replay files for future reference, and names
like “Toxic Slums (434).SC2Replay” are not very illuminating. The
replay browser also doesn't display the relevant data like players and
map name in a tabular form, so players are forced to hunt for the
right replay manually.
I tried to persuade one my programmer friends to tackle on this
relatively simple problem using my MPQ library, but apparently the
documentation is still lacking as he found it a little too
intimidating. I was pretty confident it was a 10 minute job and said
so, and he challenged me to prove it. I added the renaming feature in
about an hour of continuous development time — not quite the 10
minutes but definitely not too hard either.
The tool is currently far from what it can become — as I said
before, I've barely scratched the surface. There is a lot of data to
mine from the replay files, starting from winning percentages in each
matchup all the way to statistics of APM and build orders in each
game. I'm definitely going to create some sort of batch analysis mode
where you feed the tool a directory of replays and it will aggregate
the data inside. I'm also thinking about packaging this library inside
a dedicated tool for only renaming the replay files. It would watch a
given directory for new files and automatically rename them using the
library. Stay tuned for version 0.2!

Aku Kotkavuo

@eagleflo