Tag Archives: software

The Strassen Algorithm in C++

Quite some time ago I wrote about some experiments I did with some matrix multiplication algorithms. I’ve finally got around to cleaning up and posting (most of) the source code I used to generate the data in that post.

The source is now hosted in my GitHub repository here.

In the project is a few toys for playing around with matrices, but the meat is in

src/strassen/strassen_matrix_multiplier.hpp
src/strassen/parallel_strassen_matrix_multiplier.hpp

There’s a few other algorithms in there for comparison’s sake, as well as a simple testing client that provides some timing information. I’m using CMake to generate the Makefiles; if you’ve never used it before, you should. It’s great.

Some things to note:

  • The Strassen algorithm has a very high constant factor; for this reason, for smaller matrices or sub-matrices, the code falls back to an optimized naive multiplication algorithm
  • The algorithm needs to pad matrices with zeroes if they’re either non-square and/or have dimensions which are not a power of two. For this reason, if your input matrices have a square dimension something like N = (2n + 1), the algorithm works with a matrix almost twice that size. For that reason it will perform badly on matrices with this characteristic.
  • This implementation is mostly a demonstrative toy in the hope someone finds it useful; its not particularly optimized and the parallel Strassen class is kind of naive.

Happy hacking!


Customizing TR1 unordered_map Hashing and Equality Functions

A common problem people run into when using the C++ std::tr1::unordered_map or std::tr1::unordered_set is when they want to use a custom or aggregate type as the key in the hash table, and aren’t sure how to do it.

There isn’t really any official documentation about this; you can find some forum and StackOverflow discussions which may help; but I’ll run through a detailed example here too. Let’s start by taking a look at how the unordered_map class is defined:

template<class Key,
         class T,
         class Hash = hash<Key>,
         class Pred = std::equal_to<Key>,
         class Alloc = std::allocator<std::pair<const Key, T> > >
  class unordered_map;

The templated types in the class are:

  • class Key: key type (duh)
  • class T: value type
  • class Hash: hash function for type Key
  • class Pred: an equality predicate for type Key
  • class Alloc: the memory allocator for <Key, T>

The function represented in class Hash produces a hash value given a reference to type Key. This is the value that the map will use to locate the appropriate bucket to insert the corresponding value. The function represented by Pred is a predicate; it returns true if the values (as opposed to hash values) of Key types are equal. If the Key type is a class with operator== () defined, then defining a separate Pred function won’t be an issue.

Changing the Hash and Pred types is easy, but non-obvious. The most common reasons for this are either using a custom type as the key for the hash table, as I mentioned above, or for using a custom hash function to achieve better performance from the container.

Let’s say we have the following setup:

typedef struct
{
  uint32_t a;
  double x;
  double y;
} AggregateKey;

{ ... }

AggregateKey k;

{...}

std::tr1::unordered_map<AggregateKey, int> M;
M[k] = 1;

{...}

Attempting to compile this will generate a ton of error output. Depending on the situation, it’ll complain about Pred with something like

error: no match for ‘operator==’ in ‘__x == __y’

or about Hash with

undefined reference to `std::tr1::hash<AggregateKey>::operator()(AggregateKey) const'

Fix the issue by defining the following functors:

typedef struct
{
  long operator() (const AggregateKey &k) const { return my_hash_fnct (k); }
} AggregateKeyHash;

typedef struct
{
  bool operator() (const AggregateKey &x, const AggregateKey &y) const { return my_eq_test (x, y); }
} AggregateKeyEq;

AggregateKey k;

{...}

// Now, hash value generation and equality&nbsp;testing are
// defined for the AggregateKeyHash&nbsp;type, so declare the
// map using the functors&nbsp;above in the object's template list
std::tr1::unordered_map<AggregateKey, int, AggregateKeyHash, AggregateKeyEq> M;
M[k] = 1;

{...}

The same logic applies to the std::tr1::unordered_set, with the exception that its template list doesn’t include a class T value type. Also note that you can grab the any map’s default hash function and equality predicate at any time from the object:

std::tr1::unordered_map<int, int> M;

const std::tr1::unordered_map<int, int>::hasher &hfn = M.hash_function ();
const std::tr1::unordered_map<int, int>::key_equal &eqfn = M.key_eq ();

long h = hfn (123);
bool b = eqfn (1, 2);

Learning Functional Languages

I took it upon myself to begin learning Haskell recently. This isn’t my first adventure into the realm of functional languages; I spent some time a year or two ago learning Erlang as well. As someone who spends most of their time developing in (and quite frankly, enjoying) C and/or C++, it might seem like a strange foray into bizzaro-world. However, I don’t have any definite or particular interest in the functional paradigm, only in the sense that it offers a new perspective on the way software is built, and perspective is good.

From the imperative perspective, functional programming is a bizarre and imposing approach to writing programs. These languages are all concerned with things like side effects, mutability, and state, and its often difficult to identify the advantages of functional languages over more conventional ones with respect to these concepts and others. In addition, they often have the stereotype of being less practical and more academic (translation: interesting but useless) technologies – although this is not necessarily the case.

I’ve been learning Haskell from this tutorial, which I recommend not only for its technical completeness, but also the author’s sense of humor. I’m not at the stage yet where I was with Erlang (which was, actually building programs), but I could be if I spent a little more time on the subject. So far, one thing I’ve found to be an incredibly useful aspect of Haskell is:

  • Its clean, rigid syntax

However, the thing that probably drives me the most insane about Haskell is:

  • Its clean, rigid syntax

Make no mistake, that when you begin building some application in Haskell, your code will look pretty much identical to what is produced by your coworkers. If you have some sort of writing style, or you don’t like the default formatting and structure of the language, too bad. It sort of forces you into structuring your application a certain way, and to be honest, this is invaluable when you’re collaborating with others – you should know what to expect in their work.

Haskell people will also argue that the immutability of variables, the brevity of code for tasks, and stateless program design will reduce defects as well in the final product. I can’t find any numbers to back that up, but that seems like it might have a reasonable amount of truth to it. The tradeoff being, spending five years finding a developer who is an expert in Haskell.

I stopped learning Erlang at the time because I couldn’t stand the syntax; Erlang has a lot of very strange characters and formatting in the language. Certainly, you would have to write considerably less code than for an equivalent program in C++, but I’d use Python in that case any day over it.

Anyways, stay tuned for more feedback about Haskell. Should be an interesting adventure.


ptrace() Tutorial

The ptrace() system call provides the unique functionality under many *nix systems allowing a process to examine the data, and control the execution of, another running process. This includes the ability to read and write register values, arbitrary values from memory, and signals. The tracing process (from here on, the parent) may establish this relationship with the process being examined (the child) by either fork()ing the process as a literal child process, or by assuming the temporary parental responsibilities of the running child process by attaching to it during execution. The most useful applications of this system call would be building debuggers and process tracing tools. I’m also going to focus on the GNU/Linux version of ptrace.

There isn’t a ton of online documentation about how to use ptrace, probably due to the fact its undoubtedly one of the more infamous and disliked system calls available in POSIX. If you’ve never had to use it before, you’ll find it both an educational and frustrating experience. The manpage isn’t bad, but lacks a lot of context.

I’m going to try and avoid using the word process from here on in, and use task instead. This is because ptrace can be applied to individual threads within a process (at least, under Linux). Due to the complicated ambiguity between threads and processes on Linux, I’ll try to limit confusion and just refer to tasks instead.

Let’s take a look at the function definition from the manpage:


#include <sys/ptrace.h>

long ptrace (enum __ptrace_request request,
             pid_t pid,
             void *addr,
             void *data);

Taking a look at the parameters to the function:

  • __ptrace_request request: A code provided to ptrace telling it which operation to perform. More on this later.
  • pid_t pid: The task ID to perform the operation on
  • void *addr: The address in memory for which to read or write for certain ptrace operations; other operations ignore this parameter.
  • void *data: Address for various data structures to be read/written to/from the process. More on this later as well.

ptrace also returns a long integer, which for all ptrace operations except the peeks return 0 for success and -1 for error. For the peek operations it returns data read from the child, and -1 on error… sometimes. I’ll cover that too.

As you probably noticed from all the special cases above, this is not a straightforward or simple system call to use. There are a lot of special cases for input and output to consider depending on what you happen to be using it for at any given time. I’ll flesh out some of these further below. But firstly, let’s talk about the behavior of a child task under ptrace.

A child being traced has two basic states: stopped and running. ptrace operations cannot be performed on a running child process, thus, they must be done either when

  1. The child stops on its own
  2. The parent stops the child manually

Typically a process will stop (I’m talking about a ‘T’ status here) when it receives a SIGSTOP signal. However, when being traced, a child will stop upon receiving of any signal, with the exception of SIGKILL. This is true for signals that the child is explicitly ignoring as well. After receiving notification that the child has stopped via wait(), the parent can take the time to perform various ptrace operations, or can tell the child to continue executing through ptrace, either delivering or ignoring the signal which caused the stoppage.

If the parent process would like the child to stop (for example, after user input in a debugger), it can simply send it a SIGSTOP through the usual methods. Again, technically any unused signal besides SIGKILL would do the job, but its best to avoid ambiguity. It is important to ensure that the child task is stopped before doing anything to it; otherwise ptrace will return an ESRCH error: “no such process”.

Let me itemize the states involved in stopping, ptrace()-ing, and running a child process in this scenario:

  1. Child process is running
  2. Child process stops after receiving signal (SIGSTOP/SIGTRAP/other)
  3. Parent receives child signal status from wait()
  4. Parent performs various ptrace operations
  5. Parent signals child to continue executing

Any ptrace operations performed outside of step 4 will fail. Make sure that you have appropriately been notified that the child is stopped before trying to use ptrace. I mentioned above using the wait() call to retrieve process status of a traced child process. This is correct – as with a conventionally fork()ed process, the tracing parent uses wait() to determine task state after receiving a signal. In fact, it might be easier to use waitpid() so that you can specify the exact task you’re waiting for, in case you’re tracing multiple tasks/threads simultaneously.

Alright, let’s talk about some of the more interesting ptrace codes. I’ll provide a short code sample of a call for each respective request; any NULL argument is an argument unused by ptrace for that request. Firstly, the codes that deal with initiating and terminating the tracing of the child task.

PTRACE_TRACEME


long ret = ptrace (PTRACE_TRACEME, 0, NULL, NULL);

This is the only ptrace operation which is used by the child. It’s purpose is to indicate that the child task is to be traced by a parent and to grant it necessary ptrace permissions. The 0 in the pid field refers to the child task’s parent. As soon as the child makes a call to any of the exec() functions, it receives a SIGTRAP, at which point it is stopped until the tracing parent allows it to continue. It is important for the parent to wait for this event to happen before performing any ptrace operations, including the configuration operations involved with PTRACE_SETOPTIONS.

PTRACE_ATTACH


long ret = ptrace (PTRACE_ATTACH, target_pid, NULL, NULL);

This is used by a task when it wishes to trace the execution of another task. For the most part, this will make the process represented by target_pid the literal child of tracing task. By and large, the situation created by using PTRACE_ATTACH is equivalent to what would’ve happened if the child had used PTRACE_TRACEME instead.

An important note is that this operation involves sending a SIGSTOP to the targeted process, and as usual, the parent needs to perform a wait() on target_pid after this call before continuing with any other work to ensure the child has properly stopped.

PTRACE_CONT


long ret = ptrace (PTRACE_CONT, target_pid, NULL, 0);

This will be the request you’ll use each time that wait() indicates that the child has stopped after receiving a signal to get it running again. If the data field is anything besides zero or SIGSTOP, ptrace will figure its a signal number you’d like delivered to the process. This can be used to actually deliver signals to the child which caused it to stop and notify the parent before acting on them. For common signals like SIGTRAP, you probably won’t want to do this. However, if you’d like to see if the child properly handles a SIGUSR1, this would be one way to go about it.

PTRACE_DETACH

long ret = ptrace (PTRACE_DETACH, child_pid, NULL, 0);

Completes the tracing relationship between the parent and child, and if the parent attached to the child, “re-parents” the child back to its original parent process. Then it continues the child with a SIGCONT.

Now that we’ve covered the basics of how to get a tracing running, let’s get to some of the more interesting stuff.

PTRACE_PEEKTEXT | PTRACE_PEEKDATA

long word = ptrace (PTRACE_PEEKDATA, child_pid, addr, NULL);
if (word == -1)
  if (errno)
    fail ();

On GNU/Linux systems, text and data address spaces are shared, so although these two codes would be used interchangeably here, on other UNIX platforms this would not be the case. The purpose of this request is to read words from the child task’s data address space and inspect the values. I mentioned above that peek operations require a little extra effort when detecting errors, which is briefly outlined in the code snippet above. Although ptrace will return -1 for error on a peek operation, -1 may also be the value stored at the provided memory address. Thus, errno must be checked in these situations to ensure an error actually happened.

The utility of this request is obvious – reading values from memory addresses in another task’s address space. If you consider GDB, printing variables or setting breakpoints would all need to use this request.

PTRACE_POKETEXT | PTRACE_POKEDATA

long ret = ptrace (PTRACE_POKEDATA, child_pid, addr, new_val);

Conversely to the peek functions, the poke functions do the opposite – write arbitrary values into the memory space of the child task. This is useful if you’d like to examine the change in behavior of the child task given different parameters, or for debugging tasks such as inserting breakpoints. This is turning into a pretty long post, but I can cover how to insert breakpoints into a child task’s address space on a later blog post.

PTRACE_SINGLESTEP

long ret = ptrace (PTRACE_SINGLESTEP, child_pid, NULL, NULL);

The single-step request is actually several operations batched into one. A PTRACE_SINGLESTEP request will execute a single instruction in the child task, then stop the child and notify the parent with a SIGTRAP. The operations involved include setting and removing a breakpoint so that only a single instruction is executed. This can be used to slowly step through the execution of a program, and assist with the usage of the other ptrace operations above. Think “stepi” from GDB.

PTRACE_GETREGS | PTRACE_SETREGS

#include <sys/user.h>
user_regs_struct regs;
long ret = ptrace (PTRACE_GETREGS, child_pid, NULL, &regs);
#ifdef __x86_64__
regs.rip = 0xdeadbeef;
#elif defined __i386__
regs.eip = 0xdeadbeef;
#endif
ret = ptrace (PTRACE_SETREGS, child_pid, NULL, &regs);

These ptrace requests involve reading and writing the general-purpose register values for the child process. The above example does three things:

  1. Reads the values of all general-purpose registers associated with child_pid
  2. Sets the instruction pointer of the user_regs_struct structure to a not-so-random address
  3. Writes the edited user_regs_struct back to the child, likely causing a crash upon re-execution due to the new instruction pointer setting

Similar functionality is available for the designated floating-point registers as well through the use of PTRACE_GETFPREGS and PTRACE_SETFPREGS.


Virtual Memory (Theory vs Application part 3)

I think I was reading Reddit a little while ago when a pretty interesting article from the ACM Communications by Poul-Henning Kamp came up. It might be a little sensationalist, but its a pretty valid examination of what you can do to program performance when you consider how modern operating systems manage virtual memory. Kamp spent a lot of time building Varnish, and although I’d never heard of it before, it sounds like a very impressive utility. Its an HTTP accelerator and cache whose performance apparently blows Squid out of the water.

The meat of the article discusses how conventional thinking about the implementation of algorithms and data structures on modern computers is broken because they fail to consider the consequences of cache misses and page faults when accessing data.

On a modern multi-issue CPU, running at some gigahertz clock frequency, the worst-case loss is almost 10 million instructions per VM page fault. If you are running with a rotating disk, the number is more like 100 million instructions. What good is an O(log2(n))algorithm if those operations cause page faults and slow disk operations? For most relevant datasets an O(n) or even an O(n2) algorithm, which avoids page faults, will run circles around it.

Kamp provides some images and diagrams to visualize some of what he’s talking about. As a traditionally more systems-focused guy, I might be a little biased, but I feel that modern CS curriculums pump out students who have absolutely no idea about how computers actually work. Knowing the asymptotically optimal algorithm for everything is great; but if your practical implementation sucks because you’re causing paging all over the place, what good is it? A reasonable mixture makes sense – and the curriculum could make room by removing that course on CVS (and maybe even calculus – barf!)

I discussed a little earlier about how more intelligent practical implementations can have enormous effects over more naive solutions even though they remain asymptotically equal. Imagine wasting a hundred million instructions because your algorithm’s implementation unnecessarily left some node on disk!


Parallel MapReduce in Python in Ten Minutes

Almost everyone has heard of Google’s MapReduce framework, but very few have ever hacked around with the idea of map and reduce. These two idioms are borrowed from functional programming, and form the basis of Google’s framework. Although Python is not a functional programming language, it has built-in support for both of these concepts.

A map operation involves taking a function f and applying it on a per-element basis to a given list L. For example, if f was the square-root function, then the map would take the square of each element in L. A reduce operation (also known as folding) is similar in that it also applies a function g to a given list L, but instead of isolating on each element, g systematically accumulates or collapses the contents of L into a single result. The canonical example of this would g performing a summation of the elements in L.

Now, there’s a lot of documentation on MapReduce so I won’t replicate talk about it much here. I’ll just briefly mention that it is an efficient way of distributing the workload of huge embarrassingly parallel problems over multiple processors, or over a grid of machines. The map operation takes input which has been fragmented into pieces by a master node, and applies its function to the input on each of the worker nodes in the system. The data is then organized and the useful information extracted by the reduce step on various worker nodes in the system. Google provides a diagram of the operation in their documentation:

Pretty straightforward... right?

Anyways, this post isn’t specifically about MapReduce, its about Python. I mentioned above that Python provides built-in map and reduce functionality. Lets say we have a problem where we have several lists, and we’d like to know the number of total elements in all the lists combined. In Python, a solution (using lambdas for brevity) might look like this:

a = [1, 2, 3]
b = [4, 5, 6, 7]
c = [8, 9, 1, 2, 3]
L = map(lambda x:len(x), [a, b, c])

# L == [3, 4, 5]
N = reduce(lambda x, y: x+y, L)
# N == 12
# Or, if we want to be fancy and do it in one line
N = reduce(lambda x, y: x+y, map(lambda x:len(x), [a, b, c]))

Now, this is an interesting way to think about getting the combined lengths of these lists, but it isn’t any more efficient than getting the lengths with any other conventional method. But what if we had a large problem which we could solve in parallel?

Let’s pretend we’re interested in finding the frequencies of title-cased words (proper nouns, starts of sentences) in an enormous corpus, and we’d like to use map and reduce to help compute this problem on our multi-core computer. Most people’s first instinct would be “let’s break out the Python threading module!” which, of course, would be a mistake – thanks to something called the global interpreter lock used in the Python interpreter. The GIL basically handicaps parallelism in the Python interpreter, by serializing execution of user-threads for various complicated reasons. A solution is to use the Python multiprocessing module, which will spawn entirely separate processes to execute common code on. And, as you’d expect from Python, it provides you with a super-easy-to-use Pool class to manage your own worker process pool.

Considerably more frightening than the programming language

Probably the best thing about the Pool class is that it provides a map function of its own – which will automatically partition and distribute input to a user-specified function in pool of worker processes. This is like map in the functional sense, but it doesn’t necessarily mean it’s performing the map step in MapReduce – you’ll see what I mean. So let’s see some code for the problem I mentioned above; we’ll start with the map, partition, and reduce functions:

"""
Given a list of tokens, return a list of tuples of
titlecased (or proper noun) tokens and a count of '1'.
Also remove any leading or trailing punctuation from
each token.
"""
def Map(L):

  results = []
  for w in L:
    # True if w contains non-alphanumeric characters
    if not w.isalnum():
      w = sanitize (w)

    # True if w is a title-cased token
    if w.istitle():
      results.append ((w, 1))

  return results

"""
Group the sublists of (token, 1) pairs into a term-frequency-list
map, so that the Reduce operation later can work on sorted
term counts. The returned result is a dictionary with the structure
{token : [(token, 1), ...] .. }
"""
def Partition(L):
  tf = {}
  for sublist in L:
    for p in sublist:
      # Append the tuple to the list in the map
      try:
        tf[p[0]].append (p)
      except KeyError:
        tf[p[0]] = [p]
  return tf

"""
Given a (token, [(token, 1) ...]) tuple, collapse all the
count tuples from the Map operation into a single term frequency
number for this token, and return a final tuple (token, frequency).
"""
def Reduce(Mapping):
  return (Mapping[0], sum(pair[1] for pair in Mapping[1]))

So, what’s going on here? Well, there’s three steps to this process. First of all, the Map function receives independent lists of tokens from the source text. In parallel, it will attempt to strip off any leading or trailing punctuation (lines 12-13), and if it is a properly capitalized word, adds it to the results list with a count of 1 (lines 16-17). All such tokens are processed in this way, and duplicates are also appended to this list.

After the Map operation is complete, the program needs to organize all the processed data. This is where the Partition function comes in. This function receives a huge list of lists – each constituent list was processed by an instance of Map and thus contains a sequence of (token, 1) pairs. This function organizes all the data into a dictionary, so that now each token key has a list of (token, 1) tuples as a value.

The final step is in the Reduce function. It receives arbitrary key:value pairs from the dictionary built in Partition, which it uses to generate the final term counts for title-cased words. In parallel, the list in each dictionary entry is collapsed, summed, and the final count for each token key is returned by each instance of Reduce. The final result is a list of (token, count) tuples, where count refers to the total count of the token over the entire document.

Let’s get to the Python multiprocessing code, and the other stuff we need to make the above functions work. I’ll create a worker pool of eight processes, and distribute the above workload to those processes via the Pool.map function:

import sys
from multiprocessing import Pool
"""
If a token has been identified to contain
non-alphanumeric characters, such as punctuation,
assume it is leading or trailing punctuation
and trim them off. Other internal punctuation
is left intact.
"""
def sanitize(w):

  # Strip punctuation from the front
  while len(w) > 0 and not w[0].isalnum():
    w = w[1:]

  # String punctuation from the back
  while len(w) > 0 and not w[-1].isalnum():
    w = w[:-1]

  return w
"""
Load the contents the file at the given
path into a big string and return it.
"""
def load(path):

  word_list = []
  f = open(path, "r")
  for line in f:
    word_list.append (line)

  # Efficiently concatenate Python string objects
  return (''.join(word_list)).split ()

"""
A generator function for chopping up a given list into chunks of
length n.
"""
def chunks(l, n):
  for i in xrange(0, len(l), n):
    yield l[i:i+n]

"""
Sort tuples by term frequency, and then alphabetically.
"""
def tuple_sort (a, b):
  if a[1] < b[1]:
    return 1
  elif a[1] > b[1]:
    return -1
  else:
    return cmp(a[0], b[0])

if __name__ == '__main__':

  if (len(sys.argv) != 2):
    print "Program requires path to file for reading!"
    sys.exit(1)

  # Load file, stuff it into a string
  text = load (sys.argv[1])

  # Build a pool of 8 processes
  pool = Pool(processes=8,)

  # Fragment the string data into 8 chunks
  partitioned_text = list(chunks(text, len(text) / 8))

  # Generate count tuples for title-cased tokens
  single_count_tuples = pool.map(Map, partitioned_text)

  # Organize the count tuples; lists of tuples by token key
  token_to_tuples = Partition(single_count_tuples)

  # Collapse the lists of tuples into total term frequencies
  term_frequencies = pool.map(Reduce, token_to_tuples.items())

  # Sort the term frequencies in nonincreasing order
  term_frequencies.sort (tuple_sort)

  for pair in term_frequencies[:20]:
    print pair[0], ":", pair[1]

As you can see, with three calls to the Python multiprocessing.Pool class, I can create a worker pool of 8 processes, and perform a distributed map and a distributed reduce operation (lines 64, 70, 76). Pool.map takes care of the partitioning and distribution of the data over the worker pool by itself. Don’t get confused by the two calls to map though – although it is technically a map operation, the first call is mapping a second map function to the data, and the second is mapping a reduce function to the data.

Finally, it prints out the top twenty token:count pairs. As you can see, the Python multiprocessing code is really easy to use. You don’t need to organize your problem as a MapReduce problem in order to properly use the process pool class either – I just used this opportunity for illustration. This definitely isn’t the most efficient example of how to use map and reduce to solve big problems, but I hope it’s a reasonably clear introduction. Happy Hacking!


Jack Dorsey Talking at Digg

Today, Ken sent me this video of a tech talk by Jack Dorsey while visiting Digg. I usually don’t have the patience to sit through 40-minute video lectures, but I did watch this one in its entirety.

Dorsey was informally talking about some of his experiences leading up to his founding of Twitter, the challenges they faced early on (especially after their feature at, and subsequent popularity after, SXSW in 2007) and some of the approaches they took to resolve them. I’d recommend watching it as well. Let me summarize some of the points Dorsey made which I found particularly interesting:

We didn’t really turn the company and technology around for a year. And, Some of the reasons for that were that we had no instrumentation at all. We had no idea of what was going on in the system.

Apparently Twitter was going down daily after the SXSW conference, and the engineers at Twitter were not able to isolate all the bottlenecks and points of failure in their systems because of inadequate system monitoring, logging, analysis, etc. Dorsey says that at his new company (Square), he learned from the above mistake, and one of their initial priorities was building an exceptional dashboard which became a critical component of development at the company.

That’s a pretty tough situation to be in, especially for a new company with limited resources. When you’re a small company on a limited budget, you need to make all that data accessible in order to make the right decisions. You need to be able to pinpoint every fault, every corner case, and know everything about them. That’s the difference between having to buy a new set of webservers and realizing your caching policy is just designed poorly.

Next:

We’ve had some bad engineering discipline, where we would try to isolate ourselves a bit too much in our work. So, we had some tendencies to really work on singular pieces of code [sic] and it came to the point where there was only one person in the company who knew how the entire system worked. That create a huge single point of failure. We were so afraid that any little change would bring down the entire site.

Twitter approached this problem simultaneously by creating two teams:

  1. The architecture team: redesign the entire Twitter system from the ground-up, with the knowledge they’d gained
  2. The iterations team: build instrumentation for every aspect of the system, and incrementally fix and improve it

Now, I would have thought that re-designing a broken system from the ground-up would have resulted in the best approach to the problem. What’s interesting is that the architecture team failed to produce a viable alternative, and in the meantime, the iterations team actually fixed all the problems using meticulous system analytics and group review of code. In addition, part of the solution involved the iterations team adopting pair programming. In addition, the pairs of people constantly rotated around the company, so that everyone had at least some sense of how the entire system worked. As a result, they had a lot more people talking, more perspectives on problems, and in general, better solutions. In addition, it avoided the problem of isolating people. Dorsey says that it created a very creative environment because of the constant exchange of ideas.

I, like most people who write software, make that “…ew” face when I hear someone mention PP. However, I realized that there is no better way to get more brains solving more common problems than the solution Twitter came up with. Big problems would receive much greater exposure to different perspectives, different experiences, and general discussion than if the conversation never left a small team (or individual) who was responsible for fixing them. I had an epiphany, kind of like that time Gerry invited Ian and I to that design patterns meeting. Lots of good ideas that solve real problems.

Anyways, I hope you watch the video and find it as interesting as I did. It’s a good story.


The Curiously Recurring Template Pattern

The CRTP is a C++ idiom which is both confusing when you read about it, but straightforward enough that you may have already used it without realizing it. Its kind of funny that way; a strange recursive templating technique.

I was working on some new system at work which involved some tedious class hierarchies in performance-critical components. While doing some research, I fell up on the CRTP wiki page while researching the enduring battle between the performance of polymorphic code and the complexities of modern CPU architectures.

Hold on a minute. Some background first.

Dynamic polymorphism is a language abstraction in which different types of objects can use generic interfaces, and the type of object (and thus, the nature of the work to be done) is determined at runtime. This is achieved in C++ though the use of virtual functions, which in turn have entries in a class virtual function tables. Because parent classes and sibling classes can share virtual function names, the purpose of a vtable is for a class to say “my implementation of function f() is located here.” When an object of a particular type is instantiated, the compiler throws into it a pointer to the vtable for the proper class. Class functions which are non-virtual do not require entries in a vtable because there is only one implementation and location of that function.

Static polymorphism is a language abstraction where different types of objects are determined at compile-time, and as a result do not require work at runtime to determine the type of the object or variable. As a result, class hierarchies built-in this fashion don’t require a vtable. This is where the CRTP comes in.

Say you have a class hierarchy like this:


class A
{
public:
  virtual void f () = 0;
  void g ();
};

class X : public A
{
public:
  void f ();
};

class Y : public A
{
public:
  void f ();
};

A is a parent class with a pure virtual function f(), with X and Y implementing A. A has an additional non-virtual function g(). Now say you have a function foo(A &a):


void
foo (A &a)
{
  a.f ();
}

When foo() calls A::f(), the following things happen:

  1. The object’s vptr (pointer to the class vtable) is used to retrieve the appropriate vtable
  2. The address of f() according to the vtable is loaded from an index into the vtable
  3. The program jumps to the loaded address

You can use the “-fdump-class-hierarchy” flag in g++ to generate the class hierarchy and vtable structure for your code. For the above code, the vtables look like this:

Vtable for A
A::_ZTV1A: 3u entries
0     (int (*)(...))0
8     (int (*)(...))(& _ZTI1A)
16    __cxa_pure_virtual

Vtable for X
X::_ZTV1X: 3u entries
0     (int (*)(...))0
8     (int (*)(...))(& _ZTI1X)
16    X::f

Vtable for Y
Y::_ZTV1Y: 3u entries
0     (int (*)(...))0
8     (int (*)(...))(& _ZTI1Y)
16    Y::f

You can see above entries in each class for the default constructor, the copy-constructor, and then an entry to the pure virtual function f(), with their offsets into the vtable. They’re eight bytes apart because its table of pointers. Note that A::g() does not appear in the vtable – this is because g() is a non-virtual function in A.

Obviously this requires a little more work than a non-virtual function, and there’s been research to determine what exactly the cost of this extra work is. As I mentioned earlier, there’s a lot of argument over the topic, and although it’s really no more expensive than de-referencing a pointer for every virtual function call, there are more subtle consequences regarding branch prediction, pipelining, etc that weigh in.

Regardless of all of that, if you have some very intensive code that uses lots of dynamic polymorphism, it’s going to cost you. Maybe not a lot, but some.

So what does the CRTP have to do with this? Well the goal is to resolve as much type information as possible at compile-time so that you can skip the vtable lookup step at runtime. We can do this by using templates inside the parent class definition. Now consider the following, revised class hierarchy:


template <typename Child>
class A<Child>
{
public:
  void f ();
  void g ();
};

class X : public A<X>
{
public:
  void f ();
};

class Y : public A<Y>
{
public:
  void f ();
};

Now we’ll implement A::f() as so:

template<typename Child>
void
A::f ()
{
  (static_cast<Child*> (this)) -> f ();
}

Ok, this probably need some explaining.

Note how the parent class uses a type Child as its templated typename. In typical uses of generic programming, this is not particularly unusual. Now, each of X and Y are defined as a subclass of A<Child>, where they pass their own class name as the templated type. This is confusing; however, totally syntactically legal. At compile-time, function Child::f() will get placed directly into A::f(), thus avoiding the need for a runtime vtable lookup. As a result, the virtual identifier is dropped from the parent class A, and now the vtables for all classes are empty. Thus, no runtime overhead.

What’s the tradeoff? Well, our function foo(A &a) from earlier no longer works, because A now requires a template parameter. You’d have to create overloaded functions or come up with some other way to deal with the fact that you’re using both A<X> and A<Y>. That’s the flexibility you lose with static polymorphism!

I said earlier that you may have used this pattern without even realizing it. This happened to me when I was working on the project I mentioned – and at the time I didn’t even consider it polymorphism (and thought I was getting away with a hack). Turns out it’s actually a recognized idiom. I guess C++ is like that.


Apache Thrift Tutorial – The Sequel

I’m going to cover building a simple C++ server using the Apache Thrift framework here, while my buddy Ian Chan will cover the front-end PHP interface in his own blog post.

The other day Ian and I were talking and thought it would be cool to do another Facebook/Apache Thrift tutorial, but this time he’d do the front-end client interface and I’d do the backend stuff. He really wanted to do an example of something that you’d find useful to send to a backend for processing from a PHP frontend client. So, we came up with the following thrift interface:

namespace cpp calculator

typedef list<double> Vector

enum BinaryOperation
{
  ADDITION = 1,
  SUBTRACTION = 2,
  MULTIPLICATION = 3,
  DIVISION = 4,
  MODULUS = 5,
}

struct ArithmeticOperation
{
  1:BinaryOperation op,
  2:double lh_term,
  3:double rh_term,
}

exception ArithmeticException
{
  1:string msg,
  2:optional double x,
}

struct Matrix
{
  1:i64 rows,
  2:i64 cols,
  3:list<Vector> data,
}

exception MatrixException
{
  1:string msg,
}

service Calculator
{
  /* Note you can't overload functions */

  double calc (1:ArithmeticOperation op) throws (1:ArithmeticException ae),
  Matrix mult (1:Matrix A, 2:Matrix B) throws (1:MatrixException me),
  Matrix transpose (1:Matrix A) throws (1:MatrixException me),
}

As you can see, we defined a simple calculator with a couple more functions for doing some basic matrix operations (yes, this seems to come up often); something that would suck in PHP. Generated the code with

thrift –gen cpp calculator.thrift

And away we go with the autogenerated C++ code. After you run the thrift generation for C++, it’ll make a directory called gen-cpp/. Under this, you can find relevant files and classes to do work based on your Thrift definition.


$ ls gen-cpp/
calculator_constants.cpp  Calculator_server.skeleton.cpp
calculator_constants.h    calculator_types.cpp
Calculator.cpp            calculator_types.h
Calculator.h

I renamed the generated Calculator_server.skeleton.cpp file (you’ll want to make sure you do this so your work isn’t overwritten the next time you generate Thrift code), and filled in the function stubs adding more functionality as necessary. This file is the only file containing code which you need to edit for your server – you need to fill in the logic here. The other autogenerated files contain necessary transport class, struct, and function code for your server to work. On the other end of things, Ian generated the PHP code and filled in those stubs – you can find his blog post for this project here. We also threw all the code online under Ian’s Github account – you can find all the source here.

Below I’ll list the code I filled in for the backend-side of this project.


#include "Calculator.h"
#include <stdint.h>
#include <cmath>
#include <protocol/TBinaryProtocol.h>
#include <server/TSimpleServer.h>
#include <transport/TServerSocket.h>
#include <transport/TBufferTransports.h>
#include <thrift/concurrency/ThreadManager.h>
#include <thrift/concurrency/PosixThreadFactory.h>
#include TThreadedServer.h>
using namespace ::apache::thrift;
using namespace ::apache::thrift::protocol;
using namespace ::apache::thrift::transport;
using namespace ::apache::thrift::server;
using namespace ::apache::thrift::concurrency;
using boost::shared_ptr;
using namespace calculator;

class CalculatorHandler : virtual public CalculatorIf
{
private:
/* It might be cleaner to stick all these private class functions inside some other class which isn't related to the Thrift interface, but for the sake of brevity, we'll leave them here. */
  double
  __add (double lh_term, double rh_term)
  {
    return (lh_term + rh_term);
  }

  double
  __sub (double lh_term, double rh_term)
  {
    return (lh_term - rh_term);
  }

  double
  __mult (double lh_term, double rh_term)
  {
    return (lh_term * rh_term);
  }

  double
  __div (double lh_term, double rh_term)
  {
    if (rh_term == 0.0)
      {
        ArithmeticException ae;
        ae.msg = std::string ("Division by zero error!");
        throw ae;
      }

    return (lh_term / rh_term);
  }

  double
  __mod (double lh_term, double rh_term)
  {
    if (rh_term == 0.0)
      {
        ArithmeticException ae;
        ae.msg = std::string ("Modulus by zero error!");
        throw ae;
      }

    return std::fmod (lh_term, rh_term);
  }

public:

  CalculatorHandler ()
  {
  }
/* Given the ArithmeticOperation, ensure it's valid and return the resulting value. */
  double
  calc (const ArithmeticOperation& op)
  {
    switch (op.op)
      {
      case ADDITION:
        return __add (op.lh_term, op.rh_term);

      case SUBTRACTION:
        return __sub (op.lh_term, op.rh_term);

      case MULTIPLICATION:
        return __mult (op.lh_term, op.rh_term);

      case DIVISION:
        return __div (op.lh_term, op.rh_term);

      case MODULUS:
        return __mod (op.lh_term, op.rh_term);

      default:
        ArithmeticException ae;
        ae.msg = std::string ("Invalid binary operator provided!");
        throw ae;
      }
  }
/* Multiply A and B together, placing the result in the "return value" C, which is passed as a Matrix reference parameter. */
  void
  mult (Matrix& C, const Matrix& A, const Matrix& B)
  {
    if (A.cols == B.rows && A.rows == B.cols)
      {
        double tmp;

        C.rows = A.rows;
        C.cols = B.cols;
        C.data.resize (C.rows);

        for (uint64_t i = 0; i < A.rows; i++)
          {
            C.data[i].resize (A.cols);

            for (uint64_t j = 0; j < A.cols; j++)
              {
                tmp = 0;
                for (uint64_t k = 0; k < B.rows; k++)
                  {
                    tmp += A.data[i][k] + B.data[k][j];
                  }
                  C.data[i][j] = tmp;
              }
         }
      }
    else
      {
        MatrixException me;
        me.msg = std::string ("Matrices have invalid dimensions for multiplication!");
        throw me;
      }
  }
/* Take the transpose of A and stuff it into the return Matrix T. */
  void
  transpose (Matrix& T, const Matrix& A)
  {
    T.rows = A.cols;
    T.cols = A.rows;
    T.data.resize (A.cols);

    for (uint64_t i = 0; i < A.rows; i++)
      {
        for (uint64_t j = 0; j < A.cols; j++)
          {
            T.data[j].push_back (A.data[i][j]);
          }
      }
  }
};

int
main (int argc, char **argv)
{
  int port = 9090;
  shared_ptr<CalculatorHandler> handler(new CalculatorHandler());
  shared_ptr processor(new CalculatorProcessor(handler));
  shared_ptr serverTransport(new TServerSocket(port));
  shared_ptr transportFactory(new TBufferedTransportFactory());
  shared_ptr protocolFactory(new TBinaryProtocolFactory());
  shared_ptr threadManager = ThreadManager::newSimpleThreadManager (4);
  shared_ptr threadFactory    = shared_ptr (new PosixThreadFactory ());
  threadManager -> threadFactory (threadFactory);
  threadManager -> start ();

 /* This time we'll try using a TThreadedServer, a better server than the TSimpleServer in the last tutorial */
 TThreadedServer server(processor, serverTransport, transportFactory, protocolFactory);
 server.serve();
 return 0;
}

Finally, the code was compiled either with the Makefile I posted onto Ian’s Github repo, or the following build script:

g++ -o calc_server -I./gen-cpp -I/usr/local/include/thrift/ CalculatorServer.cpp gen-cpp/calculator_constants.cpp gen-cpp/Calculator.cpp gen-cpp/calculator_types.cpp -lthrift

So this really isn’t the most complicated program in the world, but it gets the job done fairly simply and effectively (and yes, it actually works!). Note that as opposed to last time I used a TThreadedServer as the base Thrift server type here. Its a little more complicated to set up, but obviously is more useful than a single-threaded server. Interesting things to note:

  • Use of TThreadedServer for a multithreaded server
  • You fill in Thrift exceptions like any other struct, and throw them like any other exception
  • You can use typedefs and enums
  • You can’t overload function names

The last point is a real pain, as far as I am concerned. I’m not sure why the Thrift people couldn’t just mangle the function names so that they resolve to unique entities, but whatever. Anyways, what’s really cool is that we managed to build three common programs in two completely different languages using a single Thrift definition file. A backend in C++, and a frontends in PHP. Hope you find this useful – happy hacking!


COBOL and the collapse of the Canadian government

Sheila Fraser made an announcement last week which brought an extremely important issue onto the national stage: the Government of Canada’s IT infrastructure is about to implode. In fact, it’s so bad that key government services may shut down because the systems behind them are in such bad shape.

Apparently the National Immigration Program runs on COBOL and a database system which has been dead for over twenty years. Also the Department of Public Works’ pay and pension system is about to collapse. And the Auditor-General says it’ll take billions of dollars to fix.

The problem here is that the government never implemented any sort of continuous maintenance and update strategy for its technical infrastructure. And there apparently isn’t any sort of reasonably competent department to run such a program. The fact is, if your organization (or, government) is going to rely on an IT infrastructure for all its day-to-day operations, you can’t let it rot. You need technically competent people to maintain the system. You need to keep up-to-date with changes in the industry. And you need to investigate how newer and better tools will make the continual growth and expansion of your IT services manageable. Otherwise it’ll crumble. The situation out in Ottawa is exemplified by Stockwell Day’s brilliant statement:

“As you know, with technology, there are always people who are saying you should have newer and better.”

Apparently the problem with software is that it needs to be updated every forty years or so. Other government tools such as cars, pencils and coffee makers don’t suffer from the same issue.

My biggest concern isn’t the cost to fix the issue, it’s potentially more serious. If the Government of Canada can’t keep its legacy systems up-to-date (we’re talking almost half a century old technology here), then how is it supposed to protect sensitive information about itself and the citizens of the country from malicious hackers and other threats?