The Lonely Unicorn

Once upon a time there was a unicorn. The unicorn was extremely lean and agile, and yet she was lonely. That is because she lived in the land of the horses.

Oddly enough, the horses were all kept in silos, rather than corrals or stables. When they wanted to communicate with each other, they just threw things over the walls of the silos. This also was odd, as, since silos have roofs, you would think the things they were throwing would just bounce back down on their heads.

In any case, the horses were let out occasionally during the day, but usually only got to graze on brownfields, and almost never got to see a nice greenfield. They ought to have been out playing freely in an ecosystem, but instead were trapped in an echo chamber. And so the lonely unicorn sat in its silo, day after day, dreaming of the greenfields.

So things went, until one day a magical chef, along with his equally magical puppet, and his servant, named Jenkins, arrived at the silos.

"Hello, lonely unicorn," the magical chef said, "would would you like to visit the beautiful greenfields, and graze on their very enabling feature sets?"

"Would I ever, " exclaimed the lonely unicorn, "but how could I do that?"

"Well," replied the magical chef, "all you need to do is to git into my machine here. It will open up seams in your silo, and provide you with silver bullets that you can turn in for free lunches and golden hammers in the greenfields."

"But I don't see any machine!" responded the lonely unicorn.

"Ah, that's because it is a virtual machine!" the magical chef replied. "It only exists if you are willing to completely buy in!"

"Okay, I buy in," said the lonely unicorn, as she stepped into the virtual machine.

But, to her great dismay, nothing happened. "What's wrong?" she asked, "I don't see the greenfields."

"Ah," replied the magic chef, "that is because your cost profile is still too high for you to fit through the seams. But inside the virtual machine is a magical container. It will containerize you, and reduce your cost profile enough that you can fit through one of the seams."

"Ah, now I get the kernel of the idea," said the lonely unicorn. She climbed into the container, and found that inside, there was a name space that was built especially for her. Her cost profile having shrunk by an order of magnitude, she could now be deployed to the greenfields, hundreds of times a day, if she wished, where she could frolic with the many other unicorns, and never, ever have to throw anything over a silo wall again!

And so she lived happily ever after.

Nothing to see here, move along!

The Garden in May


The relevance of van Bavel

As you may know, I am currently reviewing Bas van Bavel's The Invisible Hand? for History: Review of New Books. As I am reaching the end of the book, I am ready to write the introduction for my review!

Van Bavel's work might best be characterized as "applied history." (Students of Michael Oakeshott will recognize that this means van Bavel, while doing serious historical research, is primarily dealing with the "practical past," i.e., the past viewed as providing lessons for present choices.) The context in which this work is set is the ongoing debate over optimal economic policy. For a time, from the collapse of the Soviet Union until about a decade ago, it seemed that this debate might be settled: neoliberalism had triumphed, and the best political economy prescription clearly involved a heavy dose of free markets. Certainly, there was debate at the margins: Should healthcare be publicly provisioned? How big a welfare state should one have? What is the proper role for central banks and international economic institutions like the IMF? And there were always heretics like Marxists and Catholic social theorists who demurred from this consensus, but they were like flat earthers or creationists, and could be safely ignored.

But then came the financial crisis of 2007-2008, and all that had seemed settled was at play again. The response to the crisis by free-market advocates most typically ran along these lines: "Yes, the crisis was bad, but it was the result of crony capitalism, not of true free markets. If the central banks and the international economic institutions had not gotten in bed with the big banks, this all could have been avoided." And there is, I think, a good deal of truth in this response.

But van Bavel's work provides a very important counter-response. Van Bavel's main thesis is that open markets work well, for a time, at producing wealth and lifting all boats. But with the rise of factor markets, meaning market dominance over the allocation of land, labor, and capital, there arises an increasingly wealthy financial elite, and that wealth gives them an increasing influence over social conditions in general. In particular, this elite becomes more and more able to bend the legal system of their society to serve their own interests. In short, unfettered free markets produce crony capitalism as their probable (at times van Bavel seems to suggest inevitable) outcome, much as consuming crystal meth, while providing a boost for a time, sooner or later produces a collapse in health.

Van Bavel backs his thesis with extensive evidence from three case studies: Iraq between 500 and 1100 CE, Northern Italy from 1000 to 1500, and the Low Countries between 1100 and 1800. He also touches, much more lightly, upon other instances of market societies, such as England, the United States, early modern China, and the Roman Empire.

My next post on this topic will be the conclusion of my review.

The most rapid alteration of human behavior in history

And it seems nobody is paying it much attention. Well, of course not: they are too busy checking twitter!



My enthusiasm for DevOps

It might appear that I am simply latching on to a trendy topic.

But it actually goes a little deeper than that: the previous time I was involved in professional software engineering ended in about 2004. At the time, my friends and I had been pushing ideas like software as infrastructure, why one should prefer open-source software, and the advantages of text-based systems. But we faced a lot of opposition.

Fast-forward a dozen years: I dive back into the professional development world, and discover... we won! And the name of that victory is: DevOps.

Of course, like every other marketing term, "DevOps" will be over-hyped, and claims about its wonderfulness and ability to make babies' poop smell good need to be taken with a grain of salt. And, of course, in 2004 we hadn't yet forseen every aspect of the DevOps revolution: after all, a whole lot of smart people have devoted a whole lot of thought to this topic in the dozen years I was gone. And I am now in the process of enthusiastically absorbing the many great ideas they have added to what we knew in 2004. But these are great ideas built on top of the approach to developing and deploying software that we are ready knew was the best approach out there.

And I am far from alone in having experienced this: I have been listening to the DevOps Cafe podcast, and a number of guests have remarked that when they first encountered "DevOps," their imediate reaction was, "I've been advocating DevOps for years: I just didn't know that's what it was called."

What's more: the DevOps principles don't just apply to software development. You may not have noticed, but I've increasingly been "DevOpsing" my writing projects. For instance, as you know, I have been working on my review of The Invisible Hand? recently. In doing so, I have been creating "continually deployable" parts of my review: the blog posts you have seen here. Furthermore, I have been continually integrating those "gists" into my actual review.

DevOping the MS computer science curriculum

What I am going to say here applies mostly to training software engineers. Training theoretical computer scientists is a different matter, and my guess is that that is already being done pretty well.

The problem I perceive is that software engineers are being trained at universities by using methods more appropriate to training theoretical computer scientists: what's students get when they sign-up for an MS in computer science is the first portion of the curriculum used to train theoretical computer scientists for doing a PhD. Now, that knowledge is not useless to a working software engineer: all of the top working engineers have some understanding of theoretical computer science. Certainly, a good engineer should know what is being said when someone points out, "But your algorithm will run in exponential time," and have knowledge of how to determine the asymptomatic complexity of an algorithm they are considering.

But this is a minor part of most working engineers jobs. When presenting a proof for the runtime complexity of an algorithm in one of my lectures, I pointed out to my students, 99% of whom intend to be working engineers, and not theoreticians, that we engineers certainly should not be frightened of these proofs. But, I noted, they really won't play much of a role in your future jobs: in 20 years of professional software development, I told them, I had never once had my manager come to my office and demand a proof of the runtime complexity of the code I was developing.

To properly train of working engineer, one has to have them develop software, especially real software, that will actually be used by real users, rather than toy software developed simply to pass a class.

This implies that to properly train software engineers, MS programs must be able to offer them real projects to work on. But what kind of software for real-world use can an MS program possibly be involved in developing?

Answer: the program's own courseware! Of course.

So the new paradigm is: make your courseware open source. Make it buildable. Make it testable. Allow continuous integration and setup automated testing. Create monitoring systems that provide rapid feedback as to the success of the courseware. Put your courseware in an open source repository. But most of all:

Realize that, if you are training MS Computer Science students, you are a software company. Treat the entire enterprise as a single software project, developing agile courseware. Maximize the opportunites for sharing code and knowledge amongst all courses, and make the MS students an integral part of this process.


I'm number one!

A reader just wrote to tell me, "Your book Oakeshott on Rome in America has gone to number one at Amazon.com... in the category 'Books on Oakeshott, Rome, and America.'"

Crony Capitalism: The Free Market Cycle, Part II

"Also, in the course of the cycle described here, those groups and organizations in society who would aim for changing the arrangements of the market in order to balance or reduce negative externalities, gradually lost their economic and political power... Their revolts in the later stages of the cycle proved futile. This is also because of the consolidation and entrenchment of the elite in these later stages... Exactly in the last phases of the cycle the elites... closed their ranks...

"In all these cases, the state increasingly came under the influence of those who benefited most from the market system...

"In the first phase of the cycle, in each of these cases, the role of the state was not yet very prominent,  and it figured next to all kinds of other organizations and associations that fulfilled semi-public roles... In the second phase, states... increasingly stimulated the rise of markets in more direct ways, induced to do so because of fiscal reason. These markets, with the associated monetization and commodification of land and labour, enabled them to tap resources and tax transactions and wealth more easily than with other forms of exchange and allocation, such as barter or communal redistribution.

"Each of the cases discussed saw growing state repression, armed violence, and warfare by state and public authorities, in the last stage of the cycle. It is telling that this was done after the militias of ordinary people were replaced by professional soldiers or slaves, hired or bought in the market and being more dependent on their employers or masters than the independent producers in the former militias were."

-- Bas van Bavel, The Invisible Hand?, pp. 268-270

van Bavel on McCloskey

"I do not think the market fosters immorality of individuals... On the single point... I would agree with McCloskey that such a critique would be mistaken. However, this is not because capitalism will improve our ethics, as McCloskey argues, but rather because such a critique misses the crucial mechanism and the essential point. Even if the market itself is not anti-moral, and market behavior at the micro level is not immoral either, the outcome of market dominance at the macro level in the longer run will likely be a negative one, as shown in the cases investigated... This negative outcome is bound to occur particularly within a skewed social context..." -- Bas van Bavel, The Invisible Hand, p. 267

Microservices running on unikernels

That's the future of computing, my friend.

For example, that you were writing the weather prediction application, that needs to rapidly execute matrix operations on very large matrices. Your first thought may be, "Well, I find a good linear algebra library, and compile it into my program."

But there are several problems with this approach:

  1. Every time the producers of the library find a bug and fix it, you will have to recompile your program and redeploy it.
  2. Every time the producers of the library add a feature you want or improve the performance of the features you use, you will have to recompile your program and redeploy it.
  3. If there are security holes in the linear algebra library, they are now in your program also.
  4. Your program is now larger by the size of the linear algebra library.
  5. If there is specialized hardware for running these sorts of computations, you will need to make sure your program runs on it or forego the speed improvements it would provide.

A better solution is for your weather application to rely on a microservice. Some specialists in computational linear algebra will build the service, run it on some highly optimized hardware, and allow people to subscribe to it. When you need a large matrix multiplication, you will passe this service a message across the Internet, and it will do your computation for you, and bill you based upon the size of the computation. Every time the specialists fix a bug, you immediately have their bug fix. Every time they upgrade their hardware, you have upgraded your hardware! Every time they speed up the calculation, you have instant access to the faster version.

And these microservices will increasingly run on unikernels: they will be built by taking a library of operating system components, and compiling in just the minimum set of modules that will enable the service to run. So, for example, the linear algebra microservice does not need a graphical user interface. It does not need mouse drivers, or printer drivers, for display drivers. It does not need code for handling user logins or starting terminal sessions. It does not need process or memory management: it is built to run a single application. It probably does not even need a filesystem: just a way to access disk cache for intermediate results.

Tech-support running out the clock

"Please install the latest updates to your phone, watch, Apple TV, Mac Book, and thermostat before we go any further."