• Insta-Declarative DSLs!

    Recently I’ve focused on retaking rules for developers, where rules aim to be a developer-friendly way to untagle complex logic. Yet some problems call for policy changes without the involvement of developers. We need a simple way to write simple rules.

  • Clara Rules Site

    Anyone looking for information related to Clara should check out the new clara-rules.org site. Complete developer documentation and resources are there.

  • Micro Benchmarks versus Macro Optimizations in Clojure

    Here’s a simple question Clojure users hear often:

    What is the overhead of Clojure’s persistent data structures?

    I ran into this question headlong when profiling and tuning Clara. Clara aims to draw ideas from expert systems into the Clojure ecosystem, making them available as first-class Clojure idioms. Therefore Clara’s working memory works like other Clojure data structures: it is immutable and “updates” create a new, immutable working memory.

  • Clara 0.5 Released

    Clara 0.5 has been released! This is mostly a big fix release with some usability improvements. See the changelog and the github page for usage information.

  • Clara 0.4 Released

    Clara 0.4 has been released, and is available on Clojars! See the github page for usage information. I wrote about the significant features in the Rules as Data post.

  • Rules as Data

    This started as a topic for a Lambda Lounge meetup but seems worth sharing broadly.
    I’ve posted previously about treating rules as a control structure, but for Clara 0.4 rules are now first-class data structures with well-defined schema.  The simple defrule syntax is preserved but the macro now produces a data structure that is used everywhere else. For example, the following code:

  • Crossing the (data) streams: scalable realtime processing with rules

    Pipe-and-filter architectures are among the most successful design patterns ever. They dominate data ingestion and processing today, and give 1970's hackers yet another chance to remind us how they thought of everything years ago.

    Unfortunately modern variations of this can run into an ugly problem: what are the semantics of a "join" operation between multiple infinite streams of data? Popular systems like Storm point out this ambiguity, as does Samza. Both provide primitives to support correlation of events between streams, but the higher-level semantics of a join are punted to the user.

  • Rules as a Control Structure

    Rule engines seem to draw love-or-hate reactions. On one hand they offer a simple way to manage lots of arbitrary, complex, frequently-changing business logic. On the other, their simplicity often comes with limitations, and edge cases pop up that can’t be elegantly solved in the confines of rules. There are few things more frustrating than a tool meant to help you solve problems actually creates them.

  • A long time ago, we used to be friends

    No one ever intends to let their blogs go dark, but here we are.  I'm planning on dusting off this blog to write about some new personal projects, but for now I thought I'd link to writing I've done in other venues:

  • Start by embracing your limits

    Nothing in human history has offered more promise but has seen as many failures as software. We’ve all seen moments of greatness, where a program seems like magic -- but such gems are surrounded by minefields of bugs and indecipherable interfaces.
  • Can "Agile in the Large" Succeed?

    Agile Development isn’t perfect, but it got something right. Right enough, at least, to gain enough momentum to become a buzzword -- and for consultants to latch on by selling “Agile” to big enterprises. The result is “Agile in the Large”.

    Agile in the Large ranges from the Half-Arsed Agile Manifesto to Cargo Cult practices. Too often we simply bolt on a few agile practices without understanding how they provide value. Adding scrums and iteration reviews to your current process and expecting improvement is like building a runway in your back yard and expecting planes to land.

  • Beware of the Flying Car

    Every so often we developers are asked to build a flying car. Our users obviously need one to avoid traffic and get to meetings on time. So we set down the path to meet those needs.
  • And We're Back

    After an unbelievably long hiatus, I'm going to start blogging again. It's funny how becoming a parent makes everything else seem to go away for a while.

  • The Guru Myth

    Anyone who has worked in software long enough has heard questions like this:

    I'm getting exception XYZ. Do you know what the problem is?

    The questioner didn't bother to include a stack trace, an error log, or any context leading to the problem. He or she seems to think you operate on a different plane, that solutions appear to you without analysis based on evidence. This person thinks you are a guru.

  • Tearing Down the Software Factory

    Tom DeMarco said it well:

    The idea of a software factory is a joke -- that we can build software by rote -- that's ridiculous. If the work is deterministic, we will do with it what we do with any other big piece of deterministic work. We'll let the computer do the deterministic portion, leaving the person who interacts with the computer -- the other half of the system -- to do the work whose roteness has decreased, not increased. Every time you automate something, what's left of the person's work is less deterministic, until eventually, when you automate enough, there's no deterministic element left for the person's work--no rote. ... Our work is not deterministic. It's far too inventive. We're knowledge workers, not factory workers.
  • End of software's dark age?

    Software has held more promise and yet met with more failure than any other technology. Meanwhile the hardware guys have been merrily creating more and more CPU cycles for us to swamp.
  • What we can learn from databases

    While not perfect, relational databases are among the most successful software ever used. Their behavior is easily understood even under high concurrency. Sets of changes can be composed with almost no effort, with strong guarantees of consistency. The question is: would this be true if we built our databases the same way we build applications?

  • The Design Crisis

    Imagine waking up tomorrow and learning the construction industry has made the breakthrough of the century. Millions of cheap, incredibly fast robots can now fabricate materials out of thin air, have a near-zero power cost, and can repair themselves. And it gets better: given an unambiguous blueprint for a construction project, the robots can build it without human intervention, all at negligible cost.

  • von Neumann's Long Farewell

    A big step in solving the software crisis is to accept that programming is not about computers. Yes, it used to be, and still is for brave souls working on operating systems or compilers. But in the long list of the world's software struggles, compilers rank pretty low.

  • Why are we so polarized about Java?

    It seems impossible to constructively praise or criticize Java without eliciting strong reactions. It comes from both sides; some seem to love to hate Java, and parts of the Java community become defensive as a reaction.

  • Why Java lost its mojo, and what Sun is doing about it

    It's old news to some, and hotly debated by others, but Java has lost its mojo. The Java community still has growth from large companies, but it hasn't been the darling language of hackers for a while (if it ever was). But don't think Sun isn't doing anything about it.

  • The Imaginary Concurrency Debate

    It's finally being accepted: the shared memory, critical region model for concurrent programming is broken. Even good programmers slip up and introduce race conditions, and bad programmers don't even understand the semantics. (You mean I have to lock an object just to read its values?)

  • Imperfect Approximation of Perfect Code

    It needs to be said: the quality of a programming language is not inversely proportional to the number of bytes needed for a Sudoku solver. Not that there's anything wrong with comparing Sudoku solvers; they can contrast certain aspects of languages, and differing solution styles can help readers think differently about code. But this only gets us so far.

  • Part 4: The Killer App

    Technologies get adopted when there is a compelling reason. People bought PCs for the killer app; game consoles sold for the killer game. Programming languages take off because they have a killer feature. Philip Wadler pointed this out nine years ago:

    Instead, experience shows that users will be drawn to a language if it lets them conveniently do something that otherwise is difficult to achieve. Like other new technologies, functional languages must seek their killer app.

  • Part 3: Building a Firewall Against Complexity

    In the first two posts in this sequence, I argued a few key points:

    • A language must be usable by mediocre developers to achieve widespread use.
    • A language that prevents quick and dirty application development will not be widely adopted.
    • Widespread adoption is important so good programmers can use better languages in their day jobs.
    • Haskell shows important innovation in language design, but has major hurdles to widespread adoption.
  • Part 2: Languages and the Lesser-Skilled Developer

    My last post ended with an assertion: we need a language with the best of Haskell in a more accessible form. This leads to an old debate. After all, building good software requires tremendous skill. How can we make progress if we tailor it to the weakest practitioners? Bjarne Stroustrup made this point well:

    The idea of programming as a semiskilled task, practiced by people with a few months' training, is dangerous. We wouldn't tolerate plumbers or accountants that poorly educated. We don't have as an aim that architecture (of buildings) and engineering (of bridges and trains) should become more accessible to people with progressively less training. Indeed, one serious problem is that currently, too many software developers are undereducated and undertrained.
    Stroupstrup is right, of course. Too many developers are undereducated; we should demand greater skill. However, this doesn't change the fact that language adoption for a given shop has as much to do with politics as technical merit. Businesses have a perceived need to hire armies of developers to quickly write bad code and bad software. We'll be asking "WTF?" for a long time to come.

  • Part 1: Programming and the Metaphorical Mind

    Hello, World
    This is the first of a set of posts on what we need from a new programming language. Everything I discuss exists in some language -- at least in part -- but no language yet has them all. This content is a mixture of my experience and many borrowed ideas, which I will do my best to cite. My goal is to represent the language needs of some segment of developers.

  • The Software Engineer's New Clothes

    The problem with experts is they're too smart. The problem with the rest of us is we don't have the expertise to reign in the experts.

    Consider the following conversation:

    Mortal Developer: This system seems complicated. Can we do without the WhackAMole feature?
    Expert: No, some clients might need WhackAMole to interact with their legacy abacus.

    Now our poor mortal developer is stumped. After all, this is the expert we're talking to here -- and if he says we need WhackAMole you better get whacking.

  • The real reason we need strongly-typed languages

    There aren't many hard-and-fast facts about software, but here's one:

    The more code you have, the more bugs you have.

    More formally, lines of code is a strong predictor of defects. So, any language that allows you to do the same work with less code must be a Good Thing. This is why Python is better than Java is better than C++ is better than assembly, right?