Do You Believe in (Coding) Magic?

February 25, 2007

This article reflects merely my own thoughts and opinions. I doubt it applies to you personally, so don’t think I’m proposing a new law.

Last night, as my LAN-party-attending friends and I drove home from the pizza place, I was asked to explain the concept of Turing-completeness. I managed to give a somewhat coherent overview of the topic without simultaneously befuddling my listeners, and gave examples of Turing-complete and non-complete languages. Listing off my favorite programming languages, all of which fell in the former category, I was struck by a realization, and drifted off into thought until awoken by the derision of my peers. :)

I appreciated the furor on reddit regarding my How to Beat Rails and Blocks != Functional Programming posts; debates between programmers are often informative and always entertaining. The contests between languages, though they rarely (if ever) change anyone’s mind, are fertile ground for inflammatory quotes and flame wars. But the realization I came to in the car yesterday was this – since almost all programming languages are Turing-complete, what’s the difference? At a fundamental level, they’re all the same, right?

Then why do I like Python and Ruby so much more than C or Java? Why, indeed, doesn’t everyone just write bare machine code? Am I wasting my time wondering about abstracts instead of actually coding? I pondered these questions through many hours of video games, and eventually came to a conclusion.

The largest factor in whether I like a language is its attitude towards “magical” behavior. “Magical” behavior is difficult to define – my best effort is ‘the ability to redefine the most fundamental aspects of the language’ – but here’s an example, taken straight from the perlvar man page:

$[
The index of the first element in an array, and of the first character in a substring. Default is 0, but you could theoretically set it to 1 to make Perl behave more like awk (or Fortran) when subscripting and when evaluating the index() and substr() functions.

Perl, which I loathe with every fiber of my being, condones – nay, encourages – the deep, evil, horrible black magic to find the quickest way around a problem. Ruby, which I like more and more with every passing day, allows for happy, Good-Witch-of-the-North-esque magic with metaprogramming, different eval() scopes and near-total introspection. Python, which I’ve loved since the seventh grade, really discourages the use of magic – custom infix operators are the extent to which Pythonistas have dabbled therein. However, it always tries to permit the clearest, most concise non-magical solution to a problem. Scheme, which I respect, isn’t so much a language as a wellspring of magic through which you can do whatever you want. In the right hands, it’s brilliant, and in the wrong ones impotent. Rails and Pylons reflect the different attitudes towards magic – Pylons requires you to render the templates explicitly, while Rails just evaluates instance variables and goes with the flow.

In summary, my preferred language depends on attitude towards magic – my favorite languages either allow for positive, as-readable-as-possible magic (Ruby) or disallow it altogether (Python.) Which is better? It depends on your situation – and on the necessity of good documentation. (Documentation, and therefore long-term maintainability, cannot coexist with magic.) Overall, though, I enjoy a little code magic – do you?

Entry Filed under: code, magic, pylons, rails, silly. .

13 Comments

  • 1. Michael  |  February 25, 2007 at 8:50 am

    Good stuff, Patrick! I’ve been looking for just the right words to explain why I don’t like to build anything new in perl.

    And now I’ve found those elusive words:

    “Perl, which I loathe with every fiber of my being, condones – nay, encourages – the deep, evil, horrible black magic to find the quickest way around a problem.”

    Schweet.

  • 2. yeep  |  February 25, 2007 at 2:01 pm

    not that i don’t like ruby, but i’d hardly call its metaprogramming as-readable-as-possible

  • 3. Me Notyou  |  February 25, 2007 at 3:22 pm

    Perl neither encourages or condones any behavior. It simply gets out of the programmer’s way and lets them program however they want. That is what it condones and encourages. If this level of freedom frightens – nay, confuses you – stick to ruby. Keep cranking out ultra-readable code that allows you to override the methods of a built-in type at run-time because that’s a Good Thing.

  • 4. Nimish Batra  |  February 25, 2007 at 5:53 pm

    Patrick…

    I may be completely wrong, but C++ does pure good “magic”

    They call it operator overloading.

    You can redefine most of the operators + – * / , (yes comma) *= += etc etc

    But there are strict typechecking rules.

    So you have to involve a CLASS or a USER DEFINED DATATYPE in there.

    You just can’t do this if the operator operates only on the predefined datatypes (int. float. char. double.)

    Type checking, I believe, gets rid of the evilness of “magic” in C++.

    I might be wrong, I haven’t done operator overloading in a year or so. But you can easily check it out.

  • 5. kardanok  |  February 26, 2007 at 1:23 am

    Great post, personally I’m a huge Python fan, though I’ve dabbled a bit in C++

  • 6. C. Alexander Leigh  |  February 26, 2007 at 3:58 am

    This is all well and good, as long as you are the only programmer on a project, ever… I’ve been programming for many years and when I go back to maintain something I wrote 5 years ago I might not even have used that language recently, let alone remember whatever strange alexgarian code conventions I may have used. Even worse when all this was someone elses problems – how many projects have we gotten as enterprise programmers where it was faster to just re-write than re-factor someone elses mess.

    Strong code conventions – that do generate more lines of code or at least longer winded code – are easy habits to learn until they become second nature, and I find writing readable code really isn’t any slower than writing mangled code. But the longterm payoff is definitly there, if only so when you get rich and hire other programmers you don’t have to explain what your code does.

  • 7. bernie  |  February 26, 2007 at 5:36 am

    Really check out erlang.org – best prog lang ever!

  • 8. Hsiao Wei Chen  |  February 26, 2007 at 7:51 am

    Are you really a high school student? you are such a genius. ‘coz even i, a second yr college student major in computer science student can’t explain turing-completeness on the fly.
    interesting post. But i’m still more of a java and C fan.

  • 9. rayke  |  February 26, 2007 at 8:12 am

    I’m just a random dude.

    But I am also a (random) fan of Ruby, I only wish more people knew how to do it. (People that I know, at least.) I always wanted to “collaborate” with some of my coding friends, but they stand by C++ and PHP.

    Never really tried anything with Perl. I guess I should give it a shot.

  • 10. jc  |  February 26, 2007 at 8:28 pm

    I believe Magic in languages is only good if its theres a developer community who embraces learning that magic as if it was part of the language. Its no longer magic, its “convention” or an “idiom”.

    When learning Ruby (a few months ago), this community acceptance of weird syntax blew me away. Their library of idioms is just part of the language, and once you pick them up, you can immediately recognize them. Almost like having language specific features for a bunch of common programming scenarios since they are instantly recognizable and easy to write (setting default values, checking for nil case, etc).

    In some other languages, its not a good idea to be too clever. In some recent C# code I wrote (inspired by ruby), I decided to use a delegate interface to my API in order to perform an action over a dynamic set of control instances, where my delegate was essentially the block. It was extremely elegant until someone had to make a modification and even though it was commented, another developer rewrote the entire thing in a for loop because he hadnt understood what the code was trying to do. If I had wrote the initial code that way from the beginning it would have saved us both a lot of time, with the only downside of it being 50% more code and me not feeling as elite while writing it.

  • 11. ThoughtEngineer  |  February 28, 2007 at 10:02 am

    “- since almost all programming languages are Turing-complete, what’s the difference? At a fundamental level, they’re all the same, right?

    Then why do I like Python and Ruby so much more than C or Java? Why, indeed, doesn’t everyone just write bare machine code?”

    Because writing machine code is a pain in the arse. Give me a ruby like assembly language that allows me magical control over my hardware, and I’ll send you my available ram when my system is idling.

  • 12. Michael Barry  |  March 7, 2007 at 2:33 pm

    I completely agree with your observations.

    Its simple. The more magical the language the more you can get done.
    Syntax and readability are overvalued. Explicit is not “always” better than implicit.
    Less is more….etc.

    People will argue that with too much abstraction you can create an unreadable mess. Ya, you always have the option of being stupid.
    (or just inexperienced)
    The difference is that if you “have” the power available and know how to create meaningful indirection you can save yourself an amazing amount of time.

    Other arguments are centered around
    readability of code. This is ridiculous. After a few years of real programming experience you realize the best code only has only been touched by a “single” mind.
    The more cooks in the kitchen the more of a mess the code becomes.
    Putting too many controlled processes in place to control the mess slows development down too much.

    Every developer in the small group knows the system they are developing inside and out. Therefore they know about their wacky meta code solutions.
    Smaller teams using more magic (meaningful indirection) can dramatically outproduce a larger team using less magic. They have too. (Its economically advantageous)

    In the past (say 20-30 years ago) the more magical languages were not as popular because of efficiency concerns. The popularity of more magic has increased over time because its practically possible.
    This trend is going to continue. (Its economically inevitable) If you have the power to do more with less….then you should do more with less or go extinct.

    I think the majority of new programmers are realizing this. This is great. This is going to push us to a new language that has more features from the functional programming paradigm. Functional programming is not a silver bullet but anytime you can avoid state in code the happier life is. Managing state can be like trying to solve a suduko puzzle. Too many dependencies. While some may think that is fun, its incredible waste of time that can be avoided. Save your brain power for meaningful problems.

    I’d use scheme more today if it
    had better online community support and libraries.

  • 13. Bruno  |  March 13, 2007 at 9:50 pm

    May I disagree on your assertion about Python “discouraging the use of magic” ?

    While it’s true that Python requires more explicit code, and favors readability (which is definitively not a “ridiculous” concern, n’en déplaise à Michael Barry), it doesn’t mean the language doesn’t have a great support for metaprogramming (and one that doesn’t rely on eval()).

    Have a look at the source of Ian Bicking’s FormEncode package – more specifically on the ‘declarative’ module – and you’ll find out that Python can get really magical.

    About template rendering in Pylons vs Rails: it should be quite easy to make Pylons behave the Rails way using the __after__ hook in the project’s base controller class.

    My 2 cents…


About Me



I'm Patrick Thomson. I'm a sophomore at George Washington University, passionate about technology, Apple, and programming in Cocoa, Python, Ruby, and Nu.

Why "important shock"? It's an anagram of my name.

Meta

Blogroll

a

Top Posts

Blog Stats