On Turning 30

While I shouldn’t care that I’m turning 30; I still feel depressed. Why would birthdays make me depressed? Every year at this time I end up in an existential crisis.

I used to think that I didn’t like birthdays because I don’t like being the center of attention, but that’s stupid, we all love and crave attention. If anything, I find myself getting annoyed that I have to entertain, laugh at everyone’s jokes, thank them for the thoughtless cards, and make them feel good at my expense. Yes, yes, I’m old, that’s hilarious.

The irony is, as I’m depressed I still smile and joke with everyone. I can’t help but to smile when smiled at, nod and laugh at the appropriate moments. Dance Monkey, DANCE! I guess it makes me feel better making someone laugh rather than cry.

The more I think about it, I guess birthdays are about everyone else. We’re all equally selfish, the poor bastard having the birthday usually does more work than the people supposedly celebrating the birthday. If anything, having a birthday gives you more responsibility to entertain friends and family.

But why all the depression?

I will admit, my reaction to birthdays has never been typical. I’ve noticed two distinct reactions in other people.

The first reaction is the imaginary todo list. Where you reflect on how many items you’ve checked off this list. Some people have “live in a foreign country” while others have “marriage and kids”. The principle is the same in all cases. It’s completely arbitrary, and no doubt will cause depression since turning 30 becomes a deadline where it’s not clear what you’re suppose to do afterwards.

The second reaction is “I don’t think it’s important”. I know a lot people who have this reaction, and I suspect most of them are full of shit. You’re getting older, it’s perfectly natural to reflect on your life and think about past accomplishments and mistakes. This is how we grow and learn. I would argue that your life isn’t worth living if you’re not taking the time to reflect and examine. Take it up with Socrates if you don’t agree.

All that said, my reaction to birthdays seems bizarre by comparison. I’m starting to fear I’m alone on this one. My reaction to every birthday is an unprovoked existential crisis. Normally I love reflecting on existence, pondering those seemingly unanswerable questions. But on a birthday it just hits me; as if some higher power decided that on your birthday you’re going to think about why you exist and what you’re suppose to be doing with your life.

So, every year, it’s time for the status report. Only I lost my orders, and have been playing video games instead. “Sorry boss, I have absolutely no idea what I’m suppose to be doing”.

Quite the perplexing state: we desire to know why we exist, knowing only that we do exist and with a feeling of self-importance. We can’t help ourselves but to be ignorant of why we exist while simultaneously overstating our importance to exist.

We assume our lives are important even though we don’t know why.

In all likelihood my life isn’t that important, but believing that is contrary to the human condition. It’s like Nihilism, even with a good argument, it’s so contrary to the human condition that the only people who believe in Nihilism are either being cynical or they do so with a Pandoras Box approach (where Nihilism is the path that breaks down the interpretations of the world that prevent us from understanding our right course). Nihilism tends to contradict because of this instinctive idea that our existance is important; that there is a right path for humanity. It seems that our existance is reason enough to believe in this importance.

Most of the year it is. Most of the year my existance is reason enough to assume that my life is important and that through intuition I can understand what I ought to be doing.

It’s different around my birthday. “Hey, you’re 30”. I laugh, I make jokes. I get depressed. I think about why I’m depressed. On my birthday, the fact I exist doesn’t provide me enough reason to believe that my life is important. Depressing, but confusing too, since the rest of the year is different.

Normally I trust my intuition, faith in myself if you will, that the only way to overcome not knowing why you exist is to trust that you exist for a reason, that all of those crazy emotions are key. We are guided, without reason, by an intuitive sense of purpose, a need to belong and do what is ultimately right. Slaves to our own intuition, fed by the instinctive desire to feel important, to feel needed.

See what I mean?! Every year is like this, some worse than others. It’s the weirdest thing… you’d think I’d just write a list like everyone else.

Making Java less Difficult

I’m sure that you’ve heard this before, but I’ll say it anyway: Why do Java developers make simple problems difficult? There are plenty of people who have written on that topic, and I have no desire to further expound the point. I think we all get it. What I do want to talk about is how to make things less difficult, in particular how to make web application development as easy in Java as it is in Perl, PHP, Python, or Ruby.

I challenge that the answer is not to build Yet Another Crappy Web Framework ™. I for one, am terribly sick of complicated frameworks that are more work to configure than a web application is to create manually.

What I’ve noticed, is that developing in Perl is often faster than Java mostly due to the built-in regular expression handling Perl offers; and this is despite the fact Java offers a comparable regular expression engine.

Two things that slow me down as developer:

1) Staring at code that involves a StringTokenizer, while loops, and several nested substring statements where I’m counting characters on my figners. Why do Java programmers continue to write code like this? I suspect the reason is because of the next issue.

2) Wanting to use a regular expression, you first create a Pattern object, followed by a Matcher object, remember to compile your pattern, and then finally match the pattern using the Matcher object. Your matched object can use the group method, which is implemented from the MatchResult interface. I’m going to repeat myself, as this bears repeating, why do Java programmers make simple problems difficult?

These two issues cover 90% of the cause for Java development to be slower than Perl. Solving these two problems should then speed up my development in Java to be on par with my development in Perl!

Let’s look at some sample Perl code:

if (/a(.*)b/) {
  print "$1 is between a and b";
}

We could do this fairly easily in Java, but let’s be honest, the real strength of Perl is when I start doing things like:

 @bar = split(/:/, $_);
 @foo = grep(/^#/, @bar);
 print join(':', @foo)

While that example is simple to any Perl programmer, it is, admittedly less intuitive and less clean than a Java version. Although it is only three lines rather than the twenty or more it would take in Java. But what about something like this:

XYZ bar = XYZ.split(":", arg);
XYZ foo = bar.match("^#");
System.out.print( foo.join(":") );

This seems easy, and perhaps clean enough not to bewilder the Java developer the way Perl code tends to. Perhaps we can create a utility Java class that supports the needed data structure and methods to write code like that. What other methods would we need to empower Java developers to stop writing the usual tokenizer/substring mess? Consider just the basics:

  join - join the values of an array into one string
  split - split a string into parts
  match - match the parts
  matchAll - match repeated patterns
  trim - trim the ends of a string

This much is already provided in Java in one class or another. The String class already lets me match and split, but it’s not enough. I need more than just a String class, I want an array of Strings. No, I want a dynamically sized array of Strings that I can easily get and put from. While I’m at it, I want it to be associative array, where I can treat it like a normal array and it will retain index order, or I can add my own keys (similar to PHP arrays). Given a data structure like that, add in the above methods to that data structure, and I think we’d be in business.

The scripting languages provide a default context that includes regular expressions, hashtables, dynamically sized arrays and various string manipulation features. I can do all of that in Java, but it’s not conveniant creating a different object for multiple data structures and methods, especially since they’re almost always part of the same context.

Let’s create one class that contains the data structure and the methods. This should solve both of my issues, making it easy to do what once was difficult and speeding up development for most all applications.

Even better, we can leverage what Java is good at to get this done. Let’s extend a Java Hashtable which solves most of the data structure part.

public class Preg extends Hashtable {

 public Object get( int inInt ) throws NullPointerException {
  Integer cast = new Integer(inInt);
  return this.get(cast);
 }

 public void put( Object value ) {
  Integer key = new Integer( this.size() );
  this.put( key, value);
 }
}

I’ve added the ability to put values without providing a key, this will allow us to use this object as both a HashTable and a Vector (i.e. a resizable array). For those of you who are upset that this is inefficient, you’re absolutely right. Although I would argue that it’s a negligible difference. But since we’re speeding up development, this will more than compensate since I’ll actually have time to optimize my code and do some refactoring before the deadline (a luxury I rarely see Java developers having time for).

The next step is to create the methods, we should also provide static versions of the methods to gain maximum convenience. For example, I may only implement one “join” method, but I’ll provide several prototypes:

 public String join() {
  return Preg.join("", this);
 }
 public String join( String glue ) {
  return Preg.join(glue, this);
 }
 public static String join( Preg src ) {
  return Preg.join("", src);
 }
 public static String join( String glue, Preg src ) {
  ... the actual implementation
 }

After I do this for all of the methods, I’m left with a very powerful object that allows me to accomplish the previous Perl example:

Preg bar = Preg.split(":", arg);
Preg foo = bar.match("^#");
System.out.print( foo.join(":") );

Three lines of Java that for once is equal to three lines of Perl!

All the fancy trickery that we do in Perl or PHP can be done in Java with just the right mashup of HashTable, Pattern, and Matcher. You can split a string into parts, filter, trim, and arrange to your hearts desire without ever having to count a character offset for a substring!!


Note: I have all of this working for one of my projects, which will hopefully move as fast as it would in Perl or PHP (it’s a web application, so I suspect this will in fact compensate for that 90% slowdown I mentioned earlier). If there’s interest, I can provide the source and/or Javadoc.

Even with my extremely verbose Javadoc it’s surprisingly not that much code.

New Website

When I posted my article on creating a RESTful SOA it occurred to me that my own website was badly abusing every architectural direction that I was advocating. I decided to put my architecture to the test by creating a RESTful SOA for my own website. While I’ve recently applied the RESTful SOA to enterprise web applications I’ve never applied it to something so small.

So, I created a FrontController, ResourceLookup, and a full MVC utilizing a RESTful SOA written entirely in PHP and all for this one site! I even decided to keep the code backwards compatible with PHP4, so I didn’t get to use the cool reflection capability in PHP5, but I wanted to prove that I can still apply proper SOA fundamentals to PHP4.

I kept things simple and even managed to remove all dependencies to MySQL (which my previous site utilized). I only have three types of resources so it was relatively easy coding.

The end result: everything you’re seeing here can be represented as a web page, or as a service to be manipulated using HTTP methods, or as a news feed. I also added a text representation, which doesn’t serve much of a point except for demonstration.

The framework only took me a few hours to create, which was great, as it proves that a RESTful SOA is a flexible architecture that applies to web applications small and large. The problem was, I spent an entire weekend editing stylesheets, creating a new look and feel, and basically redesigning the entire site! That last part took longer than I wanted, and I’ve still got some bugs to work out.

Creating a RESTful SOA

I’m going to keep this discussion purposely language agnostic, however, I currently have two projects where I’m implementing a complete RESTful SOA (Service Oriented Architecture). One in J2EE, with a Model 2 architecture using Servlets, JSP, Oracle, and XSLT for the presentation-layer transforms. The other project is LAMP-based (Linux, Apache, MySQL, and PHP) using mod_rewrite and a custom front-controller.

It’s worth pointing out that a service-oriented design is different from object oriented design; and further, these two concepts are orthogonal to RESTful web applications. The PHP application I’m working on is not object-oriented (OO) while the J2EE project is rigorously OO. Both projects follow the same RESTful SOA pattern that I am presenting here.

Both projects have elements of aspect-oriented design (AOD), such as the authorization and logging aspects. I’m going to avoid this topic, and instead focus on the specifics of a RESTful SOA, but if you’re interested it’s worth understanding AOD as a way to compliment the traditional object-oriented design principles that don’t often lend themselves to things like REST.

On with the architecture!

Context of Use

The first step in building a RESTful SOA is to understand, and even better, document your context of use. An easy way to do this is to create a spreadsheet with three columns: Roles, Environment, and Goals. Create a list answering the following questions:

Roles: Who is going to use this site?
Environment: Where are they working?
Goals: What are they trying to do?

This is a simple exercise that is often overlooked. Ignoring this will just give you more work and a crappy product. The beauty of a RESTful SOA is that it provides resources that map to the different types of users and their goals. Don’t add anything beyond what your users need to do. You’ll find yourself making simple pages that provide a positive user experience.

Resource List

Walk through your context of use and create a list of content categories that meet your users goals. Even if you’re dealing with advanced applications, don’t complicate this process, tackle each of the users goals separately with different content sections.

Create a simple file hierarchy representation of your content, mapping closesly to your users and their goals. Here’s an example snippet:

/
/support
/admin
/it
/support/cases
/support/cases/num *
/support/request
/support/metrics

The REST is easy

You’ll find that in this process you’ve listed all of the resources that are needed to fulfil the needs of your users. Don’t invest your time into a CMS or a portal, the next step is to build your own application. You’ll find that a RESTful SOA will be quicker to build than configuring “Hello World” in a jsr-168 portlet!

Front Controller

This is critical, avoid JSF, and stop inserting random code into your html via JSP or PHP or whatever. You’ll want one small program that intercepts all traffic. Every language has a different way of doing this, whether you have to use mod_rewrite or servlet-mapping I don’t care; but funnel everything into one program and grab the path:

  class FrontController {
    path = getPathInfo();
    ...
  }

The Method Matters

The beauty of REST is that we leverage HTTP, URI, and our own web-server to do most of our work. Your front controller needs to process each RESTful HTTP method (GET, POST, PUT, DELETE) separately. Also make sure a request object gets passed in (query string, post variables, cookies, etc). You should have something like this:

  class FrontController {
    path = getPathInfo();
    ...

    function handleGet(path, request) {
      ...
    }

    function handlePost(path, request) {
      ...
    }

    function handlePut(path, request) {
      ...
    }

    function handleDelete(path, request) {
      ...
    }

  }

KISS your CRUD Goodbye!

I should say CRUDL (Create, Read, Update, Delete, List), but that didn’t sound as cool. The next step is to parse your path, this varies from one application to another, but generally your path will contain what I like to call a “resource” followed by “input”. For example:

/support/cases/50

50 is not a resource! However, /support/cases most certainly is. Also, if the user goes to /support/cases, I should be showing them a list of cases rather than details about a specific case.

The simple way to solve this is that when you process your GET request and you haven’t been given any “input”, just a resource name, you’ll want to call the “List” method rather than “Read”. This keeps it simple and very, very RESTful.

Put the DAO to REST!

The astute reader may have noticed I glanced over how you’re suppose to derive a resource and an input from a sigle path. Well, REST assured, this is trivial. Create a simple data model with a Resource entity that maps to itself (one-to-many) to allow for multiple children resources. So, “/support/cases/50” would involve two entries:

+-------------------+
+ NAME    | PARENT  |
+-------------------+
+ support | NULL    |
+ cases   | support |
+-------------------+

Obviously, you’ll want to use unique numeric IDs so you don’t have name collisions, but you get the idea. This will allow you to lookup unique resources and an optional input parameter. The “resource” entity should also contain things like “Title”.

The children resources represent the sub-elements that you may want to appear on a menu navigation. So you’ll want to keep these handy also.

Finally, you now have a unique resource, with optional input and one of four HTTP methods. This all maps nicely into one pretty package, which should be something like:

  class RESTController {
    path = getPathInfo();
    resource = getResource( path );
    input = getPathInput( path );

    function handleGet(resource, input, request) {
      if ( empty(input) )
        resource.list( request );
      else
        resource.read( input, request );
    }

    function handlePost(resource, input, request) {
      resource.update(input, request);
    }

    function handlePut(resource, input, request) {
      resource.create(input, request);
    }

    function handleDelete(resource, input, request) {
      resource.delete(input, request);
    }

  }

Presentation and RESTful Content

Probably one of the biggest architectural mistakes you’ll find with web applications is an incorrect MVC abstraction. It’s easy to say “Model, View, and Controller”, but rarely is this done right. Typically the architecture looks pretty, but on close strutiny calls for tightly coupled templates with functions that belong in the model. Do yourself a favor, don’t let your view talk to your model.

You’re already using XML

Whether you like it or not, you’re on the web, you’re producing html for your content and hopefully keeping your presentation in css. The great thing about XML isn’t that it’s easy to parse or to read. I hate XML, it’s ugly, stupid, and slow to parse. What it is good at is transforming from one schema to another (via things like XSLT).

When you ask your model for a “read” or a “list” operation, you should be getting back pure conent. Content in the form of XML. This is content without a presentation, and it belongs in your model not in your view!

Putting the Representational into REST

If your model is dumping out content in the form of XML, the only thing left to do is for your fancy front controller to transform this XML into whatever representation that the user asked for! It’s a good idea to make the default representation html. The code should now be looking like this:

  class RESTController {
    path = getPathInfo();
    resource = getResource( path );
    input = getPathInput( path );
    view = new ViewDispatcher( request.getType() );

    function handleGet(resource, input, request) {
      if ( empty(input) )
        model = resource.list( request );
      else
        model = resource.read( input, request );

      output = view.transform( model );
      print output;
    }

    ...

That’s it, you can take your presentation-less content and transform it into anything you want (html+css, flash, pdf, plain text). You could even make it a SOAP envelope, autogenerate a WSDL and now you have REST and SOAP without doing any extra work!

So there you have it, this is the basis for a RESTful SOA, that if done right, will allow you to build really cool web applications. Since it’s RESTful, every resource is accessible in a browser, or as a service to an API, or whatever REpresentation you want!

Funny thing, it seems that the whole point of REST is to do MVC the right way, and de-couple everything. I’m not sure why all the big CMSs and portals claim to follow an MVC (or Model 2) architecture but fail to provide a decoupled view.

Why aren’t you working on important problems?

I just finished reading the transcripts from Richard W. Hamming’s speech ‘You are Your Research’.

“If what you’re doing is not important… why are you working on it?” he asked.

In context of the latest GTD (Getting Things Done) craze, this is a critical question. A question that should keep you up at night.

As you’re processing your inbox, tracking your next action for each project, ask yourself: What are you working on, and why?

Do your actions connect, not just to an arbitary project, but to a meaningful goal, to the greater good of your life?

The first step in managing your life, in meaningfully getting things done, should be to follow Richard Hamming’s advice and ask yourself:

What are the most important problems in your field?

Put another way, what are the important problems that you could be working on? And why aren’t you working on them?

Stepford Geeks

One of the things that makes us geeks is that we used to have too much time on our hands. We developed technical and creative skills often times out of boredom. Alone, in a dimly lit room – we made web pages, wardialed, operated a BBS, wrote an OS, designed a computer, and so on through the history of geekdom.

Now we have no time. Those geek skills have given us cool jobs. We’re no longer bored, we’re creating a new world and there’s just never enough time to do all of the things that we really want to do. The latest trend is to use our geek skills to hack at our life so that we can get more work done.

But what if this is wrong? I’m also a movie geek, so let me put it likes this:

“When you come back, there will be a woman with my name and my face, she’ll cook and clean like crazy, but she won’t take pictures and SHE WON’T BE ME!” Stepford Wives

I need to make sense of this. Let me take a step back.

This week, I started my new job at Qualcomm. For an email client, I got to choose between Eudora and Outlook. I thought to myself, did I just go back in time… people still use this crap?

I almost forgot what old mail clients were like: New messages don’t reference the conversation thread from the archive. Wait, there is no archive. But there is a 500 MB quota; meanwhile my free gmail account just hit 2700 MB.

I’ve spent so much time life hacking I forgot how complicated email can be with the wrong tools.

Like any life hacker with proper geek skills, I designed a simple system that minimizes my reliance on Outlook, by dumping email into a private Subversion controlled directory that also works as my personal file space that can be shared/searched across multiple computers. Hooray, geek skills to the rescue!

In this process, I started looking through some of the old files that I’ve kept around. You know, my old web pages from 1995. Oh and a recursive decent parser for Firewall-1 I wrote for no reason except to figure out how to do a recursive decent parser for Firewall-1. I also found some “experiments” in Photoshop and 3DSMax. Really terrible stuff. My parser was horribly slow and looking at it now, it’s some of the worst code I’ve ever seen.

But this is the byproduct of my bored youth. I used to create web pages for fun. I learned Photoshop for fun. I learned HTML/CSS/Javascript for fun. I learned to program and make cool Java applets for fun. I learned how to manipulate data with Perl for fun. The list goes on and on to include real-time environmental sensors, databases, data modeling, but it also includes photography, graphic design, comic books, movies, writing, and even television. Basically, things that geeks get excited about!

Now that I’m out of time, I wonder if I’m using my geek “life hacking” skills to destroy the part of me that makes me a geek? I get excited about odd things because I’m bored with my life, but if my life hacking makes me so good at life that life ceases being boring… then what?

When you come back, there will be a geek with my name and my face, he’ll design and code like crazy, but he won’t take pictures and HE WON’T BE ME!

sigh…

I guess we should remember that if we’re going to life hack, we must always remain oddly discontented with our lives no matter how good at life we get. We should remember to hold such unrealistic and fanciful ideals for ourselves that our daily world is dull by comparison.

Good Design, Bad Code

I’ve been reading PHP 5 Objects, Patterns, and Practice, a great book exploring modern day design practices and the conventional wisdom of good software design. For serious PHP developers it’s a great update to the “gang of four” in a PHP context and really shows how PHP stands up as a serious development environment for the enterprise.

Unfortunately, all of this presents some serious misgivings about conventional software engineering methods that this book serendipitously stumbles upon. One of the code excerpts reads:

  class Army extends Unit {
    private $units = array();

    function addUnit( Unit $unit ) {
      foreach ( $this->units as $thisunit ) {
        if ( $unit === $thisunit ) {
          return;
        }
      }
      $this->units[] = $unit;
    }
   ...

I’m sure to the hot shit J2EE/EJB software architect this is perfectly good code in the context of a composite pattern. The above case represents a composite class that holds Unit objects, and the addUnit method simply adds Unit objects to the composite. The great part is this class will rarely, if ever, need to be updated since the business logic is separated into individual leaf classes and will be easier to maintain. Java programmers would be proud to see PHP like this.

On the other hand, every 15 year old hacker will now have something in common with old-school C programmers. They will recognize that this code is shit. It’s utter crap. It’s inefficient, sloppy, and scales poorly while being difficult to debug.

Have we as software engineers become so pig-headed that we no longer care about code efficiency and happily let a big-O(1) operation become a big-O(N) operation just because it uses the right pattern? When did software engineering mean you have to be stupid? Can’t I be a software engineer that follows good, disciplined, development methodologies while still favoring good code? We’re assuming that the intelligence of our design patterns will make up for the bad code.

For the J2EE crowd, PHP arrays are all associative, similar to a Vector class. In this case you’re adding individual leaf objects to a composite class; and every time you do you’re looping through an associative array to determine if the object already exists in the array. Each time you build this composite, which in a web environment is every page load, you’re doing a mountain of extra work. This is great for a small number of objects, but what if you have thousands, or even millions of objects at hand. This is what scalability is all about, right? When did our design patterns make us stupid?

The goal of software engineering is to produce good software, good software requires good code. Disciplined software engineering may have something to learn from the hackers who would recognize the bad code excerpts in this book.

Leaving Academia!

I resigned today. Quit my job. After several years I have decided to leave the University and head to the private sector. Before I leave, I want to remember the wisdom of Dr Ray Stantz:

“Personally, I liked the university. They gave us money and facilities, we didn’t have to produce anything! You’ve never been out of college! You don’t know what it’s like out there! I’ve *worked* in the private sector. They expect *results*.”

While at the University I have had the pleasure of working with some of the finest minds around. I have collected real-time coastal data at Scripps Institution of Oceanography. I have built distributed data grids for the Biomedical Informatics Research Network. I have worked on the Storage Resource Broker at the San Diego Supercomputer Center. I have architected data repositories and web applications for the Network for Earthquake Engineering and Simulation.

There is a culture within those ivory towers that I will miss. A spirit of intellectual pursuit that is a part of my very being. I am at heart an engineer as well as an academic. I am uncertain of what awaits me, but curiosity killed this cat.

I began a job search within the last couple of weeks. The initial response was tremendous! Since I completely missed the .COM era I was unprepared for the barrage of offers and interviews. In the past, I was lucky to get one interview. This time around I had a half dozen offers. I ended up creating a decision matrix with weighted discriminators to help me determine the best position.

In the end, I have accepted a fantastic opportunity with Qualcomm. I will be working with the MediaFLO group architecting the IT infrastructure, business intelligence, and data warehouse. Of course, as Ray Stantz reminds us: “they expect results”!

Balut!

Eating four hard-boiled fertilized duck eggs followed by a bottle of beer isn’t as bad as you’d think. Never mind the four dead baby ducks, four hard-boiled eggs is gross even without duck fetus!

I took tons of pictures while in the Philippines and also in Hong Kong, more on this later, but for now, please enjoy the galleries!

Centrifuge Earthquake Modelling

Visited Troy, NY (near Albany) for the NEES/UCDavis/RPI Centrifuge Workshop. Learned how to use the RPI centrifuge for earthquake engineering research, and learned more about data acquisition and sensors (LVDTs, strain gauges, pore water pressure sensors, etc) than I’ll ever remember.

Centrifuge Panoramicview pictures