21 December 2009

Building Mup on Ubuntu Jaunty 9.04

I really like the Mup music typesetting program. It had been a while since I had needed to do some music, and I had never installed it on Linux before, so I thought I would give it a go.

It was harder than I thought, but I got it to work. Here is what it took:
  • download and extract the source tarball
  • install the following packages: gv, playmidi, libx11-dev, libfltk1.1-dev, libxpm-dev, libxext-dev
  • change the makefile to link in some different libs, here is what I got to work:
# You can use fltk_jpeg or jpeg library, whichever you have
JPEGLIB = fltk_images

# You can use fltk_png or png library, whichever you have
PNGLIB = fltk_images

# You can use fltk_z or z library, whichever you have
ZLIB = z
  • sudo make install
Now mup works, and the new mupmate and mupdisp, too.

The license code goes in a single-line text file: ~/.mup -- similar to the DOS mup.ok contents.

06 October 2009

Starting a box with git

In The Creative Habit Twyla Tharp talks about "starting a box" to keep track of all the creative ideas that come on a project.

I've used git to just start tracking ideas in digital form. A brand new git repo is my digital box. Usually, the ideas start out as a whiteboard diagram or scratching on paper, which I then scan to PDF and put in the folder. The organization varies, but chronological file naming has worked for me, as has thematic subfolder organization.

Being able to just capture the work frees me up to try new things and not worry about whether that stuff I just did is safe or whether I can get back to it if I try something new out.

Git does such great cross-file compression that even after several revisions across binary files, the resulting repo sizes are surprisingly small.

04 September 2009

An entire sprint without an SVN branch

A typical development flow has been:
  • request an SVN task branch (wait for CM to help)
  • do stuff (1-3 weeks)
  • run all the regression tests (not on the latest stable)
  • hope that nobody did anything that conflicted with us
  • merge to the pending release (and pray we resolve any conflicts correctly)
Now we do the following:
  • set up a team git repo (able to do this without CM's help)
  • clone off a repo for each team member
  • set up a daily cron job to get the latest stable from the pending release
  • set up a daily hudson build to merge that in and push to team if it passes unit tests
  • pull from team as we go (fast!)
  • run all the regression tests (on the latest stable)
  • use git to merge on top of the pending release and commit back to SVN (with minimal conflict windows)
Well, we finally got through a whole sprint without an SVN task branch. Maybe that seems like a small victory, but it saved us a bunch of late-breaking integration risk.

03 September 2009

Exchanging gravitons with the monolith

When I was telling my colleague about my plans to aggressively modularize the monolith, he said:
You're exchanging gravitons with the monolith.
And he was right. The effect of the warped space-time in the close vicinity of the monolith was such that I lost sight of the new work we had recently embarked on. The trajectory I was on got skewed by the gravitational pull.

It is not sufficient to be doing work that is improving modularization. It is required to be doing work to deliver user value all the time -- and to take advantage of every opportunity to modularize in order to deliver user value.

Letting modularization become an end in itself is sometimes tempting, but in the end it was not the right path forward.

02 September 2009

Blog by writing post titles

I got sick of trying to think of profound things to say here.

So I just started writing post titles until I found a topic I wanted to write about. Maybe that is what I can do the next time I'm feeling like a post is over-due.

15 August 2009

Site Design Tips

In thinking about what next for fshelper.org, I've been on the lookout for good site design ideas. Here is the most recent article by Jakob Nielsen who has cared about this for much longer than I have.

Also, I found the Daleri Structure site design by Andreas Viklund to be very appealing for how I want the site to feel: informative, down-to-earth practical, and well-linked.

14 July 2009

Refreshing Take on OS Upgrade

I'm typically a laggard when it comes to upgrading my OS. I like for things to be stable -- I don't like to waste time fiddling with things unrelated to what I've got to get done for whatever project I'm working on.

So now in July, I'm upgrading to Ubuntu 9.04 (released in April). To state how much of a laggard I usually am, I think this is the most up-to-date I've ever been on an OS upgrade.

I decided to use the standard "upgrade to 9.04" feature exposed in "Update Manager". So when I saw a screen like this:

then I felt happy. I felt like I knew what was going to happen, and that I was in control of the process to some degree. This freed my mind up and let me think about what I could do to minimize any upgrade risk.

In the past, because of the helpless feeling of being out of control of what's going to happen and what I can expect, I've avoided upgrading until the pain became great enough to justify a full backup/reorg of all my files that I for sure wanted to keep.

27 June 2009

Heroes and Monoliths

I'm sure it's no surprise to anyone, but huge, monolithic software is extremely hard to ship.

It used to be that when I encountered a particularly complex software problem, I would work at mastering the complexity until (by hook or by crook) I figured it out. Then I would proclaim myself victor. It was often the case that people were waiting for me to figure out the hard problem, and would congratulate me on a successful resolution.

However, those congratulations feel empty to me now. I think I can finally appreciate what Dijkstra meant in The Humble Programmer (and here), argument #6:
... the only problems we can really solve in a satisfactory manner are those that finally admit a nicely factored solution. ... By the time that we are sufficiently modest to try factored solutions only, because the other efforts escape our intellectual grip, we shall do our utmost best to avoid all those interfaces impairing our ability to factor the system in a helpful way. And I cannot but expect that this will repeatedly lead to the discovery that an initially untractable problem can be factored after all.
Now I remain unsatisfied until I've solved that problem at hand "in a satisfactory manner", i.e. in a manner that is tractable to read, understand, and change in the future. Not just for me, but for any other competent software developer that comes behind me.

I think this links to the IEEE Code of Ethics, items #5 and #6, in which computing professionals agree:
5. to improve the understanding of technology, its appropriate application, and potential consequences;

6. to maintain and improve our technical competence and to undertake technological tasks for others only if qualified by training or experience, or after full disclosure of pertinent limitations;
While I was writing this, someone came over and asked me a question about complex software I wrote part of, and I helped him. And I laughed at myself for being unsatisfied about helping him. :)

15 June 2009

Vote with your money

There are prediction markets that people have set up. But those feel like a gimmick to me.

I want a place where I can say:
If you can provide a product that meets the following specifications, at the following price, I obligate myself to buy that product at that price.
I want to be able to state conditions like:
  • all software in this product conforms to an OSDL-approved license
  • the model of car you build has received at least 3/5 rating by Consumer Reports in such-and-such a category
  • this cell phone comes with a data-only plan
  • the out-of-print book that you publish must be hardback
  • the eBook reader you create must be able to display scaled PDFs that can be zoomed and panned
I want to be able to have a publicly-viewable, prioritized wish list. I want to put things on my wish list that do not yet exist, but that would be very useful to me. I want to be able to say "Me too" to an item on my friend's list and place it on my list at the appropriate priority. And I want to be able to set an annual spending cap, beyond which I am not obligated to purchase anything further.

And I want vendors to be able to query this site for what kind of demand there is for certain kinds of products. And I want vendors to be able to say: "OK, you said you'd buy it, I'm producing it, and if I give it to you in 6 months, you're committed to buy it." The user who posted his/her desired product can specify how much lead time would be acceptable. For instance, I could say: "I don't want you to require any more than 3 months production time for this item."

And it is a binding contract at that point, as long as the item ends up meeting the stated criteria when it is delivered. After the production process starts, a purchaser would be prevented from re-prioritizing or otherwise adjusting his list in such a way that pushes that item below the "obligitory purchase" spending cap line. And if a user defaults on the contract, then the details are publicly visible; likewise for a vendor.

My intent is to have a system that can create REAL demand for useful, not-yet-existing products without having a bunch of focus groups giving stupid requirements to business execs who work up some product that pins its users into a corner in one way or another.

06 May 2009

We thank Thee

It is common in my church to pray at the start of church meetings.

At one such meeting, I heard the following in the prayer:
We ask Thee to bless us with all other things that we stand in need of.
and I couldn't help but think how it would sound if someone prayed the following:
We thank Thee for all the things we've fogotten to, or that we didn't think to thank Thee for.
The second kind of prayer would be a more appropriate sentiment.

29 April 2009

5 Minute Send

There is something that happens in my mind when (and only when) I push "Send" on an email.

This mental process says "oh, I wonder how it will look when they get this email". So then I go back and look at the email from a completely different perspective -- from the perspective of the recipient. Everybody has a "missed attachment" feature or plugin. But nobody has a "missed idea" feature or a "missed stupidity" feature.

I wish all the email clients I use (gmail, outlook, other webmail) had the following two features:
  • "Send" (on a draft message) which means: send in 5 minutes
  • "Send All Pending" (on a global menu, not on a message)
Then I could get the benefit of focused authoring *and* self-review when I press "Send" and think "oh, what have I just done".

P.S. I really like blogger's "Post date and time" feature, and I use it all the time.

28 April 2009

Nice benefits of git

I use git now and it saved my bacon last week. Two things made a really hard, late-breaking change possible:
  1. Ease of integration enabled parallel development.
  2. Smallness of commits meant easy bug isolation.
Ease of Integegration => Parallel Development
Because of the ease of integrating small commits, my coding companion and I were able to develop two complementary sets of changes in parallel. This saved us quite a bit of time because we didn't have to wait for each other. The changes were developed as a last-minute response to a problem right before release, and without the ease of integration, it would have been hard to get the problem right on a compressed schedule.

Small Commits => Easy Bug Isolation
Also, because commits are so small (because it's easy to commit without any side-effects, because your repo is isolated from everything else), it was easy to isolate the exact changes that caused a subtle problem. I could have used bisect, but I had an inkling which change caused it, so that was faster. With svn or a mongo patch accrued over multiple changes, it would have been much more confusing and disorienting.

27 April 2009

Revert by Commit

Here is a sequence that happens to me regularly:
  1. Have a theory about how to fix something
  2. Try it out
  3. Find out it was a stupid theory
  4. Realize that there is a kernel of learning in what you tried
  5. Realize that you don't want to throw away that kernel of learning quite yet
  6. But want to get the bad changes out of the way
  7. But there are some good uncommitted changes you don't want to throw away
  8. Ack, what should I do now?
Here is a recipe that makes it really easy to save the bad experiment *and* get it out of the way in ONE step.
  • git checkout -b experiment-that-stunk
  • git add -i (pick the one or two files that contain the changes you want to get rid of)
  • git commit -m "tried to do something stupid, but it didn't work because ..."
  • git checkout stuff-i-was-working-on
Revert by commit. Now the changes are gone, saved, explained, pickled for fermentation for more ideas or conversation.

That is something that was NOT part of my experience before git.

Actually, I take that back, I did this once or twice before, but I wasted lots of time because I resisted give up on the experiment at the first sign of stupidity -- I subconciously knew it would be painful to save a whole patch, partially revert, hope I got it right, forget what I was doing, mess it up, revert all the way, re-apply the patch, repeat. So it took longer to abort the bad experiment because of the hesitation because of the perceived expense of partial revert.


I just realized this must be how "git stash" is implemented, and that it's definitely not a new idea.

01 April 2009

Software Architecture

This is what I hate about overreaching ideas and projects that accept them:

The whole "coming soon" idea is fine if it lasts for a couple days, but years and years?

Now I'm hanging my head in shame about www.sumsion.org. :)

31 March 2009

DRY plus Proper Responsibilities

It is not sufficient to blindly apply the DRY principle.

Where the single representation ends up living is just as important as avoiding the duplication in the first place.

Having the single, unambiguous, authoritative representation of a piece of knowledge living in a place that it doesn't belong is only slightly better than having it duplicated, once in a place where it seems to belong, and once in a place it does not.

So included in the task of reducing (or avoiding) duplication is the consideration of where to put the common code, *including* the task of inventing a new place if it doesn't seem to fit anywhere, now that it's being used from multiple places.

Usually, a new place is waiting to be invented -- that is why there was duplication in the first place -- the place didn't exist, or else it would have been fairly obvious where to put the logic and how to reuse it in the first place.

Perhaps that is part of what "authoritative" means.

20 March 2009

MountainWest RubyConf 2009

MountainWest RubyConf 2009 really helped me reorient in a very good way.

The emergent theme seemed to be "modularity".

Six main talks that were really enlightening:
Rack totally changed how I thought about building a webapp. Decomposing things into pieces that have the minimum to do with each other is possible in a really clean way with rack. And treating each of those pieces as a separate app, even, with its own release schedule, etc. -- that was a very refreshing view.

When I heard Jon say that the Perl script he had up on the screen was a monolith -- he was absolutely right. And I thought that the system *I* was working on was a monolith.

I was going to give a lightning talk about fshelper.org's link feature, but I wanted to rewrite it in broken-up rack pieces. I tried my best to do that between talks, but I didn't quite get the app to function as middleware, so I just gave up and listened.

19 March 2009

Tracking work to be done

My team had the task of porting code off of an old API onto a new (hopefully-mostly-equivalent) API.

Diomidis Spinellis wrote an article in IEEE Software titled "Start with the Most Difficult Part" (on his blog here, official reference here). Now seemed like a good time to apply the advice.

So I wanted to measure: What was the most difficult part. I wanted to know how many usages of the old API were present and where I should focus my efforts first.

So I wrote macker rules to detect usages to port. And I wrote a ruby script that drove the ant macker target and pulled the usage counts into a list of counts. Then I published the current counts and started a graph so we could get the usages to zero.

Here is the chart we ended up with:

This drove us to two useful behaviors:
  1. Getting our tests converted first, the most complex one first (exposed a lot of issues early).
  2. Giving our work a metrics-driven feel -- all we have to focus on is getting this to zero.
There were also two things I missed:
  1. The hardest task to port was one in which the new API didn't have anything near what we needed, and the usage measurement didn't catch that. ==> A quick pass over the tasks looking for especially risky ones (where the port would be really hard) would have exposed that earlier.
  2. Another team was doing other work that intersected with ours, and we were unaware of the impact of their efforts on what we were doing. ==> Being able to get their changes quickly and easily may have helped expose the problem earlier.
Now that we've started using git, I'm going to suggest that we do two things:
  • Get an SVN task branch so we can integrate as a team during the sprint
  • Get an SVN task branch right before merging to release and re-apply all our git patches onto that task branch so the merge is really smooth
Or maybe if we can get done in time, we could volunteer to do that for the team that has the sprint-long task (getting them on everyone else's changes with a minimum of svnmerge.py headache).

Past Bylines

Here are past bylines of this blog (in reverse chronological order):
  • [Mar. 2009] Alma 34:38: Live in thanksgiving daily, for the many mercies and blessings which he doth bestow upon you.
  • [Feb. 2009] 1 Tim 4:15: Meditate upon these things; give thyself wholly to them; that thy profiting may appear to all. (KJV) -- too overreaching

09 March 2009

Test code structure

There are a lot of ways to build an unmaintainable test suite. Jay addresses this topic straight on. The most important idea I got out of it is this: "If It Hurts, You're Doing It Wrong."

Now how to get from painful to joyful ... that is the question. Probably by just applying common sense and proper code structure to tests, not just production code.


I've done my share of painful, stupid things:
  • the monolithic build system that had super-ant-tasks with laser vision
  • the event subsystem that was really just JMS
  • the custom deploy system that really should have been one of rsync or rpm
  • the object persistence layer that was supposed to be super-generic, but was really tied super-close to the domain objects
  • ... I'm sure I could go on
The main thing I've learned is to work with the door open. And stay wide open to how to do things better and to always strive to see the things I'm missing.

28 February 2009

Links to New FamilySearch persons

Persons are referred to by id, but there is no support for hyperlinking to a certain person's page.

So I built an app that writes links to the current New FamilySearch website. Now persons can be linked to.

Here is a link to Samuel A. Shine.

27 February 2009

Helpful git aliases

I know you can go all out with git aliases. My favorite complex alias was "Use graphviz for display", documented here.

Here are ones that are useful for me:
b = branch
co = checkout
ci = commit -a -s
st = status
stat = status
l = !git-log --pretty --date=local --abbrev=10 --abbrev-commit | sed -e 's/^\\(commit .*\\)...$/\\1/' | less

k = !gitk

patch-apply = am

patch-gen = format-patch

Alias "b":
Alias "co":
Alias "ci":
Alias "st":
Alias "stat":
- so I don't have to type as much

Alias "l":
- so that I can see real dates, not UTC
- so that the commit numbers are clickable for copy/paste

Alias "k":
- in case I type "git k" instead of "gitk"

Alias "patch-apply":
Alias "patch-gen":
- in case I can't remember what "am" and "format-patch" mean (common occurrence)

13 February 2009

Opening Post

Some kind of inhibition? Write until it goes away.

Can't sleep? Write until you can.

Can't code? Write until you know what to do.

Stuck on a problem? Write until the problem is defined.

Unlike prior efforts at perfection, this blog is a work-in-progress. And unlike my prior standalone efforts, I hope to start deliberately existing in a much larger world.