Ampelofilosofies

homeaboutrss
The mind is like a parachute. If it doesn't open, you're meat.

rake dynamic prerequisites

13 Jan 2012

There’s a behaviour in rake I keep stumbling on. It’s the inability to dynamically add prerequisite tasks on a task from one of it’s prerequisites.

I’ve used this gist a couple of times now:

I’m pretty sure the first time I actually stumbled on this was sometime in 2005.

Why?

The output from the rakefile above is

when, if the dependency tree was built dynamically, ‘c’ should pop up before ‘a’:

Why would you need this you will ask?

Well, it helps to do things like task :build => :determine_sources instead of task :build => determine_sources() and avoid the performance hit of scanning for dependencies before you actually call the task.

This improves rake responsiveness which has a positive effect on team moral (no kidding, this is important!)

The workaround is to calculate all dependencies for all tasks when the rakefile loads and it does not scale.

Case in point: Currently our rakefile builds a C system with a changing number of small applications. Manually adding tasks for every new application costs too much so it is done automatically. Because no dynamic prerequisites are allowed, the method that creates the tasks also needs to scan the source and determine the complete dependency tree for every app task.

This adds unnecessary delays in rake’s startup. As we usually only call one build task we only need to calculate the dependencies for that one task (this is actually done but with a whole bunch of code that essentially controls task creation based on the task called and it is fragile and prevents us from calling multiple tasks in the same command line - basically it’s a hack)

Put your money…

The patch is actually trivial, a two line change in lib/rake/task.rb in the invoke_prerequisites method1.

Where’s the catch?

Well, this actually only works if a prereq is changing the prerequisite list it is a member of. Anything else is dependent on the sequence the tasks appear in the prerequisites list and has unexpected results.

There’s also the slight delay introduced by iterating over the prerequisites again, but this barely registers compared with the actual task invocation times.


1 And yet it took me almost 7 years to fix. Talk about procrastination…