Diff two directories

find . -name '.svn' -prune -o -type f -exec diff {} /path/to/other/{} \;
0 comments »

Testing asynchronous functions with scriptaculous' unittest.js

unittest.js is kind of rudimentary, but nonetheless handy. The biggest problem I've run into is its shoddy support for asynchronous testing, which amounts to doing something like this:

testAsynchronousFunction: function(){ with(this){
  var output = null;
  asyncFunc(function(transport){output = transport});
  wait(10000, function(){
   assertNotNull(output);
  });
}}

The problem being that sometimes asyncFunc will take 7 seconds to complete, in which case your test waits 3 seconds too long, and sometimes it takes 12 seconds to complete, in which case your test fails.

There's two ways to get around this if your asynchronous function takes a callback. If the function is for setup, you can run the test creation in the callback, so the test doesn't run until everything's setup:

asyncFunc(function(transport){
  new Test.Unit.Runner({
    ...
  }, {testlog: 'testlog'});
});

However, that won't work out of the box because unittest.js uses Event.observe(window, 'load'...) to bring up the test, so if the callback is executed after window load, your SOL. Since I'm using Lowpro, it was easy to circumvent that by using Mr. Webb's slightly different Event.onReady, which is a hook to Prototype's new dom:loaded, with the difference that it will execute immediately if dom:loaded has already been fired.

unittest.js:

Test.Unit.Runner = Class.create({
 initialize: function(testcases) {
   var options = this.options = Object.extend({
     testLog: 'testlog'
   }, arguments[1] || {});

   options.resultsURL = this.queryParams.resultsURL;
   options.testLog = $(options.testLog);

   this.tests = this.getTests(testcases);
   this.currentTest = 0;
   this.logger = new Test.Unit.Logger(options.testLog);
   Event.onReady(function() { // Ensures the event fires even if onReady already happened. Requires lowpro.
     this.runTests.bind(this).delay(0.1);
   }.bind(this));
 },

More often than not, that doesn't cut it. So I patched unittest.js to support asynchronous test with wait(null, ...) and continueTest(). Here's how it works, the patch is below. It requires Prototype 1.6.

testAsynchronousFunction: function(){ with(this){
  var output = null;
  asyncFunc(function(transport){output = transport; continueTest()});
  wait(null, function(){
   assertNotNull(output);
  });
}}

Here's the patch:

Index: unittest.js
===================================================================
--- unittest.js (revision 6610)
+++ unittest.js (working copy)
@@ -205,16 +205,21 @@
     }
   },
   
+  waiting: 0,
   runTests: function() {
     var test = this.tests[this.currentTest], actions;
     
     if (!test) return this.finish();
     if (!test.isWaiting) this.logger.start(test.name);
     test.run();
-    if(test.isWaiting) {
+    if(test.isWaiting && this.timeToWait) {
       this.logger.message("Waiting for " + test.timeToWait + "ms");
       setTimeout(this.runTests.bind(this), test.timeToWait || 1000);
       return;
+    } else if (test.isWaiting){
+      this.logger.message("Waiting for continue event.");
+      document.observe('unittest:continue:' + ++this.waiting, this.runTests.bind(this))
+      return;
     }
     
     this.logger.finish(test.status(), test.summary());
@@ -466,6 +471,11 @@
     this.test = nextPart;
     this.timeToWait = time;
   },
+
+  waiting: 0,
+  continueTest: function(){
+    document.fire('unittest:continue:' + ++this.waiting);
+  },
   
   run: function(rethrow) {
     try {
0 comments »

Stitch pngs into a pdf with ImageMagick

Damn, this saves me lots of time. No more manually stitching them together with oo.org.

$ convert *.png big_ol.pdf
0 comments »

Overcoming SVN database corruption

I think one of my find and replace commands went astray and screwed up svn's files, making it impossible to commit. One way to overcome this is to delete your working copy and checkout anew. But I had lots of local modifications.

Transmitting file data ....................svn: Commit failed (details follow):
svn: Checksum mismatch for '/home/ian/Work/main/trunk/.svn/text-base/foo.js.svn-base'; expected 'aa8e8c396d0f16dec2d807ec5ac2623f', actual: '51114bc6389fa5e9748565443c7b625d'

This svn-base file is just a copy of the latest revision from the repo. So you can get the prisitine file from svn:

$ wget https://svn.example.com/main/trunk/foo.js -o main/trunk/.svn/text-base/foo.js.svn-base --user=ian --password=god

You might also just edit the file to fix it, eg. if svn isn't easily available with wget. To check if the file is acceptably fixed:

$ md5sum main/trunk/.svn/text-base/foo.js.svn-base
aa8e8c396d0f16dec2d807ec5ac2623f  /home/ian/Work/main/trunk/.svn/text-base/foo.js.svn-base

Which should match with the hash svn was expecting.

0 comments »

quickly and dirtily discover the slowest actions in your rails app

$ cd log
$ grep "Completed in" production.log | sed 's/Completed in \([^ ]*\).*\(http.*\)\]/\1 \2/' | sort -n
0 comments »

has_one_paranoid

I ran in to a problem with AR::B#find(:include) and acts_as_paranoid, well described on the ruby mailing list.

So I whipped up this thing, which seems to work for me:

1
2
3
4
5
6
7
8
9
10
11
12
module ActiveRecord
 class Base
   def self.has_one_paranoid(*args)
     ref = create_has_one_reflection *args
     cond = args.last[:conditions]
     cond = cond.blank? ? '' : cond + ' AND'
     cond << " (#{ref.table_name}.deleted_at IS NULL OR #{ref.table_name}.deleted_at > now())"
     args.last[:conditions] = cond
     has_one *args
   end
 end
end
0 comments »

Userspace bandwidth limiter

I've been looking for a mechanism to limit the bandwidth of apps for years. Finally, someone at Ars Technica posted about the aptly named Trickle, which is exactly what I was looking for.
0 comments »

zlib and ruby

Every so often I find myself trying to compile ruby from scratch on Ubuntu. This brings up problems with zlib. I've dealt with it in as many ways as times I've compiled it, but I just found the nicest solution yet tucked away on the Ubuntu forum.

After you run ./configure, vi ext/Setup and uncomment the zlib line. Then make. Jeeeeez.

0 comments »

Starting BackgrounDRb from Capistrano

BackgrounDRb takes a few seconds to fork a new process, so if your shell exits before the fork (eg. if you're using Capistrano), the terminal will send it a stop signal, killing it. So, you just need to prefix your BDRb start call with nohup and redirect the output to /dev/null. So...
sudo -u nobody nice script/backgroundrb start
becomes...
sudo -u nobody nohup nice script/backgroundrb start > /dev/null
2 comments »

Flush Memcached

Just started working with memcached. Wanted to do a simple thing: flush everything. It's not in the man pages, but after some googling found out that this is the ticket:
$ telnet localhost 11211
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
flush_all
OK
quit
Connection closed by foreign host.
$
1 comments »

What user is a Ruby process running as?

I have a script that shouldn't be run as anybody other than the "nobody" user if it's in production mode, since it will overwrite files with the wrong permissions otherwise. My colleague Ben Tucker alerted me to Process.uid, which allows me to do something like
$ /home/ian/sw/bin/ruby -e "puts Process.uid == `id -u nobody`"
false
$ sudo -u nobody /home/ian/sw/bin/ruby -e "puts Process.uid == `id -u nobody`"
true
The rest should be trivial.
0 comments »

whois dictionary search

I wanted to find available full-word domain names:
$ for w in $(cat /usr/share/dict/words | grep "us$" | sed "s/us$/.us/"); do echo $w; whois -H $w | head -n1;done | grep "Not found"
0 comments »

ExceptionNotification undocumented filtering

ExceptionNotification has an undocumented parameter filtering feature. Defining the callback filter_parameters(parameters) in a controller will cause ExceptionNotification to replace the printed parameters with the returned hash. Unfortunately it can't selectively filter the RAW_POST_DATA, so it just checks to see it would have been filtered, and replaces it wholesale with "[FILTERED]"

1
2
3
4
5
6
7
8
  public

  # Filter sensitive data.
  def filter_parameters(parameters)
    p = parameters.dup
    p['naughty_bits'] = ExceptionNotifierHelper::PARAM_FILTER_REPLACEMENT unless p['naughty_bits'].blank?
    p
  end
0 comments »

Model Registry

Sometimes I have a module that needs to find all models that mix it in. I was trying to find all the models, and then select ones that had a given property set. Not only is this inefficient, it's bug prone too. A better way is to create a registry with the module:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
module Wacko
  @@wackos = []
  mattr_reader :wackos
  def self.included(base)
    @@wackos.push(base)
    base.extend ClassMethods
  end

  def ClassMethods
    def happy?
      true
    end
  end
end

Then you can do this to make sure all wackos are happy:


Wacko.wackos.all?(&:happy?)

Yay, except that models mixing in Wacko won't be evaluated until they're called for, which could be never. So you need to load your models at boot. If you try to do it with require, you'll run into problems in development mode that have to do with cache_classes, and are best explained by reading dependecies.rb. Instead, use dependecies.rb's very own "require_or_load" which seems to play nicely with all environments. In config/environment.rb:


Dir.glob(File.join(File.join(RAILS_ROOT, 'app', 'models'), '**', '*.rb')).each{|m|require_or_load m}
0 comments »

Hide included methods

So, I needed some module methods in one of my controllers:

1
2
3
class FooController < ApplicationController::Base
  include MyModule
end

but the writer of MyModule left the methods public so that they'll show up in FooController.action_methods and be accessible with URLs :(

1
2
3
4
class FooController < ApplicationController::Base
  protected
  include MyModule
end

that doesn't work because the module redefines the default visibility. This is the ticket:

1
2
3
4
class FooController < ApplicationController::Base
  include MyModule
  protected(*MyModule.public_instance_methods(true))
end
0 comments »