The amazing adventures of Doug Hughes

Today I was doing a quick code review on a pull request and noticed that all of our test suite had failed CI. This was surprising since the change made was so small and looked so reasonable.

This is (more or less) what the method had been:

def query
  @query ||= begin
    query = SomeModel.where(...)
    query = query.addressed if addressed
    query
  end
end

The details of addressed are irrelevant to this article, so don’t worry about them. I’m leaving it here to illustrate that the query variable may have been updated under some circumstances.

The method had been updated like this in the PR:

def query
  @query ||= begin
    query = SomeModel.where(...)
    query = query.addressed if addressed
    return query.to_a
  end
end

So, all that happened was that the last line of the begin block had been changed from query to return query.to_a. It’s not important to get into the reasons why the query was being turned into an array, it just was.

However, this caused all of our tests to fail with a “void value expression (SyntaxError)”.

At first I didn’t see what was going on, but, after some research, I figured it out. Long story short, this would work just fine without the return keyword.

It seems natural to think of a begin block as an inline method that has its own return value, but, in reality, it’s an expression that evaluates to a value. There’s a subtle difference. Consider this expression:

5 + 5

This expression evaluates to 10. It’s not a method that returns a value. If you run that in a Ruby console you’ll see as much.

This is not a valid expression in Ruby, outside the context of a method:

return 5 + 5

Running this in a console gives you an “unexpected return” error.

Next, let’s consider this method:

def foo
	5 + 5
end

This method, when evaluated, returns the result of evaluating the expression 5 + 5. The reason it returns this is because Ruby assumes that the last expression evaluated in any method is its return value, unless something else is returned first.

The above is exactly the same as this next example, we’re just being explicit now:

def foo
	return 5 + 5
end

It’s important to know that the return keyword immediately terminates execution of the method and returns the specified value (or nil). Here’s an example with an early return:

def foo
	return 75
	5 + 5
end

As you might expect, this returns 75 due to the early return terminating execution and returning the specified value.

Now let’s consider variable assignments:

def foo
	a = 5 + 5
	return a
end

Running this method returns 10. Keep in mind that the return statement is immediately terminating the method and returning the value of a.

Here’s another variable assignment example:

def foo(bool)
	if bool
		a = 5 + 5
	else
		a = 75
	end
	return a
end

In this example we’re setting a to 10 or 75 depending on a conditional statement and then returning a. This is one way to write this logic. We could also have written it this way:

def foo(bool)
	a = if bool
		5 + 5
	else
		75
	end
	return a
end

This works because if statements in Ruby (like case statements, blocks, etc) evaluate to a value. It’s tempting to think of them as returning a value, but that’s not what’s going on. If they returned, the foo method would immediately terminate and return the value of the statement. To put it another way, a is set to the result of evaluating the entire conditional statement.

But what about this example?

def foo(bool)
	a = if bool
		5 + 5
	else
		return 75
	end
	return a
end

If you try to paste this into a Ruby console you’ll get a “void value expression” error. But why? Isn’t it effectively the same as the example above? Nope! Ruby sees the a = and knows it’s expected to assign something to a. The “something” being assigned is the result of evaluating the expression to the right of the = operator.

It’s pretty clear what this should be if bool is true, but what if it’s false? Our instinct might be to assume that the if statement returns a value, but, as we determined above, that’s not how return works.

Effectively, Ruby sees that it could be in the middle of setting a and be interrupted to return 75 from foo, effectively leaving a abandoned. That is, the expression to the right of the = operator either evaluates to a value or not at all (IE: a void value). Ruby doesn’t like this and raises a “void value expression” error.

So, looking back at the query method above, it’s trying to set @query to the result of evaluating the begin ... end expression. But, before it can do so, it encounters a return statement that wants to immediately exit the query method, which Ruby dislikes.

The solution to this problem is simple: Just remove the return keyword. Since Ruby assumes that a statement’s value is the last line of code evaluated, Ruby will set @query to the result of evaluating query.to_a.

I was challenged recently to write a Java method that could validate whether a provided number was “perfect” or not. A perfect number is one where the sum of its divisors (excluding itself) add up to the number. For example, 28 is a perfect number because it’s divisors are: 1, 2, 4, 7, and 14. If you add these up you get 28, the original number.

However, consider 16. It’s divisors are: 1, 2, 4, 4, and 8. If you add these up you get 19. Therefore, 16 is not perfect.

This calculation was to be done via a static method that accepted an argument num. The method was to return a boolean indicating whether the number provided was perfect or not.

My first attempt was naive. I stared by using a for loop to loop backwards from one less than the number being tested to 1. I tested each step to determine if the number was divisible by the step. If so, I collected both numbers into a tuple which I collected into a HashSet to remove duplicates. I then iterated over the set and summed the values in the tuple and added 1. It worked, but was considerably inefficient.

It was pointed out to me that, because I was looping backwards, I was doing two times the amount of work needed. If I looped forward to half the number being tested I could simply create a sum of the valid divisors. Understanding this, I decided to try a few other approaches…

First, I wanted to see if I could make a one-line solution to this puzzle. In particular, I wanted to experiment with streams of numbers in Java. My first attempt used a stream of integers. I reduced this to those which were valid divisors of the number being tested and summed the resulting set. If this was equal to the number, the validation passed. If not, it didn’t. This is the code:

public static boolean reduceIsPerfect(long num) {
  return LongStream.rangeClosed(2, num / 2)
    .reduce(1, (sum, test) -> num % test == 0 ? sum + test : sum) == num;
}

Mission accomplished! It’s one (wrapped) line of code. However, this led me to wonder if I could parallelize this. So, my next test was to make a parallel stream of numbers that I then filtered to those that cleanly divided the number being tested. I then summed these, checked for equality to the original number, and returned that.

public static boolean parallelFilteredIsPerfect(long num) {
  return LongStream.rangeClosed(1, num / 2)
    // make this a parallel stream so we can find the whole divisors more quickly (ideally)
    .parallel()
    // filter out any non-whole divisors
    .filter(test -> num % test == 0)
    .sum() == num;
}

This too worked, though arguably it’s more than one line of code.

This success made me wonder how much (if at all) more efficient this method was than my first attempt. I wrote tests for the first six perfect numbers and found that the sixth one, 8,589,869,056, took quite a while to process. My first attempt with reduceIsPerfect() took about 45 secs to validate. My second attempt, parallelFilteredIsPerfect() took 23 seconds, so it was about two times faster. But, this left a bad taste in my mouth. 23 seconds is a really long time.

I tried a few other approaches, but found that, in general, most of my attempts tested in the 45 to 50 second range, which was disappointing.

So, next, I did what any good programmer would do, I turned to Google. I reasoned that the part of the process that takes the most time is determining if a number is a divisor or not. Google led me to a more efficient algorithm.

The way I was testing for perfection was by looping from 1 to half the number being tested. At each step I tested to see if the number could be divided cleanly. For example, with 28 I was doing this:

Start with a sum of 0. Loop from 1 to 14 (inclusive).

  1. 28 % 1. Yes! Add 1 to sum. sum is now 1.
  2. 28 % 2. Yes! Add 2 to sum. sum is now 3.
  3. 28 % 3. No!
  4. 28 % 4. Yes! Add 4 to sum. sum is now 7.
  5. 28 % 5. No!
  6. 28 % 6. No!
  7. 28 % 7. Yes! Add 7 to sum. sum is now 14.
  8. 28 % 8. No!
  9. 28 % 9. No!
  10. 28 % 10. No!
  11. 28 % 11. No!
  12. 28 % 12. No!
  13. 28 % 13. No!
  14. 28 % 14. Yes! Add 14 to sum. sum is now 28.

So, 28 is perfect.

But, Google pointed out to me that half of this information is already known by the time I calculate it. The divisors are 1, 2, 4, 7, 14, 28. As soon as I discover 1 I can quickly determine that it’s paired with 28. 2 is paired with 14, and 4 with 7. I don’t need to continue working at all once I’ve gone to 4! This is less than the square root of 28, 5.291502622129181.

So, if I loop from 1 to the square root of the number being tested, I can just take the number being tested as well as it’s paired divisor at the same time. Add these to the sum and I’m in business:

Start with a sum of 1. Loop from 2 to the square root of 28 (rounded down to 5) (inclusive).

  1. 28 % 2. Yes! Add 2 and 14 to sum. sum is now 17.
  2. 28 % 3. No!
  3. 28 % 4. Yes! Add 4 and 7 to sum. sum is now 28.
  4. 28 % 5. No!

And, again, I’ve confirmed that 28 is perfect.

With this in mind, I rewrote my original one line solution like this:

public static boolean reduceSqrtIsPerfect(long num) {
  return LongStream.rangeClosed(2, (long) Math.sqrt(num))
    .reduce(1, (sum, test) -> num % test == 0 ? sum + test + (num / test) : sum) == num;
}

And viola, my test of 8,589,869,056 went from taking 23 seconds to 3 milliseconds!! After all, rather than looping from 1 to 4294934528 and testing each step along the way, I am now looping from 2 to 92681! This is significantly less work.

And so, I’m fairly happy with the result. That, and, after all that, it’s just one line of code.

The full code and tests for this project have been posted to Github here.

What is Java?

This is an article I wrote for my Java class at The Iron Yard. It’s being published here with permission.


Writing Software in Java

To write a program is to “speak” in a language a computer can understand. We don’t actually “speak” to a computer, but we can write messages that it can understand. These words aren’t in a natural language, though. Instead, they’re in a programming language.

My class at The Iron Yard is primarily about the programming language Java. Java is just one of hundreds of programming languages, but it has the distinction of being the most popular.

To a fresh eye, programming languages often look like gibberish. For example:

int total = scores.stream().mapToInt(Scoreable::getScore).sum();
return total / scores.size();

This Java code could be translated into english as something like, “Calculate the average score from a set of scores.” This example is intended to illustrate a point: Java isn’t english.

Read the rest of this entry »

Getting Help

This is an article I wrote for my Java class at The Iron Yard. It’s being published here with permission.


Getting help

Often you will run into problems where your debugging strategies just don’t give you anything useful to work with. In these cases, you have to turn to other resources.

Craft a good question

The first step in getting help is to figure out the correct question to ask. Vague questions don’t lead to specific answers.

Instead of saying “when I click this button, it doesn’t work,” explain what you hoped it would do and what it actually does. To do this, you have to understand what it is you’re actually trying to accomplish. It might be a surprise to realize that sometimes you don’t know this. Read the rest of this entry »

This is an article I wrote for my Java class at The Iron Yard. It’s being published here with permission.


If debugging is the process of removing bugs from code, then what is programming?

Undoubtably you’ve already run into a few bugs in your code. Figuring out the cause of these problems can be a frustrating and tedious experience. But, it can also be incredibly rewarding. There’s nothing quite like the feeling of finally squashing a particularly challenging bug.

As a professional programmer, you’ll be spending most of your time debugging. It’s virtually impossible to write code that works perfectly the first time. In fact, code that appears to work correctly on the first test will probably start to set off your mental alarm bells. The worry being, “what am I missing?!” Read the rest of this entry »

I heard that someone going by the name of Doug Hughes landed a gyrocopter on the lawn of the capitol building in DC. Sorry, but I am not that Doug Hughes.

It’s been fun and amusing to receive calls of support for the other Doug Hughes. I’ve received half a dozen calls for him. Most have been people gushingly praising him. Some have offered legal and financial support. Others have asked for interviews. Perhaps I should have accepted the cash and given the interviews?

Anyhow, I do support the other Doug’s ideas. We need to get the influence of cash out of our government. Corporations are not people, cash is not free speech, etc, etc.

I recently got ahold of the other Doug’s email address. I’m going to see if he minds me forwarding on these messages, but for now it’s safe to assume that I’m not able to forward these messages.

Ye Olde GyrocopterI’ve Always Wanted a Gyrocopter!

I hear there may be one up for auction in DC soon.

Hmm…..

Tag Cloud