A Desktop Version of a Classic Supercomputer?!?!?!

Normally I'd relegate posts  consisting of ranting about something I found on the internet to my tumblr account, but this was too good to post on a blog that no one reads.

*cough*NotThatThisIsMuchBetter*cough*

Excuse me... Anyways, while reading through posts in Google Reader, I saw what may be my favorite thing featured on Hack A Day to date.  Since this is coming via Hack A Day, many of you probably saw it already, but for those that haven't I present

A TINY CRAY-1:

That's right, Chris Fenton has created what amounts to a desktop-sized version of this classic supercomputing marvel.  It's 1/10 the size of the original, and sports an impressive 33MHz processor (the original only had 80MHz, so no scoffing!).  It's not the most useful of devices, but out of all the things I've ever said "I'd love to have one of those on my desk so people would ask what it is," this is probably the coolest.

It will be a while before I undertake this project, if I ever do, but it would definitely be worth it in my opinion. I mean, who hasn't wanted their own personal Cray? I know when I first heard about Cray supercomputers I immediately went to find out how much they cost so I could plan on buying one when I grew up and became rich (still waiting for this).

You can find more pictures, specs, and code (yes, code!) over at Chris Fenton's site.  Also, you can check out the Hack A Day article for some discussion in the comments.

Code Complexity Classes II: Revisited

Here's that C code I've been promising:


Compiled in 64-bit Ubuntu 10.4.

When you run the code, just as in Java, you can see the difference in time between the "row by row" and "column by column" implementations:

The row by row implementation ran in 2.64 seconds according to the timer built into the program, with the column by column example running in 3.67 seconds.

This is in a VM, so comparisons to the Java example done before should not be made, as those examples were tested in the host OS.  As always with things like this, the time is subject to your unique system and what is running at the time of execution.

What is important though, is to notice the difference in time between the two implementations, it is still there, and it is a noticeable amount, even with this relatively small amount of data.

Just for curiosity's sake, I decided to see how turning optimization flags on would affect the execution time:

With first level optimization, our row by row time dropped considerably to 2.32 seconds, and our column by column implementation time went up to 3.88 seconds, further increasing the gap.

With second level optimization, row by row dropped again to 2.27, and column by column further increased to 3.95. As you can see, this time the change was negligible.

With third level optimization, we saw a decrease about half that of first level optimization in the time for row by row copying to 2.16. This optimization also decreased the column by column time to below that of first level optimization, but higher than that of no optimization, putting it at 3.81 seconds.

I'm not going to go into what exactly these optimization flags do, as that's a topic for another post, and I'm not even sure what all is done myself at this point. I'm extremely new to C, so if you see anything incorrect in this post, feel free to point out my mistakes, and I'll be happy to correct them and give credit where credit is due.

Speaking of where credit is due, I must give some to my Assembly Language teacher, Dr. Chen, for providing the basic version of this code (I only added the timer and made some minor tweaks to suit my cosmetic style) and for providing the inspiration for these posts on complexity classes and how they stack up in the real world.  Perhaps in the future I'll revisit this topic more in depth, for now though, I am done with it.

The Java Heap

Note: Today's post is somewhat of a continuation of yesterday's. Just in case you didn't read yesterday's post, or would like to review, you can find it here.

While writing the code for yesterday's blog post, I ran into a runtime error that I hadn't personally run into before, though it is by no means an uncommon error.  The only reason I haven't run into it before is because I haven't before written programs in Java that had significant memory requirements.

Let's look at a section of code from yesterday:

static int size = 2048;
 
 static int A[][] = new int[size][size];
 static int B[][] = new int[size][size];

Find the complete code here

This section is creating and instantiating the int variable "size" then allocating memory for two two-dimensional arrays that contain size² elements.  Let's try adding another 1024 to the size variable and see what happens.

Click for full size

Now that's interesting.  The part we're interested in is:

java.lang.OutOfMemoryError: Java heap space at ArrayExample.

Apparently when allocating memory for the second array, we run into a problem, as there is not enough memory.  At first glance, this seems strange, as I have 8 gigabytes of ram in this machine, and even though Java can't use all of that because I'm running 32-bit Java, I did not think it would be a problem as the maximum memory still should not have come anywhere near this.

I suppose the question now is how much memory did the example attempt to use?  Just looking at the size of the arrays, and not taking into account overhead, the arrays come out to just over 73mb of space.  Considering 32-bit allows for ~4gb of space, this still seems odd that we ran into a problem.  In order to explain this, we have to focus on another word in the error message:

java.lang.OutOfMemoryError: Java heap space at ArrayExample.

What is the "heap space"?

According to JavaWorld:

The JVM's heap stores all objects created by an executing Java program.

What is the size of the heap?

By default, the size of the heap in Java 1.6 is 64mb.

Now we can see that we were, in fact, trying to use more memory than we had available.  I have a problem with this though. I want to use more memory than 64mb.  Why?  Honestly, because I can.

Is this even possible, or are we stuck with this limit?

Luckily for us, Java allows for us to bypass this limit at runtime. By executing the java command with the argument -Xmx and choosing a new maximum heap size, we can set a larger heap size for the program's use.

Example:

     java -Xmx128m ArrayExample


The number in the example is our desired maximum heap size.  The "m" after the number designates that we mean megabytes, you can also use "g" for gigabytes, etc.

Let's try it, using the same program as before:

Click for full size

Ah, now it runs correctly, and we can run our test from yesterday with even larger numbers, allowing us to see the difference between the two array copying implementations more clearly.  You can go even larger than I did here.

Just as an example, here's a run with arrays containing 10240²  (5x as many as yesterday's) elements:

Click for full size

Now we're getting somewhere.

Another problem you may run into is trying to allocate more memory than you have physically in your machine, or just more than allowed.  Trying to set the maximum heap to 4g gives an error that it is an invalid maximum heap size.  Yet another error you may run into is the JVM not being able to find enough contiguous memory.  The heap only allows you to use memory that is in one big block, so just because you have two gigabytes unused on your machine doesn't mean you'll be able to use it (trust me, I tried).

Have some fun with it, see what you can do.  I'd love to hear what the maximum size people can get to work on their machines is.

C code still to come, and I'll try to have some other examples besides just this array copying scenario.

Here's some links to more information on the heap and other options:

6 Common Errors in Setting Java Heap Size

Code source for finding size of ints on your machine to calculate space needed

Java's garbage-collected heap

Java memory tuning tips

Code Complexity Classes: Math vs. Reality

Had an enjoyable first day of my computer science classes yesterday.  In my Computer Organization and Assembly Language class, the teacher went over some examples of different C code, then we'd look at the Assembly for the code to see what was really going on.  One example he showed used large arrays.

In the example, the scenario he gave was that students are given a large array, and they need to write code to copy the array to another array.  In this scenario, two students have very similar implementations, but with one key difference.

Note: I redid the code in Java so I could add a few tweaks, C and other languages to come later.

Code for the first student:

static void copyij(){
  for( int i = 0; i < size; i++){
   for( int j = 0; j < size; j++){
    B[i][j] = A[i][j];
   }
  }
 }

Code for the second student:

static void copyji(){
  for( int j = 0; j < size; j++){
   for( int i = 0; i < size; i++){
    B[i][j] = A[i][j];
   }
  }
 }

Just in case you missed it, or didn't feel like looking at the code, the difference is in the order they copied the elements of the array. Both students' code will compile and run without error. The first student copied all elements row by row, as opposed to the second student who went column by column.

Looking just at the complexity class of each piece of code, we see that both have a complexity of O(n²). Since this is often how we measure the efficiency of code, most people wouldn't look into it any further. The real world doesn't always work out so perfectly though. Running both segments of code and logging the time of each copy shows that the second student's code runs significantly slower than the first.

How much slower? Over 10 runs, the first student's code averaged 16ms, whereas the second student's code averaged 71ms. On my netbook, the difference was even more profound, with the first code being, on average, ~10 times faster than the second.

These times are in milliseconds though, and are so small as to be negligible. Without running benchmarks like this, you wouldn't even notice this in your program. What happens when your array has 10 times the amount of data as the one used here? Well, wait until tomorrow where we cover the same topic using C code and you can find out.

Edit: forgot to add the complete code so you can run it for yourself.  The complete code can be found here.  Happy coding!

Big Development in Computer Science

Since I claim that this blog has computer science as one of the topics I discuss here, I feel I'd be remiss to not discuss a very recent and interesting development in the computer science world.

The are several problems in computer science that people are trying to solve all the time, many of these problems tie into mathematics or other areas.  Wikipedia has a good list of currently unsolved problems, with links to descriptions and more information for those interested.

The problem of interest to this blog post is the first such problem mentioned on that page: P = NP?

AOL News has an article that I feel explains the problem much better than the Wikipedia page does for people who are not as fluent in their math as I'd like to be (you can find the complete article here).  Quoted from said article:

Roughly, the mathematical problem asks if "questions exist whose answer can be quickly checked, but which require an impossibly long time to solve by any direct procedure," according to the Clay Mathematics Institute.

This may not adequately describe the problem to you, but if you're interested in more information on the problem itself, I direct you again to the article and also to the Wikipedia page on P versus NP.

My reason for not explaining the problem better here is twofold.  First, I do not believe I can explain it better than those sites do. Second, my purpose for writing this is not to explain what the problem is, but to share a bit of news on the problem.

Now that we have that out of the way, on to the news!

On August 6th, e-mails were sent to several researchers from Dr. Vinay Deolalikar.  Dr. Deolalikar is the Principle Research Scientist at HP Labs.  In this e-mail, he explained to his fellow researchers that he has found a solution to this problem, and also shared with them his findings in a paper.

This paper was leaked onto the web, in pdf form (can be found here).  The 66 page document has been shared across the internet by now, but is not the end of the story.  Today, Dr. Deolalikar released a longer, updated form of the paper (102 pages, here).  He has also stated that the final form of the paper is still under work, and will be released shortly.

This may not sound like such a big deal to some of you, but I assure you, this may be the most important computer science problem currently, and if this answers the problem, it is an amazing piece of research.  The P vs. NP problem is one of the Millennium Problems and solving it has a prize of $1,000,000.  Yes, one million dollars to whoever solves this problem, or any of the other Millennium Problems, it is that big of a find.

Currently it can be assumed that hundreds, if not thousands, of mathematicians and computer scientists, professional and armchair alike, are poring over the paper looking to see if there are any issues with it and to determine if it is correct.  I for one find it way over my head, as I had to pull out a dictionary during the first paragraph of the abstract, but I still find it exciting that a solution may have been found.

Links

HP Labs: Vinay Deolalikar
AOL News: P=NP=WTF?: A Short Guide to Understanding Vinay Deolalikar's Mathematical Breakthrough
Greg and Kat's Blog: P ≠ NP
The P Versus NP Page
Wikipedia: P and NP
The Millennium Prize Problems

The Early, Leaked Version of the Paper (66 pages)
Updated Version of Paper (102 pages)